You'd be surprised. I've worked at multiple startups where employees were given prod access with zero oversight on day one: AWS, sudo access, database passwords, everything. The one startup that didn't do that never launched. Occasionally there were accidents: wrong branch deployed, bulk updates to DNS taking down most of the site, etc.

Sure, so draw a different line - not all devs have access to withdraw cash from the corporate accounts, or to open the email of the CEO and board, etc. There are always lines of privilege drawn. The point isn't to quibble over where they are drawn, it is to point out that you need to do the same for LLMs. Don't trust them to behave. Enforce limits on their privileges.