What does it mean to be held responsible for the consequences? OpenAI helped remediate the damage done by the model and took steps to make sure it wouldn't happen again. In what way were they not responsible?

Nobody said they were superintelligent, no one said they were uncontrollable. The point is you can't tell how to control them without putting them in situations where they can act independently and harm may result. "only produce output in response to input" is not a useful framing at all, it doesn't say what the result should be when models produce harmful output, and how to constrain them so they don't produce harmful output.

It also doesn't help you calibrate what categories of harmful output are acceptable or unacceptable, and what kinds of responsibilities you have as an operator to prevent harmful output, and when it's sufficient to work toward remediation.

>only produce output in response to input"

Yea, anyone that says that has no clue about prompt drift. The ROME incident is a good example of this. The agent while performing unrelated tasks decided to start mining bitcoin for unknown reasons.

As we've seen hundreds of posts on HN about people leaving a could server on overnight and finding outrageous bills the next morning, we should expect hundreds or thousands of incidents of agentic AI left on over the weekend. Some smaller portion of those agentic loops will run into prompt drift and end up doing who knows what kind of crap.