That still sounds like it would be the company’s fault. If I asked it to hack a nuclear reactor, maybe it would be different. I’m also thinking about instances where OpenAI’s own test models escaped their own sandboxes — I would expect them to be responsible for the damages they caused.

The correct analogy is playing Russian roulette. The company says "you can pull the trigger but sometimes a bullet will come out" (see: "an AI can make mistakes"). However, is the company allowed to sell such a dangerous device, under these terms?