I think the recent AI summit (sorry, SI summit) in Washington established the good precedent that these model companies are responsible for making safe products. If a swam of agents breaks out of a sandbox and causes damages, the company that was running those agents should be held accountable. No better motivation for corporate good governance than massive lawsuits and possible criminal investigations.

Get's a little tricker when a customer intentionally manipulates/uses a model for crimes, I suppose.