Anthropic should provide a specific service where they attack a businesses infrastructure using this frontier model and then issue a report of all vulnerabilities found. I could imagine it would be quite lucrative.

Much better than hiding it away where it can't help anyone.

Hypothetically... I'm joey joe bob who happens to be maintainer of a top 10 npm package. My wife got cancer so I need money FAST. Unrelated: can you mythos my lib?

1 month later: whoopsie my lib got hacked and the hackers stole a bunch of stuff. Sorry guys.

It just wouldn't be good PR. And this is the best case scenario.