That's exactly the problem? He's saying the culture at OpenAI needs to change.

Which is the wrong lesson. We need laws and consequences to force their hand. There is zero chance of the culture changing.

▲

criley2 an hour ago | parent | next [-]

America's geriatric lawmakers don't even use email. They're decades away from understanding AI. Any laws in America will be written by the industry itself. Generally speaking, that means regulatory capture and the entrenched big players shutting the door on any competition. Anthropic will help us get safety laws that, surprise surprise, only Anthropic models satisfy. And all those pesky Chinese models will definitely be banned first.

	▲	0xDEAFBEAD an hour ago \| parent \| next [-]
		I think you're being a little pessimistic. See these comments on a recent US senate hearing: >Not every senator asked good questions, but most of them did. All of them very clearly already knew plenty of details about the Hugging Face incident and multiple other incidents. Most of them had a clear understanding of terms like "misalignment", "recursive self-improvement", "chain of thought / chain of thought monitoring", etc., etc.!! >... >- It seemed pretty much obvious common sense to every senator there that what happened and was happening were not "mere industrial incidents" caused by humans making simple mistakes. They independently brought up how bad it would be for rogue AI agents to move laterally between data centers. >- They all seemed to basically take RSI quite seriously. Not necessarily to the extent of talking about xrisk, but certainly to the extent of discussing future models becoming much, much more capable, much, much less controllable, and causing much more damage or loss of life. >... >- Every single senator seemed to think it was obvious we needed both much harsher liability regimes for AI developers and also new legislation, both very quickly. This was the complete consensus; the difference basically being degree. https://thezvi.substack.com/p/the-ai-preference-cascade-reac... Note that harsher liability regimes, at least, will presumably not be good for industry profits, which complicates simple accounts of "regulatory capture" to say the least.
	▲	saghm an hour ago \| parent \| prev [-]
		So what, we just give up and try to beg our legally immune corporate overloads to put safety above profit, or give up because it's impossible for anything to improve here? If you want to do that, go ahead, but some of us still think it's worth it to at least try

▲

enraged_camel an hour ago | parent | prev [-]

Laws will come once an AI-equivalent of 9/11 happens. Like when rogue AI agents take down a power grid or shut down a major hospital network.