Make it easy for these companies by passing a law that any AI model a company offers to the public (non-government) has to be released as open weights.

This is how we deflate the bubble safely and completely, without having to introduce a government regulator that is likely to go either too far or not far enough.

This resolves the coordination problem among these potentially good actors.

▲

0xDEAFBEAD 4 hours ago | parent | next [-]

If Astra is open-weight, black hats can download it, remove guardrails, and go on an automated hacking spree.

▲

jacobgold 4 hours ago | parent | next [-]

Just like they're already doing with frontier-level open weight models? Do you imagine that Russian and Chinese intelligence don't have copies of the Mythos/Fable weights?

The point is to slow progress of future models. Nothing can stop what has already been created.

	▲	0xDEAFBEAD an hour ago \| parent [-]
		With closed-weight, we can get the models out to defenders before they're in the hands of attackers. There are also other categories of risk such as biorisk. Open-weight means no guardrails. Don't get me wrong, I like your idea of deflating the AI bubble, I'm just skeptical that this is a great way to do it. The PauseAI people have some ideas: https://pauseai.info/proposal * International body which must approve new training runs * Ban training on copyrighted material * Hold AI model creators liable The essential benefit of your idea is making new training runs unprofitable. But I imagine there are other ways to accomplish that, e.g. by taxing AI companies to the point where they are just barely squeaking by financially.

▲

preisschild 2 hours ago | parent | prev [-]

So? If the models are open more people can use it to improve their security too.

▲

blululu 4 hours ago | parent | prev [-]

What? This is a terrible policy. In addition to the obvious AI safety risks, this just amounts to a free R&D subsidy to Amazon for hosting models. It would remove any incentive to do any R&D without AWS level physical infrastructure.

	▲	jacobgold 4 hours ago \| parent [-]
		That's an understandable first thought, but consider what the goal is for this: slowing down future development. This can be achieved by dramatically reducing the market value of models.