If Astra is open-weight, black hats can download it, remove guardrails, and go on an automated hacking spree.

jacobgold 4 hours ago | parent | next [-]

Just like they're already doing with frontier-level open weight models? Do you imagine that Russian and Chinese intelligence don't have copies of the Mythos/Fable weights?

The point is to slow progress of future models. Nothing can stop what has already been created.

	▲	0xDEAFBEAD an hour ago \| parent [-]
		With closed-weight, we can get the models out to defenders before they're in the hands of attackers. There are also other categories of risk such as biorisk. Open-weight means no guardrails. Don't get me wrong, I like your idea of deflating the AI bubble, I'm just skeptical that this is a great way to do it. The PauseAI people have some ideas: https://pauseai.info/proposal * International body which must approve new training runs * Ban training on copyrighted material * Hold AI model creators liable The essential benefit of your idea is making new training runs unprofitable. But I imagine there are other ways to accomplish that, e.g. by taxing AI companies to the point where they are just barely squeaking by financially.

▲

preisschild 2 hours ago | parent | prev [-]

So? If the models are open more people can use it to improve their security too.