> Yeah, this is anthropic advocating for a ban on open weight models.

This is an ungenerous take, and I think it's important to to recognize it's reasonable to support models that are both open and safe. How this would actually be achieved is unclear though. Dario is at least proposing a solution a solution, which is the model needs to pass safety testing. This is reasonable and I wouldn't conflate this with wanting to ban open weights.

I think the deeper problem might be though that once you have safe open-weight models, it will be much easier to make them unsafe. And to be specific, unsafe means proliferation of chemical, biological, radiological, and nuclear (CBRN) weapons knowledge and similar information.

▲

jjfoooo4 3 hours ago | parent | next [-]

> How this would actually be achieved is unclear though. Dario is at least proposing a solution a solution

How it would be achieved is a pretty important bit! One which Dario is not proposing any concrete solution for other thanks hand waves at some gov safety committee.

Would this restrict downloads of an open model, or publishing?

Say we ban domestic hosting un-approved open models. How does Dario propose to ban downloads from abroad? You can’t tell what an encrypted payload contains, do we need to restrict encryption?

▲

jsnell an hour ago | parent [-]

Isn't it up to the open model advocates and publishers to come up with the solutions for making them safe?

Like, there's three plausible arguments about safety of open models:

1. Any concerns are fake news. Open models will always be safe.

2. Safety is irrelevant. Open models should not be regulated even if they're unsafe.

3. Safety is a technical problem with technical solutions. People releasing open models should invent and implement such solutions.

I think option 1 is totally out of touch with reality.

Option 2 is at least self-consistent, it's the argument being made by people who will say that all regulation is always bad. It's also like the worst possible world from an x-risk perspective (but I realize that the average HN poster believes any x-risk concerns are just frontier lab marketing).

Option 3 is playing on hard mode compared to proprietary models, which can both implement additional safeguards out-of-model and prevent modifications of the model. But if the answer to it is "it's too hard, Anthropic needs to come up with the technical solution", then that's not exactly a ringing endorsement for the safety practices of the open model labs, right?

	▲	cogman10 an hour ago \| parent [-]
		In order for model safety regulation to be effective, you need everyone capable of producing models to sign on to that safety framework. That will never happen. As such, there is no "solution" here. The best most perfect regulation in the US won't prevent a malicious actor in the US from running a dangerous model. It's simply too easy to VPN to a country that doesn't care about AI safety and to run or download that model and run it in the US. There's no solution to this, which is why option 2 is the only option. The only thing safety regulations can possibly do is blunt the usage of "unsafe" models. And the primary people that will be blunted by it are people that do not and would not use these unsafe models in an unsafe fashion. It's not that I think regulation is always bad/wrong whatever, I'm no libertarian. But I also recognize when regulation is pointless. You can't regulate away forbidden knowledge, which is effectively what a dangerous model is.

▲

cogman10 3 hours ago | parent | prev | next [-]

> This is an ungenerous take

Why should I give a multi-billion dollar company advocating for new regulations in its industry a generous take?

I'd be similarly cynical if McDonald's proposed new health and safety regulations for restaurants.

	▲	0xDEAFBEAD 4 minutes ago \| parent \| next [-]
		>I'd be similarly cynical if McDonald's proposed new health and safety regulations for restaurants. "Because McDonald's wants food regulations, we can therefore conclude that all food regulations should be eliminated." Obviously this would be rather silly. It would be helpful to stop obsessing about McDonald's finances and simply discuss the best food regulation strategy. We just can't learn all that much about the optimal way to regulate food by making cynical proclamations about which food regulations will benefit the bottom line at McDonald's.
	▲	cloverich 3 hours ago \| parent \| prev [-]
		There is an important difference between criticism and cynicism.

▲

parineum 4 hours ago | parent | prev [-]

> This is an ungenerous take

I think that's well earned.