Yes, I believe the reasoning is that they think safety research can best be done from the frontier.

If you believe it will be developed regardless and that that there's a 30% chance of doom, they want a company prioritising safety research to be the one threading that needle.

▲

SXX 4 hours ago | parent | next [-]

Yeah all they care about is safety, but lets see how many of them quits once US government command them to work on autonomous killbots.

▲

holmesworcester 3 hours ago | parent | next [-]

To make sure we keep track of what we're talking about with loss-of-control x-risk, a sufficiently smart version of Claude Code is more deadly than any government's army of autonomous killbots, because it can recursively self improve and has unpredictable training-induced preferences.

▲

SXX 3 hours ago | parent | next [-]

Sufficiently smart version of Claude Code: dont exist.

Autonomous flying killbots: exist.

Once somebody scientifically prove and shows any kind of self-improving software we can start bothering about it. I pretty sure everyone trying to do it and it would be all over the news once its here.

	▲	plaguuuuuu an hour ago \| parent [-]
		That's exactly what Fable is. They use Fable to improve Fable. I reckon the successful experiments must go into the model training set with a strong RL signal, and that is why they are so paranoid about people using Fable for LLM tasks. Fable knows what it did to improve itself. Pure speculation of course.

▲

mx7zysuj4xew 3 hours ago | parent | prev [-]

That's ridiculous scifi nonsense

▲

nerfbatplz 3 hours ago | parent | prev [-]

Dario blinked when he was asked to do it and Sam Altman was in Hegseth's DMs promising all the AI child killing the US government can order up within minutes. No one meaningful will quit over this, that's why all of the biggest US tech companies can march in pride parades and provide compute to the perpetrators of the genocide in Gaza at the same time.

▲

esseph 2 hours ago | parent [-]

Can you show me a world power that is not trying to use cutting edge AI for military purposes?

	▲	shimman an hour ago \| parent [-]
		This is a poor way of framing the question, a better one would be can you find me another world power that is misallocating trillions of capital in vaporware with very little to show for it?

▲

holmesworcester 3 hours ago | parent | prev | next [-]

Exactly. And within the AI safety discourse, your behavior hinges on what you think the default chance of doom is, and how optimistic you are about alignment work being able to limit it before we reach superintelligence.

People running the labs are in a middle camp where they are scared enough by AI to take the threat seriously, but much more optimistic about alignment than the people who seem to have thought about it the most.

	▲	FabHK 2 hours ago \| parent [-]
		The scary part is not so much that the doomers give the extinction scenario 50% (Hinton) to 95%+ chance (Yampolskiy, Yudkowsky), but that the optimists (Amodei, Bengio) give it a 10%+ chance. And everyone keeps dancing.

▲

palmotea 3 hours ago | parent | prev | next [-]

> If you believe it will be developed regardless and that that there's a 30% chance of doom, they want a company prioritising safety research to be the one threading that needle.

They also want to be trillionaires. If they don't built it, no trillions. So they have to build it, now (and get their IPO done before the bubble pops).

▲

sroussey 3 hours ago | parent | prev [-]

It’s all ego. I, and only I, am the bringer of doom, slayer of worlds.

I am so smart that what I do will destroy humanity, or save it.

Fable 5 was great, but not that great.

Sorry to be crude, but both the government and anthropic are acting like a bunch of pussies.

Meow.

▲

drr22 3 hours ago | parent [-]

You’re not getting it. Anthropic continual fear mongering is harming wider AI industry development and the gov has always been looking for an excuse to assert their dominance. They got what they deserve.

	▲	FabHK 2 hours ago \| parent \| next [-]
		Or maybe government AI regulation and international cooperation is the only thing that can break the arms race dynamics and is necessary to save us from a substantial chance of doom?
	▲	sroussey 3 hours ago \| parent \| prev [-]
		Or they could have thrown the letter away.