The degradation, polarization, and weaponization of or media landscape over the era of social media has left us utterly incapable of believing anything that anybody says.

Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic.

He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy. Please at least consider the possibility.

▲

youoy 9 hours ago | parent | next [-]

This sounds to me like a cry for help from someone thats held hostage.

His previous self is writting this from the possition of his current self who is too deep in the economic consequeces to be able to do anything meaningful other than write a consequenceless text. Any other action would now carry too much personal risk for him.

▲

0xDEAFBEAD 9 hours ago | parent [-]

What course of action would you suggest for Dario at this point?

▲

youoy 9 hours ago | parent [-]

For example I believe the world would be more secure with an open frontier AI lab, but the consequences of him doing that are too big for him.

▲

stratos123 8 hours ago | parent [-]

You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely unsafe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?

▲

youoy 7 hours ago | parent [-]

AI technology is now only audited by people who think like them. The ones that dont either dont join the company, or leave after a short stint as we have seen. That creates an information or feedback bubble which is not healthy nor productive.

▲

stratos123 7 hours ago | parent | next [-]

Okay, but suppose that Hypothetical Opensource Anthropic trains a model that turns out to be very dangerous, and releases it. Suppose that the public investigates and, not being limited by an information bubble, correctly notices that it's very dangerous. What then? The model's already released, there's effectively no way to prevent it from being used. Whatever the risks of its release were, they will now materialize, regardless of what the public wants. How is this better than the current world, either by Dario's values or by yours?

▲

youoy 6 hours ago | parent [-]

By the same logic, suppose that Dario/Altman/Jensen accumulate all capital because they are the only one that have access to AGI and end up controlling democracy, turning the world into technofeudalism or whatever you want to call it. How is that better than an opensource Anthropic. At least an open source anthropic would allow to join economic forces to try to combat that situation, for example. But there are other more clear, less distopian benefits.

▲

stratos123 5 hours ago | parent [-]

> By the same logic, suppose that Dario/Altman/Jensen accumulate all capital because they are the only one that have access to AGI and end up controlling democracy, turning the world into technofeudalism or whatever you want to call it. How is that better than an opensource Anthropic.

It'd be pretty bad but it's also a world where humanity lives on, which is better than a lot of other outcomes. A world where everyone has unrestricted access to AGI is like a world in which everyone has a tactical nuke in their pocket. If somehow AGI doesn't lead to x-risks, we will "merely" have to survive in a world where rogue agents can do whatever they want and defenders can only ever react. Technofeudalism would be bad, but this (technoanarchism?) is quite bad too.

However, neither does Dario seem to propose to become world dictator. Like, I'm sure he wouldn't say if he wanted it, but notice that he isn't particularly trying to aim for that outcome, either. The plan in this essay involves third-party oversight and federal control and global cooperation - IMO it's about as non-dystopian an outcome as we can hope for, if we build AGI at all.

	▲	youoy 4 hours ago \| parent [-]
		I agree. Dario is a very smart guy and he is probably aiming for the best solution for humanity that at the same time keeps VCs/whoever gives him money happy, and keeps him in control because he does not trust the rest (probably with good reasons). My point is more that he is aiming for a local optimum for society, but he could find a lower optimum if he was not trapped in the path that he chose early on. I dont buy the tactical nukes/end of the world narrative. Why is he not speaking about the loss of cognitive skills of the population for example? That is a more real danger that is starting to happen. There are articles already speaking about the use of LLMs as cognitive viruses. Why are his aligment teams not reasearching and publishing about this?

▲

0xDEAFBEAD 7 hours ago | parent | prev [-]

Why don't you create your own auditing org to fix this? Or at the very least, publish a detailed critique of what you believe existing auditing orgs are missing.

	▲	youoy 6 hours ago \| parent [-]
		Will they give my auditing org access to their proprietary confidential secrets? Or will that happen only if they know i agree enough with them so that i am not a risk?

▲

ThrowawayR2 9 hours ago | parent | prev | next [-]

> "Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy."

Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation.

If he believes what he's saying, then when Anthropic is sued for their AI harming another party, his statements are evidence that Anthropic knew _in advance_ that their AI safeguards were likely insufficient to keep their product from harming people. It would be a blatant admission that they were reckless and negligent. That other AI companies are doing the same would not mitigate that.

	▲	itajaja 8 hours ago \| parent \| next [-]
		> it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. not really. if you really want to go by definition in the book and apply it the CEO role, Amodei is CEO of an LTBT, so it's in the CEO's goal to benefit humanity. You can believe in his sincerity or not if you want, but using the CEO label as a definitional reason as to why he must lie is factually incorrect.
	▲	sailfast 8 hours ago \| parent \| prev \| next [-]
		Not sure you can judge a PBC under those same umbrella as a for profit company, and voting rights make a big difference in terms of responsibility. There are a number of ways to govern companies that can reduce the amount of cynicism and “shareholder value” issues - you just don’t see it very often.
	▲	stratos123 8 hours ago \| parent \| prev [-]
		> Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. This seems like useless pedantry. Suppose a person really does have belief A and expresses it for years, they become a CEO of a company, and keep saying A. Are they lying? Well, maybe their beliefs magically changed when they became a CEO, but the more likely explanation is that they think expressing their belief in A is more important than their company's interests.

▲

blfr 9 hours ago | parent | prev | next [-]

I have considered it, I've been hearing way more of it that I like and it's rationalist slop with little to no predictive power: https://foom.hyperplex.org/

	▲	kalkin 9 hours ago \| parent \| next [-]
		Hmm, I clicked that link and the first claimed bad prediction I see, from 1996, is "singularity 2035 (actually 2025)". Predicting 2025-2035 as, at least, the period when AI becomes a really big deal, seems pretty good, even if the jury's still out on "singularity". Broadly, the rationalists seem to have been pretty early to realizing LLMs were a big deal, and certainly seem to have had a much more accurate picture of how they'd develop than the people who denounce "TESCREAL" and talk about "stochastic parrots". It's fair (and IMHO correct) to ding rationalists for lots of things, but specifically poor prediction about AI seems like a bad one, insofar as anything has been tested so far.
	▲	0xDEAFBEAD 9 hours ago \| parent \| prev [-]
		Amodei and Yudkowsky are different people with different views. See e.g. https://intelligence.org/2014/01/13/miri-strategy-conversati...

▲

Avicebron 9 hours ago | parent | prev [-]

We've had enough utterances from "effective altruists" to know what they say and claim to believe is a cynical ploy.