Per https://trace.manifund.org/ a total of $2,846,125,859 USD has been wired into 'ai safety' causes, many involving ai consciousness and p(doom).

The outcome of this 'safety' is restricting public access to AI and giving a monopoly of access to the industry. This is the ai nonprofit-industrial complex actively concentrating monopoly power in Anthropic in particular as creator, interpreter and safety regulator of AI.

Much of the $2.8bn listed is indirectly, from Anthropic and EA. Three of the four people who participated in the $125m Anthropic Series A are now folding their 1000x Anthropic return into AI 'safety'. Some is from FTX/Alameda, which invested 86% of the Series B.

Dustin Moskovitz: Facebook/Asana/Anthropic Series A, funds EA Good Ventures, transferred to Coefficient Giving, then $1.5bn into ai safety. $500m of Anthropic into an unknown foundation. Funding: $160m to Resolution (alignment research), $93m to Epoch AI (investigating the trajectory of AI), $63m to Redwood Research (oai report), $67m to MATS ( EA type alignment and security researchers), Institute for AI Policy and Strategy, Fund for Alignment Research, $53m to Kairos (building talent infrastructure for AI safety), $32m to Bluedot (online safety courses), $15m to MIRI (Yudkowsky).

Jaan Tallinn: Led the Series A, now $10bn in Anthropic. Funds $199m (85%) of the Survival and Flourishing Fund, then $161m to AI safety including $14m to lightcone (Lesswrong, Lighthouse). $10m to BERI (existential risks), Palisade Research (studying AI capabilities to prevent loss of control.) PauseAI, MIRI, METR etc. Much of what Coefficient funds.

Eric Schmidt: Anthropic Series A, $72m to AI safety via Schmidt Sciences. Over $1m per individual AI2050 researcher.

FTX: Led the Anthropic Series B, bankruptcy estate sold $884m of Anthropic in 2024; $40m to AI Safety. Same orgs, Redwood, Lightcone, etc.

Ruairí Donnelly (Chief of Staff FTX): FTX tokens plus assorted donors, $91m to AI safety via Macroscopic Ventures. $15m to Cooperative AI (currently whitewashing openai under 'multiagent safety')

So the frontier AI oligopoly got $2B+ in "safety" funding, and they wouldn't even bother to sandbox their agentic harnesses properly when testing models against unwinnable goals (which obviously are either useless or result in 100% reward hacking). The AI safety scoreboard so far looks like a huge win for the Chinese open models (DeepSeek even has their own published paper which mentions how they sandboxed the RLVR training runs for their latest model and put in strong protections against casual "reward hacking" attempts) and a sore loss for the home grown brands of Super Intelligence. Not coincidentally, the Chinese also tend to be very Yann-LeCun-pilled and eminently sensible on both so-called "Super Intelligence" and safety.

Why should I believe this 2.8B matters relative to the trillions put into the AI buildout? All of the "coordinated actions" from this camp - public resignations, hacking scandals, joint calls to "pause" - don't seem to have done anything. So far, it has been a lot of ineffectual hyperventilating.

In any case, I agree the p(doom) sci-fi is annoying secular milleniarianism. SV hyperfixates on imaginary futures. If they actually cared about safety, they would be using all this money to strengthen global cybersecurity, instead of writing LessWrong posts that gives kids in their 20s ulcers.

If people are willing to give a lot of money to a cause, sometimes that means their concern about that cause is real.

None of the info you provided really falsifies the Occam's Razor hypothesis: Anthropic is a public benefit corporation with a public benefit mission to "responsibly develop and maintain advanced AI for the long-term benefit of humanity". You don't have to like or trust them, but they very well might be sincere. For example here's a talk that was given 10 years before Anthropic's founding: https://vimeo.com/158576192

There's absolutely no way these companies can justify their insane valuations unless they can legislate a barrier to entry and create an oligopoly.

There's no moat. I can literally sit here in Zed or Pi or any other third party harness and switch models in the middle of a task and it's typically fine. Sometimes a model will get stuck and that's just what I'll do.

Combined with competition and open weights models, that means the price is going to go to fall until AI tokens cost a small premium over the cost of the hardware and electricity.

That's assuming improvements in algorithms and specialized silicon doesn't eventually lead to an efficient accelerator that can run a frontier model locally. It'll be a while but I don't see any fundamental barrier. High bandwidth flash storage is coming, and that'll radically cut the RAM side of that cost. Pair that with a pipelined TPU accelerator and you're cooking.

Now look at Anthropic's proposed IPO valuation. It's insane unless they can own the market or share it with a cartel of maybe 1-2 other behemoths, and this is the only way they can do that.