It amazes me how pieces like this never stop to question whether these beliefs are genuine.

If I'm one of these researchers who's leaving their position, "I am so smart that I can make something world endingly dangerous, but also so principled that I'm willing to walk away from it" is a pretty slick bit of self promotion.

Even if I was making a career change for unrelated reasons, that's absolutely the story I'd be telling.

And no one wants to take action, they want others to. Anthropic CEO urges AI companies to slow model development amid fears over misuse [1]. If they truly believe that, they would announce "we are slowing model development."

https://www.reuters.com/business/anthropic-ceo-urges-ai-comp...

The other half of it is that it is in the incumbents interest to have these stories because they want the government to regulate ai into an oligopoly of incumbents so they can maximize benefit to themselves and stop new entrants. Ai benefits will be competed away to the consumer benefit justlike every other benefit if the government doesnt ruin it with regulation, entrenching the current incumbents and causing deadweight loss to society forever

Sounds like you built a position that's immune to evidence?

If researchers say AI is dangerous but keep their position, well they're obviously hyping up their boss and doing it for the money, otherwise they'd quit. If they say AI is dangerous and quit, they're self-promoting.

I have no idea what these researchers could do, if they were genuine, that you would accept as even weak evidence that they believed what they say.

I know one of these researchers personally (Joe Benton) and he's not the self-promoting type. I believe his concerns are genuine.