I’ve noticed a weird thing about the discourse around this, that it can only be a marketing thing or a true belief, as if everyone working in AI has a monolithic opinion. The tweet kicking off this article has that assumption “it’s not a marketing thing, many people truly believe it”.

It’s not an either/or. Many sincerely held beliefs can be used by cynical actors in cynical ways. It can be both a cynical marketing ploy by the C-suite and a genuine fear.

Personally, based on my experience with AI, the only dangers with the current technology is a) massive loss of employment, b) deployment by humans to control critical infrastructure that LLMs shouldn’t be in control of. Both of those are real, society changing risks that framing the issue as “once we reach the singularity everyone could die” minimizes. Obviously their concerns are possible, but >10% is a random guess. The loss of jobs and how that affects an already K shaped economy is already happening, and nothing is being done for that

The internet, and to some extent our own brains, incentivize the extreme positions. Either it is mind-bendingly important and will change everything, and very soon (for good or for ill), or it is a total nothingburger and everyone who says otherwise has an agenda. Nobody wants to read about how AI will deepen long-standing class tensions or pose enormous new challenges for education or force regulators to rethink property taxation.

It all boils down to accountability. If you tell people there is a massive, unsolvable problem, then you don't need to talk about what you're doing to fix it. The labs have taken this approach by saying that they're willing to talk, at some point in the future, about maybe taking unspecified steps to slow down capabilities research, as long as everyone else agrees and it makes sense to the investors and it's not too cold in SF that morning. Likewise, if you tell people that AI is going to have minimal or no impact on the world, then there's nothing to mitigate. But if you tell people that AI is going to cause serious - but solvable - problems, they're going to want to hear solutions, and nobody wants to come up with any solutions.