Most feel-good pieces on AI make the same mistake, which is looking at where the ball is right now and not where it's going. If progress were to halt today, this article would be relevant and I'd upvote it.

But since the richest and least-trustworthy people on earth are on track to make an ultraintelligent thinking machine, the risks I'm worried about are: killing most jobs, concentrating most wealth, and putting ourselves at risk of literal extinction. All I could think when reading this was, "this person really did not think this all the way through." I'm sorry.

> Most feel-good pieces on AI make the same mistake, which is looking at where the ball is and not where it's going.

Humans have a bad track record predicting where a technology is going.

I do not know where AI heads in the next 5-25 years (I have my hypotheses, but they are likely off, too), but I see a useful tool that allows many people to experiment with and try things that they could not before and I do not see why should I subscribe to the doomer view.

As far as the moral character of the leaders: that's common, too. Many luminaries were, based on recollections, seen as pretty bad human beings and that didn't run off on their inventions. My 2c.

Wealth is already concentrated. They’re just coming for the top 10% that affect the software devs now since they’ve taken everything else. You talked to anyone with an “unskilled” job in the last few years?

The thing that strikes me about the 'paperless office' as it was hyped isn't how much paper use was reduced, but what was enabled or sped up by computer databases or the internet to work with a head office. Consider how restricted most mail order companies would be compared to the modern market if they could only communicate with their customers, suppliers, logistics and other partners via phone/fax/physical mail, and they either had one central records department or the difference in efficiencies handling a network of local branches/franchises.

That leads to pondering if we're actually seeing a similar shift today with AI.

> killing most jobs

I can think of a dozen jobs off the top of my head that aren’t even remotely at risk due to LLM-style AI. And if your reply is, it’s only a matter of time until humanoid robots replace all human jobs, you’re very divorced from reality. In hundreds of years, maybe. Not in your lifetime.

This tech will already have a massive effect on society, purely from being able to generate text and software code at will. What is the constant need to exaggerate this into a Sci-fi story?

They're going to do all those things with or without AI. We need to fight the end goals at a social level rather than the temporary means

> are on track to make an ultraintelligent thinking machine, the risks I'm worried about are: killing most jobs, concentrating most wealth, and putting ourselves at risk of literal extinction. All I could think when reading this was, "this person really did not think this all the way through." I'm sorry.

While two out of those three risks do not require anything "ultraintelligent", for killing most jobs it's worth remembering that their track is an appropriation of intelligence, and the speedups happen where it's easiest to measure more than where the tracks correspond to economic value.

Unfortunately, this has the opposite effect for safety, and can still do a lot of wealth concentration even if it doesn't end all jobs.

>But since the richest and least trustworthy people on earth are on track to

They are on track to kill themselves with their decisions.

Extinction will come not with a bang but with a whimper. The boundary of the burgeoning poor — already at two nines — will expand to include pretty much everyone, and that underclass will dominate society to the detriment of everything in the commons: culture, civility, affordable food, education, thinking.

Western culture devolves into poor, corrupt, violent, petty criminality. Welcome to Mega Cairo.

> Most feel-good pieces on AI make the same mistake, which is looking at where the ball is and not where it's going.

Or imagining it’s going to the most utopian place possible, completely ignoring all the history which indicates the contrary.