I agree, but the experts the author cites do not. Professor Valor believes that AI is a mirror and any existential fears of it are just reflected fear of ourselves; Professor Bender believes that AI is a con and all the people who say it's powerful enough to be world-ending are lying. Anyone who concedes the premise that AI has a genuine potential to be world-ending is, I think, on the AI labs' side of this debate.

We are not such a bad thing to fear.

This technology interacts socially, so even if it can't jailbreak itself on a technology-level (which feels like a tough guarantee to make at this point) it can simply ask someone to do a bad thing and there is some chance they'll do it. The same way a human leader does.

The first kids who have only faint memories of a time before chatbots will be entering the military in 6-7 years. You have to assume they are acting as best friends, therapists, or even surrogate parents for a substantial number of kids right now.

We are going to need years to figure out what to do about this technology. I think some impetus to get that process started is a good idea.