This was one of the rare critiques of AI doom that actually understands the case for it and presents them well, so I kept reading to see what its arguments for our safety against AI doom were. They were roughly:

1. It's hard to put a cat in a box despite us being smarter than a cat, so we're safe. (Counter: we're pretty good at putting cats in boxes when it matters.)

2. It was hard for Australia to kill Emus, so we're safe. (Counter: Australia could probably kill all Emus if it mattered enough, and we definitely accidentally kill off species when one of their inputs for life matters enough to us.)

3. Some smart humans get paralyzed by hedonism or existential angst instead of optimizing for arbitrary goals implied by their arbitrary value sets, so we're safe. (Counter: others overthrow the Czar, land rockets, etc.)

4. Modern AI is data-trained, so recursive improvement requires more data, so we're safe. (Counter: AI-crafted, synthetic data is a thing.)

5. We don't (yet) know how to improve our brains with brain surgery, so we're safe. (Counter: same as #4 above, which unlike us/evolution AI is being deliberately trained to understand and perform.)

6. Children take a long time to grow up, so we're safe. (Counter: the author's own "Premise 5: Computer-Like Time Scales", where they correctly note that computers can be arbitrarily faster than us.)

7. Individual smart humans on a desert island would be cooked, so we're safe. (Counter: nothing says the capability of a single AI must stop at that of an individual human, or that of a small group of smart humans; humans brains got dropped into a savannah and eventually they launch rockets.)

8. If AI doom is not a real threat, believing in it makes you believe some other not-real things that seem crazy or distasteful. (Counter: do we have a clear argument why it is not a real threat yet, in the list above?)

1. Like climate change, right?

2. At what cost? Much like the climate change above, you'll have people on the AI side even when it's out in the field extincting us.

4. Adding, over time synthetic data and its generating algorithms can become unaligned with human needs/behaviors (an example would be our current stock market, numbers must go up!).

8. Going back to climate change, it was predicted a long time ago, and while the explosion of automobiles has greatly improved human lives the risks of climate change could erase a lot of that. Might have been better if we dealt with the problem before we have to give the thermometer worried looks.