> are on track to make an ultraintelligent thinking machine, the risks I'm worried about are: killing most jobs, concentrating most wealth, and putting ourselves at risk of literal extinction. All I could think when reading this was, "this person really did not think this all the way through." I'm sorry.

While two out of those three risks do not require anything "ultraintelligent", for killing most jobs it's worth remembering that their track is an appropriation of intelligence, and the speedups happen where it's easiest to measure more than where the tracks correspond to economic value.

Unfortunately, this has the opposite effect for safety, and can still do a lot of wealth concentration even if it doesn't end all jobs.