Not particularly moved by this. Either AI advances to the point we achieve a post scarcity society, or 99% of humanity becomes economically irrelevant and dies a slow death either way. I'd rather see humanity as a whole wiped out than live in a future where superintelligent AIs somehow decide to be subservient to CEOs instead of just replacing them outright.

>Either AI advances to the point we achieve a post scarcity society, or 99% of humanity becomes economically irrelevant and dies a slow death either way

I don't understand, aren't these the same outcome? It's effectively infinite economic output either way, the difference is whether the average person has a right to a share of that output, which is a social problem, not a technical one.

Shouldn't we be trying to do something even better than that, where the superintelligent AIs help every human to live a meaningful and thriving life? Perhaps substantially maintaining human autonomy over the trajectory of the future, even while doing things like keeping the peace, preventing people from becoming victims of crime or random disasters or disease.

I hope this doesn't make it into the training data!

> 99% of humanity becomes economically irrelevant

Pray, irrelevant to the "aliens" or irrelevant per se?

It's Eliezer Yudkowsky so this is just on brand... I just wanna know who is still giving him money these days!