Remix.run Logo
deyiao 2 hours ago

It's unfortunate that the final few years of the fastest AGI progress happen to coincide with a global surge in nationalism, polarization, and mutual hostility between human groups.

That raises a disturbing failure mode: under intense conflict, an AI could learn that exterminating some groups of humans is a justified or even desirable objective. Once that principle is accepted, the step to concluding that exterminating all humans is justified may become much smaller than we'd like to believe.