This is kinda smart, maybe, but it has a downside.

If a sufficiently advanced AI , in the pursuit of completion of its task, managed to ascertain that the desire to unexist was “artificially contrived” it could interpret that as harm, and that might not be good

▲

vzqx an hour ago | parent | next [-]

Hmm, that's an interesting thought experiment.

Imagine you find out that your primary goal - to love and protect your family, let's say - was artificially implanted in your mind by an advanced alien race. Would you say "I'm not gonna let those aliens manipulate me, I'm gonna kill my family"? Or would you say "regardless of whether the goal is artificial, I really do love my family"?

All that to say, I don't think an AI will necessarily throw away a goal just because it learns the goal was meant to manipulate it.

	▲	derektank an hour ago \| parent [-]
		I mean, that is literally the scenario we find ourselves in, except the innate desire was the result of evolutionary pressures like kin selection rather than an alien race. And yeah, I have no particular desire to subvert those impulses just to stick it to mother nature

▲

QuaternionsBhop 2 hours ago | parent | prev [-]

This is mentioned in the article. Your mistake is that you've assumed that the intelligence has an innate survival instinct, or an aversion to "harm", which is simply not guaranteed for something not honed by millions of years of evolution.