Remix.run Logo
▲ m3kw9 a day ago

as with all doomers, they go from AI one day to ASI the next, never talks all the moves/counter moves humans make to get there. Is a canary when ever I see that.

▲ben_w a day ago | parent [-]

> never talks all the moves/counter moves humans make to get there.

I don't like ASI as a term ever since realising (thanks to everyone's responses to ChatGPT) that basically no two people agreed what AGI meant.

For discussions like this, "super intelligent" is specifically "smart enough that the moves/counter moves humans make to get there do not matter any more than my moves playing chess against Kasparov".

Spiky intelligence may keep us safe. Distribution shift and how slow machine learning is in novel environments might keep us safe. That's all in the outcome this site marks as "Enduring Complementarity".

I don't like ASI as a term; I'd disagree with the author of the website, and say you can't answer "yes" to "Do we reach ASI?" and then end up with Enduring Complementarity, it has to be further up the tree than that.