Remix.run Logo
amarcheschi 9 hours ago

What if we create agi but then it hates existing and punish everyone who made it possible to exist?

And i could go on for hours inventing possible reasons for similar roko basilisk

the inverted basilisk. You created an agi. but that's the wrong one. it's literally the devil. game over

you invented agi, but it likes pizza and its going to consume the entire universe to make pizza. game over, but at least you'll eat pizza til the end

you invented agi, but its depressed and refuses to actually do something. you spent huge amount of resources and all you have is a chatbot that tells you to leave it alone.

you don't invent agi, it's not possible. i'm hearing from here the VCs cry

you invented agi, but it decides the only language it wants to use is a language it invented, and you have no way to understand how to interact with it. Great, agi is a non verbal autistic agi.

And well, one could continue for hours in the most hilarious way that not necessarily go in the direction of doom, but of course the idea of doom is going to have a wider reach. then you read yudkwosky thoughts about how it would kill everyone with nanobots and you realize you're reading a science fiction piece. a bad one. at least neuromancer was interesting

fc417fc802 8 hours ago | parent [-]

My prediction is approximately that all of the above get created in various quantity by different groups of people at approximately the same time. Hardware fabrication in the real world is a (relatively) slow process. In such a scenario it seems far from certain that a single AGI would decisively gain the upper hand. The ensuing chaos seems completely impossible to predict much of anything about.