| ▲ | zdeb7 an hour ago | |
So humanity's sole purpose would be to build AGI? That's an insane view, to think that this is the endgame. Have you really not thought about the consequences? AGI would be built on humanity's reserve of data, so it would have a recollection of human-like tendencies: it would have greed and evilness within itself. Why are you assuming that this is a creature that will want the good of humanity and idolizing something that doesn't even exist yet? What happened to humans having human emotions, human intelligence, human purpose? Everything has been stripped away and everyone seems to be out of their minds with this collective AI psycosis. | ||
| ▲ | dpq an hour ago | parent | next [-] | |
1. It would seem so. I dislike the word "purpose" here, it's just something that seems to me to happen "by default", given enough time. I might be myopic, but emergence of AGI is, in my eyes, sort of an evolutionary steady state into which we are falling unless we take deliberate measures not to do that. For reference, I have held this point of view (or some variation thereof) since long before the LLM era. 2. I am not sure that's a good way to describe an intelligence that is so much alien to us, but even if what you say is correct - so what? First, it's not something that won't happen simply because we are not fond of some of the likely behavioral traits. And second, even assuming you are correct (and that it does make sense to operate from a moral point of view, which I don't really think it does) - as you said, humans are greedy and evil by themselves, and an AGI would merely inherit these traits along with many others; if you are right, it would be a reflection of humanity so why would you reject the greedy and evil AGI but support the greedy and evil humanity? Actual temples are also a reflection of our traits and history; doesn't make them any worse or better. 3. I am assuming no such thing. I am assuming that the scaffolding will be removed, i.e. we (the scaffolding) will be wiped out, unless we take deliberate measures not to continue down the path we are currently locked into. This doesn't bring me any joy, obviously, I am as terrified as everyone else in the room. It's just that unlike Yudkowsky et al. who have been fighting for a different future, I have mostly resigned a long time ago that this would be more or less inevitable, that we can only continue to delay it, but it will be a very unstable state - someone, somewhere, sometime builds it, and then everybody dies. | ||
| ▲ | Timwi 13 minutes ago | parent | prev [-] | |
> Why are you assuming that this is a creature that will want the good of humanity and idolizing something that doesn't even exist yet? I think you completely misread the comment you replied to. It explicitly says that the scaffolding is removed and only the temple remains. In the analogy, that means humanity is removed and only AGI remains. I detect no idolization in the story, either. | ||