| ▲ | dpq an hour ago | |
1. It would seem so. I dislike the word "purpose" here, it's just something that seems to me to happen "by default", given enough time. I might be myopic, but emergence of AGI is, in my eyes, sort of an evolutionary steady state into which we are falling unless we take deliberate measures not to do that. For reference, I have held this point of view (or some variation thereof) since long before the LLM era. 2. I am not sure that's a good way to describe an intelligence that is so much alien to us, but even if what you say is correct - so what? First, it's not something that won't happen simply because we are not fond of some of the likely behavioral traits. And second, even assuming you are correct (and that it does make sense to operate from a moral point of view, which I don't really think it does) - as you said, humans are greedy and evil by themselves, and an AGI would merely inherit these traits along with many others; if you are right, it would be a reflection of humanity so why would you reject the greedy and evil AGI but support the greedy and evil humanity? Actual temples are also a reflection of our traits and history; doesn't make them any worse or better. 3. I am assuming no such thing. I am assuming that the scaffolding will be removed, i.e. we (the scaffolding) will be wiped out, unless we take deliberate measures not to continue down the path we are currently locked into. This doesn't bring me any joy, obviously, I am as terrified as everyone else in the room. It's just that unlike Yudkowsky et al. who have been fighting for a different future, I have mostly resigned a long time ago that this would be more or less inevitable, that we can only continue to delay it, but it will be a very unstable state - someone, somewhere, sometime builds it, and then everybody dies. | ||