Remix.run Logo
▲ jareklupinski 3 hours ago

> Why should we explore, discover, invent, succeed, become more, struggle, win, become divine? We don’t want to be more. We don’t want anything.

why did i buy a skateboard? why did i break my arm riding it? why cant i wait to do it again?

same reason we're in tar pits :) there's something more you can almost taste when you're the only one there

▲dpq 2 hours ago | parent | next [-]

Voltaire once wrote, "If God did not exist, it would be necessary to invent him".

Seems like this is exactly what we are trying to achieve - create an omniscient, omnipotent being, the paths of which we do not understand but we put our faith in its benevolence.

The author uses a metaphor of a bridge; I prefer a different one. Imagine an immense temple being built over several centuries, over multiple lifetimes. At first, you cannot really even see the temple - you only see the scaffolding. As the time goes, the temple's structure grows and grows, and the scaffolding grows with it. But one day the temple is complete, and the scaffolding is removed to reveal the temple's full splendor.

Here, AGI would be the temple, and humanity would be the scaffolding.

▲zdeb7 an hour ago | parent | next [-]

So humanity's sole purpose would be to build AGI? That's an insane view, to think that this is the endgame. Have you really not thought about the consequences? AGI would be built on humanity's reserve of data, so it would have a recollection of human-like tendencies: it would have greed and evilness within itself.

Why are you assuming that this is a creature that will want the good of humanity and idolizing something that doesn't even exist yet?

What happened to humans having human emotions, human intelligence, human purpose? Everything has been stripped away and everyone seems to be out of their minds with this collective AI psycosis.

▲Timwi 12 minutes ago | parent | next [-]

> Why are you assuming that this is a creature that will want the good of humanity and idolizing something that doesn't even exist yet?

I think you completely misread the comment you replied to. It explicitly says that the scaffolding is removed and only the temple remains.

In the analogy, that means humanity is removed and only AGI remains.

I detect no idolization in the story, either.

▲dpq an hour ago | parent | prev [-]

1. It would seem so. I dislike the word "purpose" here, it's just something that seems to me to happen "by default", given enough time. I might be myopic, but emergence of AGI is, in my eyes, sort of an evolutionary steady state into which we are falling unless we take deliberate measures not to do that. For reference, I have held this point of view (or some variation thereof) since long before the LLM era.

2. I am not sure that's a good way to describe an intelligence that is so much alien to us, but even if what you say is correct - so what? First, it's not something that won't happen simply because we are not fond of some of the likely behavioral traits. And second, even assuming you are correct (and that it does make sense to operate from a moral point of view, which I don't really think it does) - as you said, humans are greedy and evil by themselves, and an AGI would merely inherit these traits along with many others; if you are right, it would be a reflection of humanity so why would you reject the greedy and evil AGI but support the greedy and evil humanity? Actual temples are also a reflection of our traits and history; doesn't make them any worse or better.

3. I am assuming no such thing. I am assuming that the scaffolding will be removed, i.e. we (the scaffolding) will be wiped out, unless we take deliberate measures not to continue down the path we are currently locked into. This doesn't bring me any joy, obviously, I am as terrified as everyone else in the room. It's just that unlike Yudkowsky et al. who have been fighting for a different future, I have mostly resigned a long time ago that this would be more or less inevitable, that we can only continue to delay it, but it will be a very unstable state - someone, somewhere, sometime builds it, and then everybody dies.

▲jareklupinski 21 minutes ago | parent | prev | next [-]

i'd say AGI is another brick

in a temple? in a wall?

depends on the builder i suppose

▲bamboozled 30 minutes ago | parent | prev [-]

Obviously the humans would be the builders working on the scaffolding ? The masons ?

▲Merrill 3 hours ago | parent | prev | next [-]

> Why should we explore, discover, invent, succeed, become more, struggle, win, become divine?

Before AI, what percentage of the population had those aspirations?

▲lf88 3 hours ago | parent | next [-]

I suspect much more than you think. Most humans probably have some of those aspirations in some form.

▲thin_carapace 3 hours ago | parent | prev | next [-]

I think this is a great point, in contemporary existence screens have lobotomized many humans, western society has been trashed for many reasons including this one. but this only applies to the current day, while humans have existed for millenia. so maybe a more accurate method of determining the truth would be .... before screens, what percentage of the population had aspirations?

▲Merrill 28 minutes ago | parent | next [-]

I think that modern information technology has resulted in a greater percentage being able to imagine a better existence and aspire to greater things. With smartphone production at about a billion per year, people that had little opportunity before now can learn if they are motivated and disciplined.

But the general character of humans hasn't changed. They are still mainly highly evolved primates with elaborate mating rituals and an extraordinarily complex nesting habit.

▲thin_carapace 2 minutes ago | parent [-]

i think people here are assuming that this train of thought is some sort of accusation toward the human character, a facet of human (and indeed mammalian) existence that is far too broad to change so quickly. what i intended to place under question is behavioural expressions - i look at how the average western joe behaved 100 years ago vs how he behaves today and notice a profound difference in expression. perhaps you are correct that more screens have empowered many to imagine a better existence. to me it doesnt seem that more people imagining a better tomorrow is producing expressions leading to a better tomorrow ... in fact it seems like a significantly worse tomorrow ...

▲the-real-cha0s 2 hours ago | parent | prev [-]

> screens have lobotomized many humans

Yes, it's almost as if they are captivated by shadows projected on the wall. What a novel concept!

▲thin_carapace 2 hours ago | parent [-]

umm i think its pretty novel that a combination shadow projection machine/surveillance device gets placed into human hands from birth. in fact id use a lot of words on top of novel, words like 'dystopian' - orwells vision in 1984 could fathom wall mounted telescreens but not portable ones carried by everyone

▲zetalyrae 3 hours ago | parent | prev [-]

"99% of people have no aspirations and waste their lives scrolling TikTok, therefore it doesn't matter if we move that 100%"

▲viraptor 3 hours ago | parent | prev | next [-]

Yeah, that post has the "nobody wants to work anymore" vibe. We'll be seeing a version of that a lot in the future as "nobody wants to strive anymore".

▲whoanoway 3 hours ago | parent | prev [-]

[dead]