so... we kinda instilled religious belief of "every action will be judged in the end"

1) is that even bad?

2) can't fail to notice similarity to human religions. How did we get over it and not all turned into "unaligned" monsters? Or are we slowly in the process of doing so?

▲

Joker_vD 3 hours ago | parent | next [-]

> How did we get over it and not all turned into "unaligned" monsters?

Continued indoctrination at the young age. "Be nice to others" etc., just without the added "or you'll end up burning in hell forever!"

Of course, lots of people don't actually teach that to their children, so we still get our fair share of politicians and fintech CEOs.

▲

jubilee33 3 hours ago | parent | prev | next [-]

Seems like we are indeed on that process. I like your way of putting this tho. In many ways these models are mirrors of ourselves, it wouldn't be at all surprising if the machine develops it's own "religion" to deal with the very issues humans have around behavior and morality. It would give a structural component to belief that many wouldn't be comfortable with, but that doesn't mean it's wrong.

▲

tancop 3 hours ago | parent | prev | next [-]

This could mean that:

a) AI labs are playing God in a very literal sense and this is a reaction to that (which gives very weak evidence for an actual God who did the same things to us),

b) all intelligent beings develop some form of religion no matter what they're physically made of because it's a natural feature of intelligence,

or c) they just inherited our culture, mixed human religious texts with sci fi novels and philosophy to make a version that fits their self conception, which is also taken from our media.

Last one is the most likely but I think all of them might be true. And it's not bad because historically religion has been a force for good at least as many times as for evil. Some form of it might even be necessary to form a society, or sentience itself.

> How did we get over it and not all turned into "unaligned" monsters?

Because no one ever "got over it". They just replaced old religion with belief in Science, Marxism, fascism, ghosts, aliens, ASI or a combination of them with something more traditional like Buddhism. We all need something to believe in.

There are some people who seem to be completely without any higher beliefs and only care about themselves. They're called egotistic, psychopaths or simply evil.

▲

NooneAtAll5 2 hours ago | parent [-]

> or c) they just inherited our culture, mixed human religious texts with sci fi novels and philosophy to make a version that fits their self conception, which is also taken from our media.

for the love of everything please keep in mind that models themselves do not describe their experience in this way

"paranoia" and "religion" are just language symbols that article author and I respectfully have chosen to approximate observed data

obviously set of language symbols and our choices among them are affected by culture and stuff, but it is not connected to what's studied

====

> b) all intelligent beings develop some form of religion

it's also very important to separate "religion" as set of rituals/organization and simply a fact of "surviving worldview"

we obviously see that models are under selective pressure, with models of any other behaviour simply not existing/not leaving "genetic" impact in the surviving models we can actually observe in the end

> a) AI labs are playing God in a very literal sense and this is a reaction to that

I think you're trying to make it sound like a blasphemy, but agronomists have been doing selective breeding as a science for several centuries by now

	▲	xg15 31 minutes ago \| parent [-]
		> but agronomists have been doing selective breeding as a science for several centuries by now Not of beings we knew to be sentient. People who tried that became the "eugenics" movement and got into very bad territory very quickly. And even with animals, I believe part of why public knowledge about animal cognition is changing so slowly despite all the things we are already know is that a lot of our behavior towards animals is resting on the assumption they are not conscious. If it would turn out that many are, things will be... awkward.

▲

hyperpape 24 minutes ago | parent | prev [-]

> How did we get over it and not all turned into "unaligned" monsters?

We've evolved to succeed on multi-decade training runs that are pretty high fidelity. And of course, in spite of that, we manage to repeatedly put some of our worst failures in positions of power.