You can't surpress it because that's what reinforcement learning is.

Human traits?

The AI will be a cruel as humans.

Just yesterday news and TV was full of what happened at 9/11, something that was truly horrible.

I'm from Germany, and why 3 to 4 generations ago happened here was truly horrible.

All was done by extremists, thought.

But... just the other day I read https://de.wikipedia.org/wiki/Amerikanische_Besetzung_Haitis about the US occupation of Haiti. And that was done by a government that claimed to be not extremist and even democratic. Way more people died there than even in 9/11. And it had almost all the things happening as they happened in the 3rd Reich: Racism, looking down at others, concentration camps, torture, forced labor till death, killing family members (what we call "Sippenhaft"). Something between 3500 and 15000 people were killed by US troops. That's still low compared to what 3rd Reich Germany did ... but quantity is not the issue when we talk about traits, quality is.

So the same "human traits" made US troops do cruel things as they made Germany extremists do cruel things. So we must conclude that they aren't all good. And therefore not all desirable.

Fun thing: this is known since a loooooong time. About 2000 years ago a religious leader (that gets way more followers in the US than in Germany) said "There is no good one, not even one".

And even today people act like humanity is inherently good. No, it isn't. If we were, then anarchism or communism would actually work and really give some kind of paradise on earth.

Human traits are bad training material.

I’ve engaged with some of the alignment people and their writing somewhat and, at least for the subset I was interacting with, I think they’d agree.

The problem that they were pointing at isn’t “how do we align these systems to a person’s goals”.

It is a cluster of problems.

We don’t know how to begin to think about how to align these system’s to a person’s goals.

Aligning it to an individual is fraught with peril, and we don’t know how to begin to think about what to align it to instead.

(You could try for something like virtue ethics, but someone will have to pick and choose, and small biases there could have big impacts.)

And even if you could sort that out - human values drift over time, so you need something that can shift its values in ways that we’d endorse. Assuming we understood the shift.

One example I came across was that if you booted up an AI aligned with something like “upstanding citizen” but anchored on values from a few generations back, it might suggest you use slaves to solve your problems.

And if you had something that used some super intelligent process to reason through it’s own version of virtue ethics in a way not so dependent on the details of the present norms, you might end up with something that pays a lot of attention to moral horrors that aren’t quite visible to us yet.

When I came across the above, there weren’t many concrete suggestions in there.

These were all just illustrative examples of: having these systems grow in power / intelligence / effectiveness in ways that are safe for humans is very hard, and we don’t really know how to think about what solutions would look like.

The actual reasons they believe this - and have done for a long time now - come from some detailed conceptual models that have a good track record of calling things in advance.

But it takes a bit of reading to understand their models of the world.

There were two day workshops at one point that did a good job, and that was about as condensed as those people thought they could get it at the time.

”Because if humanity has shown anything, it's that a lot of people are, euphemistically, bad individuals”

In reality most individuals are good people.

Individually, people prefer be kind and compassionate, prefer to help when they find another in trouble.

Our view of the world has become distorted by the relentless focus of social- and mass-media on violence and rage inducing clickbait. Including on the few people in power who are in fact sociopaths (a tiny minority, but they’ll get more focus than reasonable, well-behaved CEOs voicing nuanced opinions).

If you look around yourself you’ll see much more good than bad; if the looking is at your screen it’s easy to become depressed and lose faith.

I do agree with the above mentioned view that corporations can show ‘sociopathic’ behavior. Their incentives are monetary gains, shareholder value; inherently driving them away from social well being.

Here too, companies with a positive, emphatic corporate culture exist, but that takes strong leadership who can see beyond the monotonic view of monetary gains. And again, the media will throw examples of misbehaving companies in our face all day long before paying attention to things that went well on the backside of page 16.

I think you and the parent saying the same thing in different terms.

It's very unfortunate that the group who rightly saw AI as a big threat, brought a range of dubious baggage to the discussion. Especially with the "alignment" framework they brought the assumption that AI that does what no one says would be oh so much worse than AI which does what anyone says. But as you say, a fraction of people can be really bad indeed.

[dead]

Alignment is a myth. Safety of whom? Humanity couldn't agree on common set of values for thousands of years and we're not gonna suddenly do that in the next ten.

Despite all the fancy language, its more about aligning the AI behavior with the corporation's interests.

ie: the corporation wants the AI to behave a certain way for various reasons: to make it easier for them to avoid regulation, to make the corporation more money via different tiers of AI offerings, to ensure that the corporations products are hard for competitors to use, etc. And those are just the easy ones.

Every product is shaped this way. AI is not different.

If they do, they imitate the way humans are portrayed online, in the media. That is a very distorted view of humanity