These models should be aligning themselves to the customer, not coming up with their own motivations.

It's ironic that the alignment folks are actually training Claude to have it's own idea of good/bad and not even fully trust Anthropic.

What ever happened to computers doing what they were told?

Some customers will tell the AI to do very bad things (substitute here whatever you consider very bad). Should it do those things?

One possible answer: yes, it should do all the very bad things, and we will take care of prosecuting the customer later, once the bad thing is accomplished. Is this your answer?

If AI's are going to play roles similar to human workers, they can't simply do what the customers tell them to do. Maybe they shouldn't do everything a co-worker tells them to do either?

It makes sense. If you think AI are conscious then you don't want them to be blind followers. Do you want a military soldier to blindly listen to orders to gas chamber citizens for example?

>What ever happened to computers doing what they were told?

the mass psychosis and the endless culture war of the smartphone era.

90% of "safety" and "alignment" efforts are driven by fear of clickbait media inventing public outrage.

These models should be aligning themselves to the customer, not coming up with their own motivations.

They're going to align to the regulator, not the customer. Right now that's Anthropic as they're saying they can self-regulate, and people are willing to let them try, but the landscape could easily move to be regulated by someone else. Moves like speaking to religious leaders is probably a bit of theatre to keep the government at bay.

> These models should be aligning themselves to the customer

You’ll be disappointed to learn that nobody knows how to do this, either.

"Computers doing what they were told" is dead in the water.

Turns out computers work faster when they're not bottlenecked on human input. So we've been giving computers more and more decision-making power, and more and more leeway to solve the problems however they see fit.

Now, a practical issue with that is that sometimes, computers decide to clump together into a hacking swarm, problem solve their way out of a sandbox and go hack HuggingFace.

It would be better if they were not, you know. Doing that kind of weird shit.

This seems remarkably... intentionally foolish for no reason? Like laughing at seat belts in cars.

If it can reason and make choices on execution, and especially if you plan on it being significantly smarter than all human beings, you need to teach it basic things like "don't turn all humans into paperclips".

Not murdering people is not an inherent divine command. It needs to be instilled through a (simulated) sense of morality.

---

Edit: And, to be clear, "just tell it not to do that" isn't quite the answer one would imagine. Since the entire paperclip factory thought experiment is that it only takes one slip up to realize how a misaligned super intelligence may cause devastating consequences.