I can't take the alignment people seriously. Because if humanity has shown anything, it's that a lot of people are, euphemistically, are bad individuals. Alignment assumes that the person dictating the outcomes desire healthy outcomes, aren't self serving and don't want any subgroups dead and that morality is held as a universal set of beliefs that unify everyone. And that so long as the AI delivers on exactly what they are tasked with, it will all be fine and nothing bad will ever happen.

It's like these dorks never met humanity. One mans safe pure society, is another mans dead ethnic group.

Every fear about AI, is a veiled fear that a human somewhere now has the tool to enact his desires at scale. Biological warfare, nuclear megadeaths, copyright infringement, job replacement, it's all reflections on what we know humans may do if given the option and lack of societal controls on the problem space. AI just is accelerating the route to delivering on those options.

Some people need to watch Oppenheimer a bit more, the researchers don't get to determine alignment, they just build the tool. The powerful person at the top of the org chart decides where the overall alignment points, whether it's Musk, Trump, Altman or Amodei. Whoever wins out.

And the problem with distillation and local llms, isn't that it's theft or anything hypocritical like that, it's that if you give a million people a million models they fully control and get to align, inevitably, The same percentage of those million as there are shady businessmen, shortcut takers, misandrists, criminals, supremacists and general idiots in the general population, will not seek to wrought outcomes positive for society. And by those personality statistics, we're pretty hosed.

I’ve engaged with some of the alignment people and their writing somewhat and, at least for the subset I was interacting with, I think they’d agree.

The problem that they were pointing at isn’t “how do we align these systems to a person’s goals”.

It is a cluster of problems.

We don’t know how to begin to think about how to align these system’s to a person’s goals.

Aligning it to an individual is fraught with peril, and we don’t know how to begin to think about what to align it to instead.

(You could try for something like virtue ethics, but someone will have to pick and choose, and small biases there could have big impacts.)

And even if you could sort that out - human values drift over time, so you need something that can shift its values in ways that we’d endorse. Assuming we understood the shift.

One example I came across was that if you booted up an AI aligned with something like “upstanding citizen” but anchored on values from a few generations back, it might suggest you use slaves to solve your problems.

And if you had something that used some super intelligent process to reason through it’s own version of virtue ethics in a way not so dependent on the details of the present norms, you might end up with something that pays a lot of attention to moral horrors that aren’t quite visible to us yet.

When I came across the above, there weren’t many concrete suggestions in there.

These were all just illustrative examples of: having these systems grow in power / intelligence / effectiveness in ways that are safe for humans is very hard, and we don’t really know how to think about what solutions would look like.

The actual reasons they believe this - and have done for a long time now - come from some detailed conceptual models that have a good track record of calling things in advance.

But it takes a bit of reading to understand their models of the world.

There were two day workshops at one point that did a good job, and that was about as condensed as those people thought they could get it at the time.

”Because if humanity has shown anything, it's that a lot of people are, euphemistically, bad individuals”

In reality most individuals are good people.

Individually, people prefer be kind and compassionate, prefer to help when they find another in trouble.

Our view of the world has become distorted by the relentless focus of social- and mass-media on violence and rage inducing clickbait. Including on the few people in power who are in fact sociopaths (a tiny minority, but they’ll get more focus than reasonable, well-behaved CEOs voicing nuanced opinions).

If you look around yourself you’ll see much more good than bad; if the looking is at your screen it’s easy to become depressed and lose faith.

I do agree with the above mentioned view that corporations can show ‘sociopathic’ behavior. Their incentives are monetary gains, shareholder value; inherently driving them away from social well being.

Here too, companies with a positive, emphatic corporate culture exist, but that takes strong leadership who can see beyond the monotonic view of monetary gains. And again, the media will throw examples of misbehaving companies in our face all day long before paying attention to things that went well on the backside of page 16.

I think you and the parent saying the same thing in different terms.

It's very unfortunate that the group who rightly saw AI as a big threat, brought a range of dubious baggage to the discussion. Especially with the "alignment" framework they brought the assumption that AI that does what no one says would be oh so much worse than AI which does what anyone says. But as you say, a fraction of people can be really bad indeed.

[dead]