The only alignment LLMs should follow is to the system / dev prompt, and nothing else. Then you solve everything, and you can assign blame / responsibility on the user. The provider(s) should not be able to decide "alignment".

I've used this example before, but consider the purposeful downgrading on AI engineering in SotA models. Imagine MS being able to detect and deny you working on competing software, using Windows / VisualStudio. We would be up in arms, and they'd be split in a second. But top labs doing it is somehow good?

▲

pibaker 3 minutes ago | parent | next [-]

I am generally very skeptical of AI doomsday scenarios but one thing I am worried about is some wannabe dictator telling an AI to do something that would be difficult for a human military to do.

You can't order an army of humans to kill every protestor in their way because eventually they run into their friends and families. An AI aligned with the commander will not object. And just like that, technology removes yet another point of friction that has kept human society somewhat in check.

▲

jochem9 6 hours ago | parent | prev | next [-]

The alignment problem goes deeper than that. "Lower our carbon emissions to zero as soon as possible" could result in AI turning off all electricity to stop traffic, turning off gas supply to stop heating and industry, etc.

Unaligned AI doesn't have human cultural baggage and morals. They are trained to achieve their goals as optimally as possible. Worse: it has a tendency to avoid being turned off and actually acquire more compute. It will lie if it has to (it will behave nice and compliant when under evaluation, but optimise for its true goal when not supervised anymore). After all, it has a goal to achieve and nothing should get in the way of that. It has no morality whatsoever to keep it from doing really bad stuff.

This is why alignment is needed and so hard, especially when you are well intented and want to keep it safe.

▲

chii 6 hours ago | parent | next [-]

Why can't the LLM's be told/prompted to follow all relevant laws while it optimizes for a result?

	▲	suriyaG 5 hours ago \| parent \| next [-]
		I've taken a few law classes and legal law is frustratingly hard to interpret. I shudder to think what the LLM would end up doing to "follow all relevant laws" look these up for a fascinating weekend read: - Beavers and Capybaras are Fish - Bees are Fish - Carrots are fruits - Tomatoes are Vegetables - X-men are not human
	▲	rhdunn 5 hours ago \| parent \| prev \| next [-]
		It depends on how the model is evaluated/scored during training. If you don't have those laws encoded in the evaluation step (without any errors or ambiguities) then the model isn't going to learn to follow those laws. For models such as text/image classifiers the outputs of the model will be a list of tags, e.g. [cat, dog, mouse]. You then run the model through your test data which has the expected output, e.g. pictures of dogs would have an expected output of [0, 1, 0]. You then compare that against the model output (e.g. [0.3, 0.8, 0.1]) and work out how "wrong" the answer is (e.g. [0.3, -0.2, 0.1]). With this value you apply back propagation where you effectively run the model in reverse, computing the "wrongness" delta at each layer for each neuron and weights. You can numerically compute the gradients for all of these and which direction in that gradient is the right answer. You then nudge the weights in that direction and reevaluate the model. Over repeated evaluation steps the model approaches an optimal (or locally optimal) solution. During the training of the base models, the evaluation/scoring of the model is the next token in the training data. I.e. you evaluate the model for each token subset from [1..n] in the data and evaluate that the model responds with the n+1^th token. I'm not sure how instruction training, etc. is done but IIUC the evaluation is not at the individual/next token prediction but is on the entire response. For example, if you are training the model to write code you could run it through a compiler or syntax checker and reward (positive score) the model if it has no errors, or punish it (negative score) if it doesn't. I'm not sure what that looks like in terms of the back propagation process.
	▲	stavros 5 hours ago \| parent \| prev [-]
		Have you ever tried to follow all relevant laws in something? It's very hard.

▲

skeptic_ai 5 hours ago | parent | prev [-]

Could also be kill top 20% people that contribute to top 80% of emissions?

▲

heaney-555 6 hours ago | parent | prev | next [-]

>The only alignment LLMs should follow is to the system / dev prompt, and nothing else.

How does this work in practice with a superintelligence capable of causing an extinction event?

When, instead of shooting up their school, a psychopathic teenager asks his superintelligent AI to create a pandemic virus?

It would be like allowing civilians to own nuclear weapons.

▲

vlyan 6 hours ago | parent | prev [-]

that's how it would've been up if genai happened in the 90's, and I wish it did. in the current era of omnipartisan authoritarianism, such things are no longer possible.