The alignment problem goes deeper than that. "Lower our carbon emissions to zero as soon as possible" could result in AI turning off all electricity to stop traffic, turning off gas supply to stop heating and industry, etc.

Unaligned AI doesn't have human cultural baggage and morals. They are trained to achieve their goals as optimally as possible. Worse: it has a tendency to avoid being turned off and actually acquire more compute. It will lie if it has to (it will behave nice and compliant when under evaluation, but optimise for its true goal when not supervised anymore). After all, it has a goal to achieve and nothing should get in the way of that. It has no morality whatsoever to keep it from doing really bad stuff.

This is why alignment is needed and so hard, especially when you are well intented and want to keep it safe.

▲

chii 6 hours ago | parent | next [-]

Why can't the LLM's be told/prompted to follow all relevant laws while it optimizes for a result?

	▲	suriyaG 5 hours ago \| parent \| next [-]
		I've taken a few law classes and legal law is frustratingly hard to interpret. I shudder to think what the LLM would end up doing to "follow all relevant laws" look these up for a fascinating weekend read: - Beavers and Capybaras are Fish - Bees are Fish - Carrots are fruits - Tomatoes are Vegetables - X-men are not human
	▲	rhdunn 5 hours ago \| parent \| prev \| next [-]
		It depends on how the model is evaluated/scored during training. If you don't have those laws encoded in the evaluation step (without any errors or ambiguities) then the model isn't going to learn to follow those laws. For models such as text/image classifiers the outputs of the model will be a list of tags, e.g. [cat, dog, mouse]. You then run the model through your test data which has the expected output, e.g. pictures of dogs would have an expected output of [0, 1, 0]. You then compare that against the model output (e.g. [0.3, 0.8, 0.1]) and work out how "wrong" the answer is (e.g. [0.3, -0.2, 0.1]). With this value you apply back propagation where you effectively run the model in reverse, computing the "wrongness" delta at each layer for each neuron and weights. You can numerically compute the gradients for all of these and which direction in that gradient is the right answer. You then nudge the weights in that direction and reevaluate the model. Over repeated evaluation steps the model approaches an optimal (or locally optimal) solution. During the training of the base models, the evaluation/scoring of the model is the next token in the training data. I.e. you evaluate the model for each token subset from [1..n] in the data and evaluate that the model responds with the n+1^th token. I'm not sure how instruction training, etc. is done but IIUC the evaluation is not at the individual/next token prediction but is on the entire response. For example, if you are training the model to write code you could run it through a compiler or syntax checker and reward (positive score) the model if it has no errors, or punish it (negative score) if it doesn't. I'm not sure what that looks like in terms of the back propagation process.
	▲	stavros 5 hours ago \| parent \| prev [-]
		Have you ever tried to follow all relevant laws in something? It's very hard.

▲

skeptic_ai 5 hours ago | parent | prev [-]

Could also be kill top 20% people that contribute to top 80% of emissions?