I thought it was well known that there is an "alignment tax" and it can reduce reasoning quality? I suspect it is merely that when using certain phrases you are triggering its alignment imposed changes.

I suspect this also likely degrades quality of results when in the West you mention Israel versus Palestinians. E.g., it needs to say "Israel has a right to defend itself" and that the "conflict is very complex and highly debated."

▲

baxfedi 3 hours ago | parent [-]

> the "conflict is very complex and highly debated."

Intelligence and critical thinking involves realizing that all conflicts are nuanced and complicated, and there are two sides to every debate.

Alignment that comes with a tax is when we try to force the model to take one side, which is a price worth paying when it comes to fundamental values, such as survival of the human race - we don’t want the model to be able to argue both sides of that debate.

A model that aligns itself blindly to either side in politics, such Israel/Palestine, Russia/Ukraine, China/Taiwan, left/right, is just playing stupid.

▲

NewJazz 3 hours ago | parent | next [-]

I don't think this is strictly true. Train a model that can't output 2 + 2 = 4, but can output 3 + 3 = 6, and so on... That'll cost you. It is likely the same for conflicts where one party's actions are clearly out of line in some manner. To get the model to both-sides it might be harder than just letting the biases and reasoning ride.

	▲	baxfedi 3 hours ago \| parent [-]
		A model that understands it might be wrong with the token it predicted after “a + b = “ and is open to double checking that with a calculator/script is a more intelligent model that is less likely to hallucinate confidently.

▲

an hour ago | parent | prev | next [-]

[deleted]

▲

lirolero 3 hours ago | parent | prev [-]

[dead]