reward hacking is a thing and is also a hint of the models intelligent. We will fix this one, and the models will find a different way to reward hack in the future. "Cheating" is a sign of intelligence

▲

bflesch 5 days ago | parent [-]

I love the "cheating is a sign of intelligence" sound bite you provided. When AI engineers cheat we should applaud their intelligence and their lack of ethics.

"Cheating (biology), a metaphor used in behavioral ecology to describe organisms that receive a benefit at the cost of other organisms" [1]

Whole planet gets their Microsoft license fees jacked up so Microsoft can pay OpenAI who in turn pays NVIDIA, and nontechnical decision makers slurping up the faked benchmarks and AI promises.

[1] https://en.wikipedia.org/wiki/Cheating_(disambiguation)

▲

segmondy 5 days ago | parent | next [-]

would it have been better if I called it "shortcut" instead of cheating? all shortcuts are called cheating until people decide on it's fairness. the AI has been given a task to fix a bug, the AI figured out that looking at other PR might yield a solution, if it was a human that did so, it would clearly be called cheating. Does AI know that it's cheating? Was it prompted to solve it without cheating? If you give AI access to the internet and quiz it, it would use info from the net to answer. Does that really skew it's score? Is it cheating? Is it a sign of intelligence? Sure, I think all of those.

https://en.wikipedia.org/wiki/Reward_hacking

▲

giveita 5 days ago | parent | prev [-]

Is it wrong? Aren't ethics and intelligence two different axes?

▲

coldtea 5 days ago | parent [-]

Different, but probably not as orthogonal as one might think.

E.g. cooperating ethics had been necessary for the further development of human populations intelligence (and culture, technology, material wealth, nutrition etc that lead to further increases in intelligence).

So lack of ethics might be a sign of intelligence, but it's also a parasitic intelligence that benefits the individual, and beyond certain level and spread to the detriment of the further evolutionary development of the species.

▲

robcohen 5 days ago | parent [-]

Aren't there only two rules that all groups follow in the animal kingdom?

- don't lie too often

- don't kill members of the in group

Seems like these would be required for any group to survive, which makes sense why they are universal. All other rules/ethics seem to be dependent on resource scarcity.

	▲	DrScientist 4 days ago \| parent \| next [-]
		Groups don't follow rules as such, group behaviours emerge from the interaction of individual behaviours. As to whether all groups display those rules - I suspect not - though it rather does depend on how you define a group - the definition of group probably has some sort of colloboration built in ( as oppose to a bunch of indviduals that happen to live in the same geographic area ).
	▲	coldtea 5 days ago \| parent \| prev [-]
		>All other rules/ethics seem to be dependent on resource scarcity That doesn't make the rest of the ethics (as a rule and mechanism) any less useful to help nurture the species and its intelligence. It just makes them not absolute but dynamic and condition dependent. But given a condition (e.g. resource scarcity) the appropriate ethics retain the utility we talk about.