Remix clone Hacker News

new | show | ask | jobs Github

	▲	godelski 2 days ago
		`> Anthropic research has shown that alignment faking can arise even when the model wasn't instructed to do so` Correct. And this happens because training metrics are not aligned with training intent. `> to specifically fake that it is aligned when it is not.` And this will be a natural consequence of the above. To help clarify it's like taking a math test where one grader looks at the answer while another looks at the work and gives partial credit. Who is doing a better job at measuring successful leaning outcomes? It's the latter. In the former you can make mistakes that cancel out or you can just more easily cheat. It's harder to cheat in the latter because you'd need to also reproduce all the steps and at that point are you even cheating? A common example of this is where the LLM gets the right answer but all the steps are wrong. An example of this can actually be seen in one of Karpathy's recent posts. It gets the right result but the math is all wrong. This is no different than deception. It is deception because it tells you a process and it's not correct. https://x.com/karpathy/status/1992655330002817095