| ▲ | YuriNiyazov 6 hours ago | |||||||
It's easiest to explain this while anthropomorphizing the model, I know some folks here hate that, sorry about that. I heard an interesting diagnosis for why Claude does this: the output is a compressed version of its thought traces, very dense because the model is under pressure to use as few tokens as it can and to pack as much (for accuracy) of its concepts into the output. One of the reasons that "don't do X" type of instructions work reliably is because you are telling the model "don't think of a pink elephant". There's also Anthropic's related research that shows that when you tell a model "don't do X", and it does X later for whatever reason, it starts acting more misaligned. This is because it thinks "well, I guess I am the sort of model that disobeys instructions, whatever" - this was specifically about cheating on tests, but you can imagine this happens in other contexts as well like following instructions on what kinds of text to output. So, what you want to do is to avoid telling Claude "don't do X", and tell Claude "in your thoughts, in memories and various notes that you write, use your Claude-ese. In your output to humans, translate everything into long full sentences." If anyone's interested, I can share my Claude Code output style that reflects this. (Hi Adnan! Long time! (Adnan is an ex-coworker)) | ||||||||
| ▲ | faeyanpiraat 4 hours ago | parent [-] | |||||||
Please share! | ||||||||
| ||||||||