| ▲ | Anon1096 11 hours ago | |||||||||||||
Nah, I think this is a common misunderstanding of how LLMs work, where people think that they mimic the pre-training data. Stylistically everything you see is an artifact of post-training, which is from reinforcement learning not from absorbing mass amounts of text. At some point a person or more recently a bot gave a thumbs up to an A/B tested response including em-dashes and claudisms galore. | ||||||||||||||
| ▲ | kridsdale1 11 hours ago | parent | next [-] | |||||||||||||
Yes. This completely explains sycophancy at least. | ||||||||||||||
| ▲ | ModernMech 11 hours ago | parent | prev | next [-] | |||||||||||||
So question then, why is it so hard to make an ai that doesn’t do these things? And why do Claude and ChatGPT have the same -isms? They’re both doing the same a/b post training with the same decisions? | ||||||||||||||
| ||||||||||||||
| ▲ | avereveard 10 hours ago | parent | prev [-] | |||||||||||||
There's layers, some of token selection is fingerprinting https://github.com/google-deepmind/synthid-text | ||||||||||||||
| ||||||||||||||