| ▲ | athrowaway3z 2 hours ago | |
This feels fundamentally right to me. The best compression you're going to achieve is when you best understand(i.e. encode) the truth of things. Perhaps you're thinking of that too narrowly - it is not that the model gets good at understanding how a bridge stays upright; it has to also understand the truth of how a specific person might understand it, and how that specific person might put it into words. Do that right, as slight deviation of some understanding of how humans respond in general instead of encoding for a single person, etc, etc, etc - and you reach optimal compression. It's more efficient for the model to encode _how_ an idea is wrong, than it is to encode the wrong idea. Its how/why they detect when they're being evaluated. | ||