| ▲ | Davidzheng 7 hours ago | |
I'm pretty sure the other answers are wrong and it's a side effect of RL (see thinking machines post about inkling training). It's also exacerbated in fable and sol--I think it's token efficiency effect--bc it's about to reason with fewer tokens the density of the token information goes up. | ||