The problem with this kind of excellent work is that the response to it is always to say "Fuck the user".

For example, when there was a paper that came out showing that having model logprobs makes distillation an order of magnitude easier, the closed LLM providers instantly yanked out support for getting the full logprobs at every time step. You get at most top 10 candidates now and I'm sure even that's on the chopping block.

People will use this to argue that a model which has exceeded Opus 4.8 (Kimi K3) somehow got most of its performance through distillation of Opus 4.8.

I still don't buy that distillation was worth more than 3 months of "catch up" time for the chinese labs. Most people who use the word "distillation" to much are revealing their sinophobia.

What I found the most interesting, and unfortunately not at all surprising, is that the reasoning of the LLMs frequently contained much more useful information than the actual answers, because the answers were censored.