Maybe a dumb question but once you have a model that spends a lot of tokens for high performance why can’t you distill it into a model that thinks less? Sure it wouldn’t generalize well to new tasks but plenty of tasks are important and recurrent enough to justify this.

Doesn’t seem like a “wall” to me.

There's a lot of research along this particular line, if you're interested!

This is a dataset basically constructed by taking a bunch of DeepSeek R1 trajectories and filtering out the incredibly long rambling ones: https://huggingface.co/datasets/open-thoughts/OpenThoughts-1...

There's also a bunch of RL research around strategies to coax the student model towards more concise trajectories.

In general, a lot of people in industry are finding cost savings these days by tuning a smaller model over longer agent trajectories (not exactly the same as distilling the reasoning traces, since those aren't exposed by many frontier labs anymore) collected from their production agents. Typically works best when you have a pretty workflow-y agent whose tasks all share a similar shape.

They do this already. Have you noticed the stronger model tends to get released first and the weaker models soon after?

The reasoning/thinking is basically trying many many different branches.

To do that you need a larger model to have many many branches to try.