How can you be cheaper if you query multiple models at once, compared to me just using a single model, which will always have the correct caching configured?

▲

rohaga 4 hours ago | parent [-]

If we cut off an unconfident/expensive model early in thinking, then we don't have to pay for additional thinking cost at all. For many tasks, this is quite a large saving.

That being said, whatever models we select to race at any point will ALL have their cache updated, so if they're chosen on the next race they'll be a lot cheaper.

▲

grim_io 4 hours ago | parent [-]

Maybe I'm misunderstanding, but this seems to oversimplify the problem.

Most frontier Models won't show you their true reasoning, but only a summary.

There is also no reason you can automatically assume the quality and price of the outcome based on the first few reasoning trace summaries.

If you look at the recent model releases, it becomes clear that the trend is that the largest models provide the most cost-effective solutions, because of the massive turn/tool call/reasoning reduction needed to find the solution, compared to cheaper and smaller models.

	▲	rohaga 4 hours ago \| parent [-]
		Our research seems to indicate you can get a decent signal based on the first few reasoning trace summaries. We analyzed hundreds of billions of tokens to figure this out. Any open-weights model also gives you reasoning, which is very helpful (because they are usually lower-quality candidates you would want to cut off early) The largest models are very well known for having large reasoning traces. The thing that cracks down on overthinking is reasoning-effort training, which is largely a function of how the model is trained (certain RL objectives/teachers) rather than size of the model. The Kimi K3 technical report has some excellent work on this: https://arxiv.org/pdf/2607.