This article, like all articles on this substack, is LLM generated.

Source: https://arxiviq.substack.com/p/coming-soon

It looks like a variant of "beam search" (using top-k instead of top-1), but it's not mentioned anywhere. What am I not getting ?

A consensus of confidence and self-consistency through consensus seem fine for certain kinds of tasks, especially ones that involve recalling training. This is a bit like scaling up determinism, obedience, collectivism vs individualism... which seems fine for many math problems. Neither confidence or consensus are the best way to confirm truth or accuracy in a generalized way.

The previous self-consistency approach and this confidence pruning approach aren't really novel, but it's nice to see the numbers run. Fundamentally these approaches are about handling contradicting results, but not resolving the contradictions or increasing the quality of reasoning. What if the rare idea is the right answer? You can squeeze the training juice harder, but if you still get the wrong answer when it really really mattered, you're just left with a stress toy in your hand.

One thing that is confusing about this write-up is that "DeepConf-low" is only mentioned once and in a screenshot, but it seems to outperform DeepConf-high for several tasks. I guess I'll need to read the underlying paper, but that seems troublesome.

Correct me if I am wrong, but by the looks of things on that chart the reduction in token use and the better score are all related to the fact that this method used 512 samples.... This doesn't seem to be of any use for local running agents or anything that has severe vram restrictions such as local models that people can run at home. So this would only benefit enterprise level systems no?

Is this article written by an LLM?

One question about how the proposed feature works: do I understand correctly, the NN remains the same but the code that handles it does the trick?

And we will be supposed to find said code at https://jiaweizzhao.github.io/deepconf at some point?

What's the difference with lowering the temperature?

Wonder what this means for the pelican riding on a bicycle test? Or will it just be good at strictly reasoning type problems.

I love this direction of research.

Reducing costs of reasoning is a huge ongoing challenge in LLMs. We're spending so much energy and compute resources today on reasoning that today's consumption rates were unexpected (to me) a short 1 yr ago. We're literally burning forests, the atmosphere and making electricity expensive for everyone.

DeepThink v3.1 made a significant leap in this direction recently -- significantly shorter thinking tokens at the same quality. GPT5's router was also one (important) attempt to reduce reasoning costs and make o3-quality available in the free tier without breaking the bank. This is also why Claude 4 is winning the coding wars against its reasoning peers -- it provides great quality without all the added reasoning tokens.

Getting inspiration from Alpha-go and MCMC literature -- applying tree weighting, prioritization and pruning feels extremely appropriate. (To improve the quality of Deep Think -- offered by Gemini & GPT5 Pro today)

So, yes, more of this please. Totally the right direction.

yet again i am asking for a mandatory "(LLM output)" label in the title like we do for pdf/video links

[dead]