Great article, even if it will be interesting to see whether things continue to develop in such a direction or not.

> There's a version of this future where the model card stops listing a knowledge cutoff at all, because what's left in the weights goes stale on a scale of years instead of weeks.

Future?

Even just recently I’ve read of two approaches to this problem:

Cactus have come up with Needle [0][1], which is their tool-calling focused 14 MB model (still an LLM!) – no world knowledge engrained.

And instead of say, tool call structure, VibeThinker [2][3] focuses on reasoning over world knowledge.

Combine these two approaches with a reliable search tool/a safe way of accessing the internet for the model, and you’ve got a probably slightly slower model for factual questions, which on the upside however doesn’t hallucinate.

[0] https://cactuscompute.com/needle

[1] https://news.ycombinator.com/item?id=49246804

[2] https://arxiv.org/abs/2606.16140

[3] https://news.ycombinator.com/item?id=48639240

▲

kennywinker 7 hours ago | parent [-]

That kind of setup is super dependent on a search engine, and search keeps getting worse.

▲

skissane 13 minutes ago | parent | next [-]

> search keeps getting worse

For a lot of use cases, you don't need a general purpose search engine – a search over a curated knowledge base works even better.

There are plenty of freely available data sets you can use, depending on the application; plus in many cases you will want to use internal-only knowledge bases containing non-public information (e.g. documentation for a corporation's internal systems and procedures). There are also many paid subscription domain-specific knowledge services available.

▲

msdz 7 hours ago | parent | prev | next [-]

Generally I agree, with two caveats/thoughts:

Providing not just any a baseline, but a correct and useful one, is ever more important the less the model is grounded in world knowledge – misunderstandings probably compound faster if there is no general grasp of (broadly) “life on earth”, or computers, or whatever.

And secondly, I think (consumer-oriented) search becoming worse and worse is a challenge that’s mostly solvable (but far from solved!) for the big labs: (Mostly) trusted or even editorialized/reviewed sources like published work, Wikipedia, etc. is something they could index internally, it doesn’t need to come from a random blog site on the public internet. Furthermore, there’s a whole slew of companies specializing in crawling-for-LLM (i.e., bypassing bot protections) now as well.

▲

jcfrei 6 hours ago | parent | prev | next [-]

search isnt that super hard to solve. the problem is that incentives are misaligned - which means companies who sell something have an incentive to share their index with the corporation that pays to be the default search provider. for a simple search engine that just delivers facts there's very little incentive to keep its index up-to-date: because that company wont get the apis to retrieve the content efficiently, so it needs more compute, bandwidth and storage to run.

▲

cyanydeez 7 hours ago | parent | prev [-]

the technicals of search did not get worse. the economics of the search provider did. If government wasn't stuck in the 50s mindset, search provider would have been nationalized just like a utility.

	▲	kennywinker 5 hours ago \| parent [-]
		I think the techinicals of search did get worse - the ai sloppification of the internet made the needle harder to find in the haystack.