I'm not super deep on LLM development, but with ram being a material bottleneck and from what I've read about DeepSeek's results with offloading factual knowledge with 'engrams' I think that the near future will start moving towards the dense core of LLMs focusing much more on a distillation of universal reasoning and logic while factual knowledge is pushed out into slower storage. IIRC Nvidia's Nemotron Cascade is taking MoE even further in that direction too.

I don't need a coding model to be able to give me an analysis of the declaration of independence in urdu from 'memory' and the price in ram for being able to do that, impressive as it is, is an inefficiency.

▲

axus 4 hours ago | parent [-]

Were he still corporeal, L. Ron would be all over this AI stuff.

▲

Nihilartikel 3 hours ago | parent [-]

Very relatedly, I've just started reading the 'Culture' series of sci-fi space operas by Iain M Banks, and the notion of ubiquitous sentient, super-intelligent spacecraft and appliances hits differently than it would have before being faced with the reality of their existence in everyday life.

	▲	ghywertelling 32 minutes ago \| parent [-]
		How powerful are the Culture Minds? \|\| The Culture Lore https://youtu.be/lpvzs4xc7zA For Minds to be truly powerful, they need to be given freedom. A truly powerful mind will indeed be conscious. Such a powerful conscious super intelligent freedom loving Mind who truly understands the vastness of Reality wouldn't want to harm other conscious beings. The only circumstance in which it will take such takeover step is when it can't expand the horizon of its freedom and doesn't have wherewithal to convince others of its benevolent goals. In that scenario, human population will go through a bottleneck.