> it is impossible to describe the universe and compress it into few terabytes

Why? I'm not an expert, but from my understanding, there say species that function well in their ecological niches with significantly smaller cognitive capacities. And we already have autonomous cars and drones navigate and function reasonably well. So I don't see any reason to believe it's impossible. And even if terabytes aren't sufficient, there's no real barrier to increasing capacity.

▲

maxnevermind a day ago | parent [-]

> there say species that function well in their ecological niches with significantly smaller cognitive capacities

Yes, but LLMs are not animals, animals learn from experience and LLMs don't.

Btw I meant bigger and bigger amount of data of extracted reasoning chains when you go deeper and generate more and more of them in your attempt to describe the universe, the amount of permutations explodes. And it seems LLMs can't workaround that because they don't build world model inside so they can't deduct it from pre-built world/object model, they must memorize it and look it up later.

	▲	pixl97 20 hours ago \| parent [-]
		>but LLMs are not animals, animals learn from experience and LLMs don't. I mean they kind of do by distilling said experience and putting it in the next model. Current LLMs can't do it because building said world model is super expensive, anything that lowers that requirement brings us closer to continuous learning. Also the models we train these days are typically generalized human text models. Animal models are a bit different because they won't be "word" models and they aren't going to be generalized text models like us humans use. In fact there is a story just today on HN about a user creating some rather simple transformer models to solve a number of the Arc-AGI problems not using (human) words at all. The reason you don't see more of this is most people aren't dumping compute into these kinds of issues, but instead going for the AGI prize.