You heard of JEPA? LLM's have all sorts of garbage they have memorized. Reasoning in latent space instead of in text significantly reduces the number of needed parameters.

▲

saberience a day ago | parent [-]

JEPA is a joke, let me know when those models do anything useful.

	▲	computerex 3 hours ago \| parent [-]
		You should look at some literature around it. I don't have time to pull it up now but it's been shown that much smaller small million parameters JEPA model outperforms much bigger LLMS in some applications. Keep in mind JEPA is area of active research.