This is the same reason why I am a bit skeptical of LLM superintelligence. LLMs in the end have to operate in natural language concepts and the complexity of natural language is bounded by limits of human cognition. I'm sure super advanced AI could use concepts that humans not only have no words for, but might not be able to understand alltogether. As such if my thesis is correct, the only way forward for true superintelligence may be getting rid of natural language COTs.

▲

nomel 4 hours ago | parent | next [-]

> LLMs in the end have to operate in natural language concepts and the complexity of natural language is bounded by limits of human cognition.

I don't think this is true.

They have to express themselves as tokens. The meaning of those tokens doesn't have to be text. See any model that can handle images/video. Also, I don't think math, svg, etc, are "natural" language.

And, only the final expression is tokens. The intermediate layers, with the encoded concepts, aren't "natural language".

But, to address your concern (which nobody can disagree with, since even humans can't fully express through text/pictures), potentially: https://news.ycombinator.com/item?id=49758615

▲

_puk 3 hours ago | parent | next [-]

Yeah, as I understand it, natural language is tokenised and vectorised, and then maths takes ahold.

The model isn't limited to concepts that can be expressed in natural language.

It's only once the AI gets to the output layers that natural language comes back into play.

After all, they're all made out of weights[0].

0: https://maxleiter.com/blog/weights

▲

timacles 2 hours ago | parent | prev [-]

Reality cannot be reduced to tokens

	▲	nomel an hour ago \| parent \| next [-]
		Can it be reduced to ion concentrations? Because that's how we perceive it. A useful perception is all that really matters.
	▲	Dylan16807 2 hours ago \| parent \| prev [-]
		Thoughts are a poor reflection of reality to begin with.

▲

booty 3 hours ago | parent | prev | next [-]

There are two separate questions, right?

1. Is natural language holding LLMs back by some %? 2. Is natural language serving as a hard gate that will prevent LLM intelligent progressing past some specific point?

The answer to 1 seems like an obvious yes to me.

Your thesis says the answer to 2 is "yes." That doesn't feel right to me. Think about all of the humans who have pushed various fields forward: Einstein, Newtown, Bach, whoever. If natural language doesn't prevent an entity from surpassing humans in one intellectual field, why would it prevent an entity from surpassing humans in all intellectual fields?

(To be clear, I'm not claiming superintelligence will or won't be achieved; I'm considering your specific thesis about whether or not natural language will be a hard gate)

▲

indigo945 2 hours ago | parent | prev | next [-]

Strong Sapir-Whorf making a comeback wasn't on my 2026 bingo card.

By the way, how good is Claude's Hopi?

▲

dist-epoch 3 hours ago | parent | prev [-]

Each letter has it's own token. LLMs can invent new words if they want to, just like humans do. All the fancy words we use today in science did not exist even 100 years ago. And that didn't stop us.

	▲	faeyanpiraat 3 hours ago \| parent [-]
		How would this word inventing thing work? And how useful would it be? Can we make a POC around this?