Remix.run Logo
chychiu 10 hours ago

Considering that brainfuck only has 8 characters and models are scoring at 6.2% I don't think tokenization is the issue

altruios 9 hours ago | parent [-]

The only issue. *

Reasoning is hard, reasoning about colors while wearing glasses that obfuscate the real colors... even harder... but not the core issue if your brain not wired correctly to reason.

I suspect the way out of this is to separate knowledge from reason: to train reasoning with zero knowledge and zero language... and then to train language on top of a pre-trained-for-reasoning model.

onoesworkacct 7 hours ago | parent [-]

LLMs already use mixture of experts models, if you ensure the neurons are all glued together then (i think) you train language and reason simultaneously