> LLMs don’t really reason

Do you have a test for this?

Or is it based on the presumption that reasoning skills cannot evolve, it can only be the result of "intelligent design"?

▲

jbritton 10 hours ago | parent [-]

I have been spent many hours with them on coding tasks. As things currently stand once context or complexity reaches a certain point they become completely incapable of solving problems and that point occurs on very simple things. They appear completely brain dead at times although they are magnificent liars at making you think they understand the problem. Although, I recently got chatGPT 5 to solve a problem in a couple hours that Claude Sonnet 4 was simply never going to solve. So they are improving. I don’t know the limits. I’m more hopeful that a feedback loop with specialized agents will take things much further. I’m extremely skeptical that getting bigger context windows and larger models is going to get us reasoning. The skepticism comes from observations. Clearly no one knows how thinking actually works. I don’t know how to address the evolve part. LLMs don’t directly mutate and have selective pressure like living organisms. Maybe a simulation could be made to do that.

▲

jbritton 7 hours ago | parent | next [-]

Gotham Chess has done chat bot chess championships. The chat bots make a few good moves, and then begin making illegal moves, randomly removing or adding pieces, and completing ignoring obvious threats and attacks. It is so obvious that the pattern matching is not resulting in reasoning. Another example is Towers of Hanoi. An LLM can write code to solve it, because that’s an easy pattern match. But it can’t write out the steps beyond a 3 disk puzzle. It has no understanding of the recursive nature of the problem.

▲

handoflixue 8 hours ago | parent | prev | next [-]

Would you agree that many humans, especially younger ones, are also incapable of reasoning?

▲

addaon 7 hours ago | parent [-]

I think it’s completely uncontroversial that younger humans are incapable of reasoning, no? The only area for discussion is at which age (if any) this changes for an individual.

▲

hattmall 7 hours ago | parent [-]

I would say that's a pretty controversial take unless you strictly mean babies. At the point a kid can walk they are certainly capable of some level of reasoning because just walking successfully prior to it becoming muscle memory requires a fair amount of reasoning and children very rapidly figure out new things both spatial and logical.

	▲	addaon 7 hours ago \| parent [-]
		So if you believe babies do not reason, and walking humans do, then younger humans up to some age do not reason, yes?

▲

whattheheckheck 6 hours ago | parent | prev [-]

Can you post your experiments please?

	▲	jbritton 5 hours ago \| parent [-]
		I don’t have a log of work that I can post.