AI simulates reasoning by lighting up the vector space (or concept space) weighted around a token so they it understands all adjacent words or concepts in that space.

This is a brillaint way to simulate reasoning, but its likely not how we reason ... simply how we store reasoning in writing.

Its useful if you know how to use it, its dangerous if you think its more than that.

But tl;dr it can (since its uncompressing our lingusticially stored reasoning from books) arrive at reasoning a DIFFERENT way than our brains did... and this isn't right or wrong.

Where it diverges is when it must move beyond the text or even the synthetic possible text of all vector spaces combined (aka novel territory) and it can't conjecture or test those outcomes well. But to be fair, neither can MOST humans.

▲

pohl 2 hours ago | parent | next [-]

I'm not convinced—and certainly don't find it obvious—that this couldn't ultimately also be how we reason as humans.

It's clear that there's an enormous amount of leverage built into language-as-practiced that one can use to engage in a broad spectrum of reasoning, from the extremely fallible off-the-cuff conclusion to the deeply-considered and rigorous proof. How do we know this leverage is built into language-as-practiced? Because LLMs can do a broad swath of it.

But how do we know we're not doing something similar?

I don't think we can assume that we're not simply by observing that we're not digital and we don't use matrix multiplication. Why immediately dismiss the possibility that there might be a similar, but biomechanical, computation at play in our heads that plays in the same space of vectors?

▲

tsunamifury 2 hours ago | parent [-]

humans reasoned before language but lacked the ability to store and transmit it. Later advanced humans developed abstract reasoning once lingustistic library became sufficiently description of reality. But this is not how we reason from first princples.

Language is one of our tools we developed to STORE reasoning, not create it. LLMs excel at uncompressing and interpreting that stored reasoning.

▲

pohl 2 hours ago | parent [-]

> humans reasoned before language

That's an interesting supposition. Are you assuming language didn't exist before it was written? Language and meaning are, if you squint, pretty ancient and have roots in things like birdsong. It could be that ur-semantics predates our species as a whole.

▲

emp17344 2 hours ago | parent [-]

It’s pretty clear that animals that don’t use a system of language are capable of reasoning. No need to over complicate things.

	▲	pohl an hour ago \| parent [-]
		Good observation. I’m certainly not trying to over complicate things. That just tells us, though, that the bio-mechanical computation part is there before language, which I took as a foregone conclusion, it being meatware and all.

▲

bohoo 3 hours ago | parent | prev [-]

Can you tell me your reasons for suspecting this likely isn't how we reason, or even a good analogy?

	▲	tsunamifury 2 hours ago \| parent [-]
		Its likely part of how we reason, but quite obviously its not the specific mechanics exactly. First, we reason every millisecond on an ongoing basis which then can alter slightly or greatly with enviromental feedback. LLMs are turn based and token by token. Second its pretty unlikely that the token is the base element of our cognition, we created language far after we could do basic reasoning (advanced reasoning ala the greeks thats more debatable). Theres a ton of research on the differences here, but I think its akin to this: we reason instinctually at an extremely high order level with super undefined "grains or vectors" that point to a wide variety of "objects or concepts or feature spaces". LLMs reason on one thing, token weights. Sort of like the difference between pixels and reality. Pixels can represent reality, but they certainly are very very very flat and low resolution renderer of them, not reality itself. Even a 4K moving image is a flat redition of reality at best with only a tiny sample of the true experience. Media theory here can take over on the differences and the effects on humanity when they mistake one for the other.