Remix.run Logo
▲ scribu a day ago

I thought theory building WAS the process of creating powerful abstractions?

▲TheOtherHobbes a day ago | parent [-]

Open question - are these proofs brute forcing, or are they generating new theory at a level of abstraction and complexity we can't understand?

Controversial statement: I think we've always done math by generalising from concrete experience. So concepts like numbers, spaces, shapes, distances, and curvature are based on physical experiences we use as anchors before expanding them far beyond physicality and looking for patterns in the abstractions.

If AI is generalising from patterns in latent space, that's a different kind of abstraction, and one that's much more alien to us, because it's going to be much, much bigger - too big to fit inside even the smartest human mind.

So AI theories and abstractions may be valid in true/false terms, but we will never be able to comprehend them in their raw form - just as we can't experience latent space directly, and will only experience tiny slices of it as outputs to our queries.

▲agnosticmantis 17 hours ago | parent [-]

> If AI is generalising from patterns in latent space, that's a different kind of abstraction

How's latent space conceptually different from neuronal activations in our brains?

> just as we can't experience latent space directly,

Your LLM also doesn't "experience" their activations in the latent space either when generating tokens.

I'm not saying LLMs are not conscious/sentient/etc. Just saying if you ask your LLM which activation was stronger that led to a particular solution, they won't be able to answer without using external tools to inspect it.

It may just be that LLMs have such a huge context window (short-term memory) compared to humans, that they can process a lot more raw data without having to compress them into theories yet.

▲teiferer 15 hours ago | parent [-]

Human short-term memory and LLM context windows are very different things.

More generally, your mental model of how the human brain works seems to be very much aligned with how LLMs work and I would very much caution against that.