This is interesting philosophy, and others have better critiques here in that regard. I'm a mathematician, so I can only work in what I can define symbolically. Humans most certainly ARE statistical models by that definition: without invoking the precise terminology, we take input, yield output, and plausibly involve uncertain elements. One can argue as to whether this is the correct language or not, but I prefer to think this way, as the arrogance of human thinking has otherwise failed us in making good predictions about AI.

If you can come up with a symbolic description of a deficiency in how LLMs approach problems, that's fantastic, because we can use that to alter how these models are trained, and how we approach problems too!

> What that means in practical terms for programming problems of this kind is that, we can say "I don't know", which the LLM can't, because there's no "I", in the LLM, no unified subject that can distinguish what it knows and what it doesn't, what's within its domain of knowledge or outside.

We seriously don't know whether there is an "I" that is comprehended or not. I've seen arguments either way. But otherwise, this seems to refer to poor internal calibration of uncertainty, correct? This is an important problem! (It's also a problem with humans too, but I digress). LLMs aren't nearly as bad as this as you might think, and there are a lot of things you can do (that the big tech companies do not do) that can better tune it's own self-confidence (as reflected in logits). I'm not aware of anything that uses this information as part of the context, so that might be a great idea. But on the other hand, maybe this actually isn't as important as we think it is.

The position of Kant does not align with the direction modern neuroscience is heading towards. Current evidence seems to prefer decentralized theories of consciousness like Dennett's multiple drafts model[1], suggesting there is no central point where everything comes together to form conscious experience, but instead that it itself is constituted by collaborative processes that have multiple realizations.

[1]: https://en.wikipedia.org/wiki/Multiple_drafts_model

It is kind of interesting you are arguing with a human about the ability to distinguish what is known and not known.

Where your claim is humans know and LLMs don’t, but the op is wrong and does not know what he does not know.

Kant was a dualist, of course he didnt think humans were statistical models. It just turns out he was (probably) wrong.

Does a chess engine know, think, or deduct?

> because there's no "I", in the LLM, no unified subject

is there in humans?

> Human beings are capable of

are they?

> without which the world would make no sense to us

does it?

I have never met a human who has a good grasp of what they know and don't know. They may have a better graps of it than an LLM, but humans are awfully bad at understanding the limits of our own knowledge, and will argue very strongly in favour of knowing more than we demonstrably do in all kinds of contexts.

I’m afraid that sense of knowing what you know is very much illusory for humans as well. Everyone is just slowly having to come to terms with that.

You are judging this based on what the LLM outputs, not on its internals. When we peer into its internals, it seems that LLMs actually have a pretty good representation of what they do and don't know; this just isn't reflected in the output because the relevant information is lost in future context.

Do you really know what you don't know? This would rule out unknown unknowns entirely.

> OpenAI researcher Noam Brown on hallucination with the new IMO reasoning model:

> Mathematicians used to comb through model solutions because earlier systems would quietly flip an inequality or tuck in a wrong step, creating hallucinated answers.

> Brown says the updated IMO reasoning model now tends to say “I’m not sure” whenever it lacks a valid proof, which sharply cuts down on those hidden errors.

> TLDR, the model shows a clear shift away from hallucinations and toward reliable, self‑aware reasoning.

Source: https://x.com/chatgpt21/status/1950606890758476264

Socrates would beg to differ

Unless what you work on is very simple and mostly mindless, using tools like Claude Code is the exact opposite of how to make the current SotA LLMs useful for coding. The models can help and boost your productivity, but it doesn't happen by letting them do more stuff autonomously. Quite the contrary.

And when what you usually work on actually is very simple and mostly mindless, you'd probably benefit more from doing it yourself, so you can progress beyond the junior stuff one day.

> If in 2022 I’d tried to convince AI skeptics that in three years we might have tools on the level of Claude Code, I’m sure I’d have heard everyone say it would be impossible because “it’s just a statistical model.”

We already had these coding models in 2022, they were already pattern matching engines with variables back then, all your imagination needed to do to go from that to Claude code today is to give it more code examples and make it bigger.

They still can't replace even a junior engineers ability to navigate tasks over even short periods of time, just like back then they need constant handholding to get anything done. So I don't see what changed except the model being larger with more examples and therefore you can get larger chunks of coherent code out of them.

People repeating the "stochastic parrot" meme in all kinds of variations if anything appear to be more like stochastic parrots than the typical LLM is.

I work in the theory of deep learning, so I can say with some authority that while we know a really good number of things, and are able to probe the internals much better than most of the public realises, when it comes to philosophical questions that compare their nature with humans, we have a long, long way to go. The biggest problem is that we're still trying to work out what it is we even want to know that will tell us whether we have achieved AGI or not.

Linear probes and autoencoders have been useful, but we're quickly reaching the limits of those techniques. And don't even get me started on approaches to theory that operate by cherry-picked examples. Anthropic's contributions have been beneficial to the field, but are far from conclusive.

> Humans build theories of how things work. llms dont. Theories are deterministic symbolic representation of the chaotic worlds of meaning

What made you believe this is true? Like it or not, yes, they do (at least to the best extent of our definitions of what you've said). There is a big body of literature exploring this question, and the general consensus is that all performant deep learning models adopt an internal representation that can be extracted as a symbolic representation.