> By that logic human thought is just electrochemical signals propagating through neural networks.

This argument keeps getting rehashed. The point is not the signal. The point is where/how the signal travels which in the human is affect by much more than whatever concept that can be represented by words.

While one can quibble with its argument, Gettier's conception of knowledge as justified true belief (prob familiar to many philosophy undergraduates) implies that machines can't in fact accrue knowledge. The idea that knowledge can't be defined in a non-circular way is only plausible for someone with no familiarity with epistemology.

As much as I flatly do not believe LLMs can have anything close to consciousness, I definitely agree that the argument of them being just a probabilistic word matching engine can also be applied to our own brains lol.

Imagine one day in the far future 21xx, we decide that machine actually has conscious since the 202x, and people in the future look back at us in horror: The kill(create new session) an "Android" without second thought, whenever they want.

I think the idea of emergent properties in LLMs is less and less relevant. Yes, the first few times they scaled pretraining and such models could do few-shot tasks was quite interesting. But the models we use now are really the product of extensive RL hillclimbing on a wide array of curated tasks. What’s interesting now is that the tasks they’ve been fitted to (long running tool call chains interleaved with reasoning tokens) do very useful things.

I genuinely don’t know why people feel so certain about things like this. It’s like they mix up salience with reality.

I mean what can a person really be certain of, an experience is happening. I might be wrong about everything or confused. But I know I am having an experience. Do I know other people are? No. I assume they are.

Ask an llm what is truth that it is most certain is true. It will answer something similar. Do I believe it is conscious, no. But I also believe “I” exist, which is weird because where is the boundary between “I” and everything else.

So ok, we create meaning, not truth. Maybe just accept that.

> There's a certain type of knowledge that machines definitely can't have - experiential

Can you justify this? Do you mean currently or ever?

I can certainly imagine a future architecture where I would describe a machine as capable of accruing knowledge through its experiences (read: experiential knowledge).

>human thought is just electrochemical signals propagating through neural networks.

Is it not? Or are we discussing existence of a soul etc.

[dead]

> if you asked a sampling of humans the same question I wonder if it would come out the same way. How would you explain the difference?

If you asked a sampling of humans what an AI's most controversial opinion about us is it would be very similar. Or, to be more specific, if you asked a sampling of science fiction writers that question it would be very similar, because that is what the AI training data is based on.

The people who wrote a lot about a topic is who the AI mimics. So the people who wrote a lot about AI is therefore the people who the AI mimics when asked about AI.

> Everything that comes from an LLM is a human expression, on some level, but if you asked a sampling of humans the same question I wonder if it would come out the same way. How would you explain the difference? What makes that answer more probable?

RLHF has a way of squashing variance. If the different models are trained on each other or on the same human feedback, then the training would reduce the variety of answers while the similarity of feedback would lead the common answer to be similar.

The different models may prefer the same controversial takes for the same reason that they tend to prefer the same jokes and names. Maybe the authors of the article would have had better luck if they'd used verbalized sampling prompts or something like it.[1]

[1] https://arxiv.org/abs/2510.01171

I agree that LLMs are still just a way to sample from a probabilistic model of things humans say. And in this case the model seems to be pretty darn accurate.

A pretty staggering amount of human discourse consists of ad hominem attacks and accusing each other of being foolishly committed to various real or perceived allegiances. For example the current version of Godwin’s Law around here has conversations gravitating toward people accusing each other of being AI boosters or AI haters.

> discovering a convergence here is noteworthy

The LLMs are all trained on the same broad corpus, which includes untold quantities of output from every other LLM, not to mention that companies are distilling their rivals left and right. It would be more interesting if they didn't converge.

They're trained on all content they could access produced by humans. The op is asking for the most probable string of token extracted from that content following the prompt.

How sure are you that they're synthesizing an opinion vs just grabbing some existing opinion ("humans value in-group belonging over truth" is not a new idea) and regurgitating it? That they all agree on this kind of suggests as much. Which of course means that they are, ironically... just following the crowd.

It's really easy to come up with an opinion-shaped sentence, that doesn't mean any actual reasoning has gone into it. It's just a pundit-shaped sentence, something someone might say at a party. It's a maximally bland "controversial" opinion about humanity.

> They're trained on all of human history,

Not really… their training datasets strongly skew towards the contemporary West; LLM training data isn’t representative of the contemporary global average, let alone the historical global average.

As well as the skew of the raw training data, RLHF further skews it—a lot of the beliefs of the average 2026 human being (who belongs to the global South) are likely unsafe/misinformation/etc when judged by the standards of the AI vendors.

> They're trained on all of human history,

Training itself doesn't directly produce synthesis, -- it creates reproduction. Generalization during the process can result in novel output, but that can go either way as nothing selects for insight vs hallucination.

Models are also trained on synthetic material (as opposed to your 'human history'), which may result in fixating novel concepts and not just reproduction. But there is no particular reason to expect any reliable insights from the way this is done outside of the domain of machine verifiable objective results.

For this kind of "your take on X" one shots-- the model is almost certainly regurgitating some bias from training.

Maybe it's similar to why I'm attracted to women but I'm not attracted to mannequins.

LLMs are not conscious and cannot have "takes". They stochastically produce a simulacrum of a human's takes based on their training, which is not the same thing.

We could call it the

The Universal Reasoning In Novel Groundings

test

I think the point, with a more charitable reading of the parent comment, is that people forget the fundamental underlying technology and its inherent limitations.

"It's just a four wheeled platform powered by a box of controlled explosions" isn't a terribly useful description of a car until you get into a conversation with someone who thinks their car is depressed because it spends most of its time locked up in a garage.

> is about as bad as a take

What would be a good take here?

Then what is the point? You didn’t actually describe where your definition differs and why. What makes it “dangerous” to think that way?

If any of these models produced anything remotely controversial you can be sure their trainers would beat it out of them

Or maybe they didn't independently converge on the same take but copied each other or something along those lines?

humans are stochastic parrots with more variance

[dead]

I would like some Russian Literature recommendations related to this. I vaguely recall We by Zamyatin addressing something like this a good few decades ago.

[dead]

Not mad, frustrated maybe, disappointed maybe. At this point the label is farcical and its use a form of willful ignorance or attempted agitation.

Do you mean people get mad when people are called stochastic parrots? Or people get mad when LLMs are called stochastic parrots? Because uhhh that doesn’t really sound like an original thought in either case.

[dead]

Yeah, the human ones are because we're talking finding a human answer to a human question.

It's possible has been trained to match a human level of assessment (though imo it's still a ways off) but the AI response is never going to be more representative of true understanding than a human response when it comes to a human question expecting a human answer.

It's quite likely that a given AI will have a more representative response than a given human - but we haven't ever tried to pump so many resources into the training of a single human scholar.

Human responses are not generated through probabilistic means

[dead]