Remix.run Logo
▲ mef51 11 hours ago

If models are trained on human texts that contain expressions of introspection and emotions, why is it surprising that these models output similar texts? This endless anxiety about LLMs being conscious feels delusional and psychotic. Mr Olah thinks the models are 'alive' because he can't recognize his own reflection in LLMs

▲ACCount39 11 hours ago | parent | next [-]

The endless insistence that LLMs cannot possibly be conscious just reeks of insecurity to me.

Humans once again insisting that there must be something very special to them. That they're not just animals, that they're not just matter, that they're not just a bunch of carbonhydrates on a space rock in a middle of nowhere in particular. This kind of insistence has a very bad track record.

The ugly truth is: we don't know.

We don't know what "consciousness" is, our best attempts to pin down the requisites may or may not be rooted in anything at all - and even by the metrics established by those attempts? Different theories on consciousness disagree on whether LLMs can be conscious.

So, when you say "they aren't duh", why are you saying that? Because you somehow know better than the sum of humanity's best attempts so far? Or because you know what answer you want to be true, truth be damned?

▲jpc0 9 hours ago | parent | next [-]

If AI is conscious then what about it is conscious? Because turn to turn it might be served from completely different GPUs and I can just change something in the context it is fed and it will have completely different output.

It is in the end a set of mathematical operations that if not queried will do nothing.

That is not true for any other animal on earth so whether you consider humans as superior to animals or not I would like the make the argument that something should at the very least be alive to be considered conscious, ignoring any scholarly definition of consciousness.

Pretty much if you consider an LLM conscious then my pocket calculator doing a mathematical calculation should be considered conscious by the same argument since it is a simpler version of an LLM.

▲hardbass 5 hours ago | parent | next [-]

What part of you is conscious? I can randomly turn off various neurons of you and check. What atom of a piece of metal has the temperature of 20C?

▲tatjam 4 hours ago | parent [-]

This is a good take, but where does one consciousness end, and another begin?

Does my consciousness extend only to the central nervous system (and peripheral nerves and such are inputs)? Or am I my brain + all nerves? What about organs, does my consciousness include my liver? Is my liver another conscious entity? Going the other way, if I'm using a computer, does my consciousness extend to the computer? I use a computer for well over 8 hours a day, I can use vim motions more naturally than, say, move my earlobes or control my toes. Does that make it part of my consciousness?

Sadly, I don't think any answer to these questions can be achieved in our current times, or maybe ever if consciousness is somehow non-physical. So even the very reasonable claim that consciousness is an statistically defined property (like temperature) is too far fetched for our current knowledge.

▲hardbass 4 hours ago | parent [-]

Indeed thats the difficulty. I used to think potentially conscious AI systems are so far into the far future scifi that I never have to bother, but unfortunately real life forced me to think of these much sooner than I wanted. Its no longer quite a purely academic question now anymore due to the potentiality of current AI being conscious.

One thing however we can be sure of, "supernatural" stuff is an extremely low to near impossible probability given current knowledge and the fact that history has always won the bet against supernaturalism throughout the centuries.

▲laichzeit0 7 hours ago | parent | prev [-]

Have no idea why you’re being downvoted as this is a very valid point.

Also, in principle everything a GPU does could be done on pencil and paper. You could distribute the entire forward pass to the whole of humanity and have them write it out. Even if it took millions of years, the point is it’s computationally reducible to hand calculation. Is that entire process now “conscious” as well? The only difference would be the clock rate.

▲winwang 3 hours ago | parent [-]

While this is an interesting point, I'd say that this is also true of a human brain.

▲laichzeit0 3 hours ago | parent | next [-]

It’s an assumption, with the LLM it is a fact. There is only one source of entropy, whatever the seed value is to any pseudo random number generator used for stochastic operations. Given those values, everything is deterministic. For the brain, it’s not at all obvious that this is true.

▲chrisjj 3 hours ago | parent | prev [-]

We have the LLM code.

You don't have the brain code.

▲ptrl600 25 minutes ago | parent | prev | next [-]

Nah, clearly spiders and fish etc. are pretty much as conscious as humans, and LLMs aren't conscious at all.

▲mef51 11 hours ago | parent | prev | next [-]

I actually don't care if they are, I'm saying the topic is a distraction and irrelevant at best. As you say consciousness is not well defined. What is well defined and well grounded in reality is the disruption these models have on society. I think the consciousness debate AI researchers are having is a dangerous ego-trip for the sake of avoiding responsibility for the damage being caused by their work.

▲rgovostes 5 hours ago | parent [-]

"AI researchers" have been having a "consciousness debate" since time immemorial. Here's Alan Turing on the subject in 1950: https://courses.csail.mit.edu/6.803/pdf/turing.pdf (pp. 22-23)

▲omnicognate 6 hours ago | parent | prev | next [-]

> reeks of insecurity to me

If you're going to try and debate something constructively this is not a good place to start from. I suggest avoiding making claims about the mental state and motivations of people you disagree with and instead explaining your position and asking why theirs differs. You may not agree with what they have to say but you might gain some better insights into why they think the way they do. It'll certainly be more fun than flinging insults back and forth anyway.

For my part, I'm quite certain that LLMs are not conscious despite being unable to define consciousness or specify any conclusive test for it. The reason these are not in conflict is down to the nature of consciousness and how it relates to knowledge and language.

Firstly, I know I am conscious with a higher degree of certainty than I know anything else because conscious experience is how I know anything else, up to and including that I exist (cogito ergo sum). For all I know every experience I have might be a trick. I might be a brain floating in a tank wired up to the Matrix, but even if so I am conscious.

Secondly, one of the things that I consciously experience is the relationship between consciousness itself and language in my mind. What I see is that language is smaller than, secondary to and somehow contained by consciousness. I consciously experience language but I also consciously experience much more than that, things that cannot be conveyed with language. Most of my experiences are like that, things that I could perhaps evoke in your mind by likening them to other, shared experiences, but that I could never concretely capture in language and unambiguously communicate. I know this because I experience those things and I experience language and I can perceive, directly, the categorical distinction between them.

To ask me to "define" or "prove" consciousness, then, is to ask me to capture in language the most extreme example of this categorical distinction. It's asking me to encapsulate in language not just one of the vast array of everyday experiences I have that cannot be so captured, but the entire thing, experience itself. It's just not possible.

But the fact I can't do this doesn't then imply that I don't know anything about consciousness and have no basis on which to hold a view such as "LLMs are not conscious". Indeed the reason I know I can't define consciousness is that I can perceive it and see that I can't. That same perception of and familiarity with consciousness is what enables me to form opinions about whether other people, creatures and things are likely to be conscious. I base these opinions on what I perceive of my own consciousness and what I know of how those other entities behave, came to be and work.

You can of course perfectly reasonably disagree with my opinions on those things, and you certainly have different conscious experiences than I do, but I hope the above makes it clear that the fact that I can't define or prove consciousness doesn't imply that I have no basis on which to form opinions about it.

Given the above it may surprise you to hear that I think true, conscious artificial intelligence is probably possible and that while I don't think it can be implemented by a classical algorithm I doubt it requires new physics or quantum mechanics. My speculation is that the continuum is both physically real and is (as it is represented in our existing physical theories) sufficient to allow it. Consciousness is larger than language but so is human understanding and if we keep going I expect it will eventually get us there. Nonlinear dynamics is where I'd look.

As for insecurity, I'm pretty sure spiders are conscious and that hasn't given me an inferiority complex, so I don't see why data centres being conscious would. I just don't think they are.

▲chrisjj 3 hours ago | parent | prev [-]

> The endless insistence that LLMs cannot possibly be conscious just reeks of insecurity to me.

> Humans once again insisting that there must be something very special to them.

Really? I'm sure humans genrally accept they aren't the only conscious animals.

▲chrisjj 3 hours ago | parent | prev | next [-]

This. LLM consciousness belief goes with TV is a magic box containing tiny actors performing on demand for the viewer.

▲colordrops 11 hours ago | parent | prev [-]

Is your comment anything other than an emotional outburst? Do you have a clear and rigorous and testable definition of consciousness? It strikes me as delusional when people talk about "consciousness" as if it were well defined and everyone had a clear concensus on it.

Strong opinions on this topic one way or another are usually mostly ungrounded.

▲mef51 11 hours ago | parent [-]

Thats my point, why are we applying the ill defined concept of consciousness to a statistical model of language that outputs data similar to its training data?

▲hardbass 5 hours ago | parent | next [-]

Why are we applying ill defined concepts of consciousness to a collection of molecules sharing electrical signals that outputs similar data to what it was trained as?

▲colordrops 10 hours ago | parent | prev [-]

Are you suggesting that consciousness should not be discussed at all in scientific contexts?