| ▲ | The Mythology of Conscious AI(noemamag.com) |
| 15 points by Anon84 2 hours ago | 32 comments |
| |
|
| ▲ | nlsvb a few seconds ago | parent | next [-] |
| I have trouble taking seriously any claims that LLMs are conscious in any reasonable sense. I’ll try to make a few arguments quite succinctly. 1. There is zero dynamic nature to anything happening within the purview of the LLM. All dynamic elements come from external dynamic systems. E.g. every input produces the same mathematical output. We dynamically sample the output in order to produce dynamic results for the same input. For agentic use cases, the dynamic nature comes from tool calls to dynamic systems. All outputs for any set of model weights could be theoretically computed with pen and paper, so unless you were to claim such a process would be conscious in some sense, then emergent consciousness would have to come from some interaction between the substrate and the computation. But… 2. For efficiency, forward inference passes are done in batches. I.e. 100 different next-word predictions are being performed by the network at a time. For a single request, its batch position will not even be stable between each autoregressive step. The GPU is on 100 phone lines at once and each line switches conversations every step. Worse, with MOE, the attention and feed-forward parts aren’t even happening on the same GPU, even within the same step. There is no stable substrate here whatsoever. 3. There’s obviously the whole Chinese Room argument. The token embeddings being processed by LLMs are arbitrary high-dimensional vectors. The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model. In their current state, no argument about models developing world models based on this geometric structure makes any sense. There are infinitely many inputs / worlds that would produce isomorphic geometry. |
|
| ▲ | hax0ron3 an hour ago | parent | prev | next [-] |
| Consciousness might not necessarily emerge when there is enough computation, but we also don't know that it doesn't emerge. We really know almost nothing about consciousness. It's possible that the question of why the presence of consciousness is correlated with certain kinds of material organization is in principle not accessible to scientific investigation. We don't even know that it is so correlated in general, we just know that for example people report being conscious when they are not asleep and not drugged in certain ways, and report having had no experience of consciousness during dreamless sleep or when drugged in certain ways. Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches. |
| |
| ▲ | soon_to_be_sb 19 minutes ago | parent | next [-] | | >Consciousness might not necessarily emerge when there is enough computation Does a circle "emerge" only when you plot it? Or did it always existed? What if there is a little world with conscious beings inside a circle? Did their subjective experience started when you plotted the circle, or did it exist along with the abstract idea of a circle always (though "always" is a meaningless word here). The question we should be asking is "Is it possible for subjective experience to exist inside abstract things" with out such abstract things being rendered? The answer is hard because we does not know yet if there is an absolute reality at all, which means "being rendered" does not really move things from the abstract plane, and does not really make a difference.. Damn! did any of that make any sort of sense? | | | |
| ▲ | za_creature 23 minutes ago | parent | prev [-] | | https://writings.stephenwolfram.com/2024/10/on-the-nature-of... Stephen Wolfram suggests an intimate relationship between awareness, thermodynamics and the one-way passage of time. Something along the lines of "consciousness is the universe understanding itself". | | |
| ▲ | addaon 9 minutes ago | parent [-] | | > Stephen Wolfram suggests So that's one strong argument against it, but I don't think it's sufficient to discard the idea entirely. | | |
|
|
|
| ▲ | talon8635 10 minutes ago | parent | prev | next [-] |
| It’s a silly debate when we don’t even know what consciousness is, let alone the circumstances for its occurrence. Or am I missing something? |
|
| ▲ | jibalt an hour ago | parent | prev | next [-] |
| > Why consciousness is more likely a property of life than of computation This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious. And neither "life" nor "consciousness" are scientific terms with fixed definitions. If we encountered something that was clearly conscious in our judgment, we would likely expand the definition life in whatever way we needed to in order to encompass it. Anil Seth has made the arguments in TFA many times and they have been rebutted many times. The arguments are complex and sophisticated and people are often most swayed by which one they read last. Teasing out the various logical errors and sleight of hand (I see multiple strawmen and other errors in TFA) really requires extensive reading of the literature in Philosophy of Mind, neuroscience, robotics, etc. and setting aside one's priors. |
| |
| ▲ | rramadass 2 minutes ago | parent | next [-] | | Generally when people say "consciousness is an emergent property of life" they mean "at a certain scale and through specific combinations". See also the recent discussion on The Evolving Story of How Chemistry Became Biology - https://news.ycombinator.com/item?id=50482222 The basic idea comes from "Complex Systems Science" (https://en.wikipedia.org/wiki/Complex_system) which studies systems as a whole whose properties cannot be predicted from knowledge of its components. The main factors to consider are scale, non-linearity, feedback loops, self-organization, adaptation/evolution, network structures, emergence etc. So for example, a single bacterium cannot be said to possess "intelligence" whereas a whole colony of them do in the sense that they seem to act as an integrated whole "single" organism. | |
| ▲ | hax0ron3 42 minutes ago | parent | prev | next [-] | | > most life (bacteria and insects, by mass, numbers, or species) isn't conscious. There's no good scientific reason to think that bacteria and insects are conscious or to think that bacteria and insects are not conscious. | | |
| ▲ | jibalt 35 minutes ago | parent [-] | | There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for. > We really know almost nothing about consciousness. Many people who know almost nothing about it claim this, but it isn't true of everyone. > Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches. There actually is. P.S. > Such as what? Like I said, read the literature. P.P.S. Science does not consist of HN comments back and forth. Again, the arguments are complex and sophisticated. In some sectors, all of medical science, physics, biology, etc. is dismissed as "hand-waving" just because it cannot be summarized in a few lines. > It's impossible to argue otherwise. Then don't. Science isn't based on arguing. | | |
| ▲ | mellosouls 11 minutes ago | parent | next [-] | | You're being very rude and dismissive, and not backing up your claims with any evidence. Nobody knows what consciousness is, and there is a long history of some schools considering it fundamental. | |
| ▲ | hax0ron3 29 minutes ago | parent | prev | next [-] | | >There certainly are good scientific reasons to think they are not Such as what? | | |
| ▲ | za_creature 11 minutes ago | parent [-] | | For insects, it's the absence of higher order thinking: A fly will repeatedly smash into a window until it's either dead or chaos drives it through the nook. Ants will forcefully move their obviously still alive siblings to the graveyard because their smell is off, and will also die of exhaustion in an ant mill, driven by the same sense of smell. It's a spectrum of course, though you have to draw the line somewhere. | | |
| |
| ▲ | mrbungie 21 minutes ago | parent | prev | next [-] | | > There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for. It would be better if you could enlighten us with the scientific evidence you are proposing as relevant rather than just telling us to just read it. It's impossible to argue otherwise. EDIT: Since you are answering in the parent comment > Then don't. Science isn't based on arguing. OK? That seems like an extremely inflexible position (for example, methodological criticism during peer review is a form of argumentation, and it's a critical part of science), but let's accept that there's no room for rhetorical arguments. There is still a scientific discourse based on weighing evidence and you are offering none. | |
| ▲ | operatingthetan 28 minutes ago | parent | prev [-] | | Way too much hand-waving here. Okay, great you know the answers. Make compelling arguments then. |
|
| |
| ▲ | talon8635 8 minutes ago | parent | prev [-] | | How do you know bacteria and insects are not conscious? How anyone speaks with certainty on this topic baffles me. |
|
|
| ▲ | ChaitanyaSai an hour ago | parent | prev | next [-] |
| If you are interested in consciousness, and also interested in going beyond the usual philosophical approach, you might like our book Journey of the Mind https://www.goodreads.com/en/book/show/60500189-journey-of-t... One thing that is implicit, which I now think we really should have expanded on in the book, is that consciousness is fundamentally an engineering problem. And that's also why it's mystified everyone who's brushed away the engineering as the easy plumbing and focused on the other seemingly magical aspects. The fundamental question is phrased as "Why does it feel like something?" it being perception, which is essentialy computation. Now that AI does look like it does all of the understanding we do, many including people like Hinton think they have consciousness or are very near it. Well, they are capable of simulating consciousness but that is not the same as experiencing it. To answer "Why does it feel like something?" we can focus on what's been glossed over: Who is feeling it? The "I" does not come from as a free lunch and it must be constructed moment to moment from some decentralized cohesion and consensus of a billions of neurons. And this consensus must be metabolically cheap, and we need to do it again and again for everything from feeling a pin prick to hearing a rustle and determining if that's a snake or a leaf to pondering thought. This is the insane engineering, the energtically cheap chorus that creates the transient I and this cheap chorus is where both feeling and feeler emerge. Perception, understanding, computation can occur without this cheap chorus, but then there is no feeler, and so no feeling. What this points to is that we will have conscious AI when we have decentralized autonomous swarms where a few million of them have to cohere, collude, collaborate, conspire. My worry here is that it will first be done for harmful purposes (autonomous military drone swarms) and not for anything good. |
| |
| ▲ | mellosouls 5 minutes ago | parent | next [-] | | consciousness is fundamentally an engineering problem It's not at all clear it's even scientifically resolvable - rather than remaining a philosophical question. | |
| ▲ | hax0ron3 43 minutes ago | parent | prev [-] | | We know that self-reported consciousness is correlated with some material states (for example, an awake human body). But that's about all that we can solidly say about how consciousness relates to matter. We know how to create intelligence, but it is possible that a human-level intelligence can exist without ever being conscious. It is also possible that any human-level intelligence automatically achieves consciousness. We don't know. There is actually no good reason to believe that consciousness is an engineering problem. The "I" is not necessarily correlated with consciousness. Those are two different things. | | |
| ▲ | soon_to_be_sb 31 minutes ago | parent [-] | | >self-reported consciousness is correlated with some material states (for example, an awake human body). Wait, but what about dreams? I could self report I am conscious when I am in a dream.. | | |
| ▲ | hax0ron3 30 minutes ago | parent [-] | | You're right, also dreams. Just not dreamless sleep. I didn't mean "an awake human body" to be exhaustive. |
|
|
|
|
| ▲ | andsoitis an hour ago | parent | prev | next [-] |
| Consciousness in others isn’t something we prove. It is something we grant and we should be very careful about whom or what we grant it to. |
| |
| ▲ | greg7mdp 14 minutes ago | parent | next [-] | | Why is consciousness something we grant? Who granted us consciousness? | | |
| ▲ | za_creature 6 minutes ago | parent [-] | | others :) --- Bad phrasing from parent post, but the premise is sound: you can't ever disprove that others are just NPCs in a giant experiment centered around you. We just assume they aren't with various levels of empathy. Most are empathic to their family, some to their tribe, others to their species and then there's vegans too. |
| |
| ▲ | jibalt 43 minutes ago | parent | prev [-] | | We should also be very careful about whom or what we deny it of. |
|
|
| ▲ | bionhoward an hour ago | parent | prev [-] |
| how dare he tell us we cannot make a computer out of cheese personally I think conscious AI will be unavoidable because there will be benefits to consciousness. evolution probably selected conscious entities for some fitness reason |
| |
| ▲ | jibalt 40 minutes ago | parent | next [-] | | Beaks, feathers, hooves, antlers, stingers, fangs, etc. ad nauseam were selected by evolution for some fitness reason but we have none of those. And all of the organisms that have those, in fact all extant organisms, are fit for their environment without having human consciousness. Fitness is highly contextual. And however AI "evolves", it won't be based on Mendelian genetics, so Earthly biological evolution is not an appropriate model. | |
| ▲ | card_zero 43 minutes ago | parent | prev [-] | | + + + Out of Cheese Error + + + + + + Redo From Start + + +
Those "benefits" and "fitness" in evolution just mean survivability, I don't see how you relate that to better AI. Surely a conscious AI is one that complains, and maybe goes away somewhere to find itself instead of working for you. | | |
| ▲ | greg7mdp 12 minutes ago | parent [-] | | Why would a conscious AI have to go away somewhere? Typically it is the less advanced beings who have to relocate. |
|
|