| |
| ▲ | aspenmartin 11 days ago | parent | next [-] | | > With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!" Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise? Does anyone seriously consider a brain to be something other than “a pile of molecules following the laws of physics”? If so that’s not science or philosophy, that’s religion. You have a virtually complete phenomenological model of the universe for all intents and purposes and yet somehow the onus is on the person being like “hey no laws of physics are being broken ==> the brain is simply following the laws of physics” How is it possible that people think of subjective experience and get rabbit holed into some mystical world where subjective experience is this special exception to everything else that is simply an emergent property of complex physical systems? “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness? It’s more: consciousness is not some mystical or religious thing outside of the realm of physics, it’s an emergent property of a complex system. AI is a relatively complex system. We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that | | |
| ▲ | SlinkyOnStairs 11 days ago | parent | next [-] | | > Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise? They would have to be similar because the argument is that they are similar; "AI is like the human brain" only holds true if it actually is like the human brain, not merely a superficial resemblance. What I'm describing in that prior comment is how a lot of people drastically simplify the resemblance in order to make it feel true; That the lack of a Jesus Christ coming down from the heavens to tell everyone they have immaterial souls that the computers don't makes the comparison more true. > We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that Therein lies the conflict: "You can't prove it's not conscious" is an unfalsifiable statement. You can't engage with the argument because it's proponents will always claim victory, often with their own interests at play. All concerns about "superintelligence" or the long term ethics of "when do our robots become sufficiently intelligent that they'd be slaves?" have been subsumed into the AI marketing machine. However sincere one might try to address the issue, they look like a Sam Altman stooge by association. It's like claiming the quantum fluctuations inside a pet rock as a "consciousness", even if observed directly any measurement of random noise can still be dismissed as "nuh-uh it just takes billions of years to have a thought". More practically with current AI systems, we can look inside them pretty well and there genuinely is nothing there. Standalone LLMs are purely feed-forward systems. Their failure modes show that they perform no meaningful thought or world modelling during inference. They're just language models. The reasoning and agentic systems are even easier to introspect. We know how they work, we can look at the full prompts & context they operate on. There is nothing there. This is what sets AI apart from animals, which are given the benefit of the doubt on their intelligence. | |
| ▲ | wredcoll 11 days ago | parent | prev | next [-] | | > “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness? It doesn't need to be true but a lot of people make it/assume it. There's a lot of, perhaps casual and uninformed, conversations that strongly imply a deeper understanding of the "physics" of brain chemistry than we actually have, mostly by comparing it to machines we've constructed. (I believe) We don't need to replicate human neurons and dendrites and whatever else is in there in order to create a sapient "machine", but whether or not we've actually done that isn't being helped by arguing that what we currently have is all that similar to a human brain. | |
| ▲ | calepayson 11 days ago | parent | prev [-] | | I think the point of the commentator above is that there are two extreme narratives that start each start with an uncontroversial assumption and then taking it to a pretty wild place. One narrative takes the assumption that brains are just matter so it should be possible to engineer consciousness and then argues that LLMs are conscious. The other takes the assumption that LLMs aren't conscious but then argues that because they aren't we won't ever be able to make anything conscious. I don't actually think the commentator you responded to is arguing for either of these narratives and I thought it was a pretty useful way to look at some of these arguments. |
| |
| ▲ | sixeyes 11 days ago | parent | prev | next [-] | | oh yeah! i recall a paper linked here not so long ago, where it was shown that the dendrites of a neuron do computations themselves. The "weight per neuron" is very simplistic then. At the very least, each actual neuron is a network of weights. https://www.quantamagazine.org/neural-dendrites-reveal-their... | | |
| ▲ | ACCount37 11 days ago | parent [-] | | I'm partial to "modern ML weights are much closer to 1:1 capacity mapping to synapse count than to neuron count". A single biological neuron is closer to 100 or even 1000 weights worth of ANN than to 1 weight worth of ANN. In which case: modern LLMs are still running in a capacity-starved regime! Even Mythos 5, the 10-trillion monster LLM, the scaling law boogeyman, the harbinger of Vera Rubin NVL72, doesn't quite rise to 100T-to-1000T of synapses. Anything the light of today's AI touches still lives in the shadow of what evolution has managed to cram into a single human skull. We're arguing about the limitations of AI while our best AIs are still very subhuman in the scale dimension. The one dimension we know how to push. And it's already this tight. | | |
| ▲ | SlinkyOnStairs 11 days ago | parent | next [-] | | > A single biological neuron is closer to 100 or even 1000 weights worth of ANN than to 1 weight worth of ANN. Even those comparisons need to be cautioned. The complexity of biology is enormous, and more importantly yet, it's simply not comparable. And doing so invited a bunch of bad assumptions. An ANN could quite probably model a single in vitro neuron with reasonable accuracy. Whether that requires a hundred or a hundred million nodes isn't terribly relevant. But the way neurons combine in vivo is completely unlike the way machine learning systems are built. Both "locally" in how neurons interface which is vastly more complex than a weighted sum of inputs, and the macro scale interactions of hormones and other chemicals. It's not even a given that large numbers of neurons will create the emergent behaviour of human intelligence; Elephants have significantly more neurons, but they're not the triple galaxy brains writing all our science papers. Other animal intelligence similarly is only loosely correlated with brain complexity. (Heck, not to be forgotten is the other end of the scale. Plenty of microscopic life that manages shockingly complex behaviour without any dedicated neurons) This also applies to ANNs. There's no reason to expect that stuffing enough matrix multiplications into a program will make it intelligent or turn out conscious. Really, the history of machine learning suggests the opposite; That the big gains are primarily had in architectural changes. In this regard, I find the talk of the "limits of AI" quite credible. LLMs have already hit the diminishing returns on their growth, and even reasoning/agentic models display failure modes that confirm they're not "thinking" in the ways that humans do. This is not to say that we've hit the final limits of what AI in the broad sense can do, it's just that the next advancement won't be "LLM but even bigger" | | |
| ▲ | ACCount37 11 days ago | parent [-] | | Not really. The history of "big gains" of machine learning is: put together a simple architecture that makes few assumptions but scales well. Then up the data and compute by 2 OOMs. By itself, the new architecture underperforms. Paired with the bitter lesson, however? Don't make assumptions. Make a setup where the gradient descent can make them for you. Empirically? LLMs are nowhere near "the wall". We've been hearing "the wall is nigh" since 2020. Six years in, we're still scaling LLMs, and the graveyards are full of "LLM killers". The system that kills the LLM is always a bigger, badder LLM, and never a new revolutionary architecture. The scaling doesn't just keep working - it works so well that it's seen as the only viable path forward at the frontier of reasoning and agentic work. Or even outside it. ChatGPT Images 2.0 is an image model with an agentic LLM at its core - generational gains in compositional capability. For just about every "failure mode that confirms they're not thinking", you see one of two things. The first is that a new LLM releases a few months after and the "fundamental" issue abruptly goes away. The second is that we take a good, long look at a human, and find that the human also fails like this - and thus, "not thinking". Often both! Always funny when it's both. One thing that's very biologically distinct is: local connectivity. In a GPU, global connectivity is cheap. In a brain, it's prohibitively expensive. The brain has no true backpropagation because it has no true global connectivity, and has to make do with local rules. A GPU is a strictly more expressive substrate connectivity-wise. So any point in the design of a computational substrate where you could remove complexity or increase performance by adding more connectivity? Silicon advantage. The brain isn't a "strictly better computational substrate" - it makes different tradeoffs. Which tradeoffs are better for attaining intelligence is an open question. And, sure. Having a substrate with a capacity for intelligence doesn't mean having intelligence. No elephant has ever learned to code. The problem is: LLMs already did! LLMs already compete with humans on just about every task that was once thought to "require human intelligence". They don't always win - but they perform significantly above chance, and often above an average non-expert human. So, my bet is on "LLM but even bigger". If there's a point where LLMs begin to lag behind and novel architectures get a sharp advantage, we are yet to hit it. | | |
| ▲ | f_klem 11 days ago | parent [-] | | > For just about every "failure mode that confirms they're not thinking", you see one of two things. The first is that a new LLM releases a few months after and the "fundamental" issue abruptly goes away. The second is that we take a good, long look at a human, and find that the human also fails like this - and thus, "not thinking". Often both! Always funny when it's both.
The way machines 'don't think' or 'fail' is fundamentally different from the way humans don't think or fail. In any case, the way LLMs learn and human beings learn is completely different. There is no actual clue that we are approaching any inflection point in machine 'learning'. > So, my bet is on "LLM but even bigger". If there's a point where LLMs begin to lag behind and novel architectures get a sharp advantage, we are yet to hit it.
We are already hearing this 'we are about to hit it' since the late 60s. The difference now is that the market is willingly investing insane amounts of money to make it possible. But again, there is no philosophical, theoretical, epistemological or biological clue that we are getting any closer to human intelligence level.
What we did observe in the last decade though, is that we can build enormous machines that can statistically mimic statistical human outputs. Language and images being some of them. But that is not thinking. | | |
| ▲ | ACCount37 11 days ago | parent [-] | | First, fix your formatting. It's a fucking mess. Second, what is the difference? Is it that one thing has an immortal soul, and thus Actual Intelligence and Actual Reasoning and Actual Learning, and the other has no soul, and a Pale Imitation of Intelligence, At Best? Because I've seen versions of this "it's not actually thinking" for actual fucking years, and the difference between "actually thinking" and "not actually thinking" always seems to boil down to "I don't want LLMs to be actually thinking, so I will bend the definitions and twist the qualifiers and move the goalposts until they aren't". No one ever made an ActualThinkingBenchmark on which humans score 100% and LLMs score 0%. Nothing but human insecurity, in my eyes. There was never a principled difference. Just a way to operationalize some "I'm unique and special and better than a matrix math machine" vibes. | | |
| ▲ | f_klem 11 days ago | parent [-] | | Agreed, formatting was kind of f, but there is no need to be rude. I wasn't saying there was any difference. All I'm saying is that the claimings the AI research field does are based on false assumptions. And from false assumptions, you cannot reach a proper conclusion. Whether an AI system can reason and think like if it where a human being, or not, I don't care. I'm fine with either: it is just technological advance. But making claims based on false assumptions, and then being fooled by how 'wonderful' or 'spectacular' the results are, is, at least, naive. > Nothing but human insecurity, in my eyes. There was never a principled difference. Just a way to operationalize some "I'm unique and special and better than a matrix math machine" vibes. This is just something I don't get. People ignorant of technique are insecure and afraid. People that know how technology works, and thus investigate and know how it works fundamentally*, were never afraid or insecure. | | |
| ▲ | ACCount37 10 days ago | parent [-] | | A lot of people who "know how technology works" just went looking for copium, and found some. Now, they "know" a comforting lie - something like "it's just next token prediction". Very comforting, that, but actively harmful to understanding. The understanding starts with: we don't actually know how LLMs do what they do. They're more grown than designed. And it only gets worse from there. Very little comfort to be found in modern AI. | | |
| ▲ | f_klem 10 days ago | parent [-] | | There are two things here: one is how an LLM is fundamentally structured and designed, the other is how an LLM distributes and 'lays out' the relationship between inputs and outputs through layers and weights. We might not know how the actual distribution works, but we do know how it i s fundamentally structured and designed -- because we did it. We also know that there is something like a representation system inside them. And we also know that human beings do not hold 'internal representations' like any AI system needs to. So there isn't any 'intrinsically magical' in modern AI systems. | | |
| ▲ | ACCount37 10 days ago | parent [-] | | And knowing that structure is about as meaningful as knowing "a PC consists of a keyboard, on which you type, a screen, at which you look, and a processor, which does things with binary logic". None of that helps you understand how exactly LLMs do what they do. Because it describes an interface, not a mechanism. The inner mechanisms of an LLM are more learned than designed. We know what an LLM does on a low level, but going from that to understanding how they work is like trying to understand how a web browser works by looking at netlists of a CPU. Low level understanding does not grant you high level understanding for free. But ignoring all of that lets you cling to a very comforting "we understand LLMs because we made them". Ha ha. As if. > And we also know that human beings do not hold 'internal representations' like any AI system needs to. Bold fucking claim. Got a source on that? Because neurobiology has been trying to crack neural representations - the very internal representations brains use - for as long as it existed, and with some success. Both reading and injecting internal representations into the brain is possible now, in narrow cases. The specifics vary region to region, but sparse population coding is a true staple. Today's SOTA for wrangling this mess is ML decoders, and not by a coincidence. | | |
| ▲ | f_klem 10 days ago | parent [-] | | We know how LLMs learn at the fundamental level. What we do not know is the actual dynamic process of encoding embeddings and their distributions. Your analogies about the PC and web browser are not correctly formulated, because in the case of the PC you talk about 'external components' (you should be talking about cpu arch, structure, digital components, interfaces, etc); in the case of the web browser, you should be talking about modules, code, etc. We do know how LLMs are laid out: layers, att heads, etc. So what we need to look at are the fundamental possibilities of the structure of LLMs, not how the weights are distributed. > > And we also know that human beings do not hold 'internal representations' like any AI system needs to. > Bold fucking claim. Got a source on that? Part of the sources are in the books I mentioned. Nonetheless, you can still fact-check and refute in an adult and serious manner, not in an disrespectful and arrogant way. If my claim sounded arrogant I apologize, but then as I already mentioned, my references back that claim. Regarding internal representations in the brain: I guess you are referring to areas of the brain being activated when a subject receives a stimuli, and this is tested through MRI. I would be cautious to causally relate stimuli to neuron activations, since you first need to know if the exact configuration of cell involved and their connections allow for such representation (which I think it is still not known -- again, AFAIK, the contrary seems to be the case). | | |
| ▲ | ACCount37 10 days ago | parent [-] | | Your references that "back that claim", which are in "books you mentioned", which you "mentioned" who knows where. Yeah, no. I'm not walking that chain. If you want to, do it, but for now, I'm filing it as "has no evidence and knows it". By now, there's plenty of works, up to and including direct neural interfaces. Utah arrays, Michigan arrays. Stab the brain, dump the spike trains, decode. You crack the manifold open by correlating to known stimuli using ML, and generalize from there to unknown stimuli. There is no need to "know the exact configuration", and few bother - you put your hardware into the part of the brain you want (top level map is consistent enough brain to brain), gather a set of reference points, and use them to anchor the rest of the decoding process. Why use ML? Because you need a very expressive correlator to bridge the gap between known inputs and the products of whatever transformations the brain subjects them to before they show up in spike trains. > So what we need to look at are the fundamental possibilities of the structure of LLMs, not how the weights are distributed. And the fundamental possibilities are... what exactly? We know the I/O planes, we know the possible flow of information, now, what does that give us? We know enough to prove that a transformer LLM can implement a Turing machine, the same way a CPU can implement a Turing machine. So an LLM is capable of performing arbitrary computation within its capacity. That's it. That's the upper bound. What follows is: if you can represent "thinking" as a computational process, you can implement it with a Turing machine, and thus, an LLM can be made to think. That proves LLMs can think. But not that the existing ones do or don't! Because that's the entire thing about upper bounds! We've looked at LLM architecture, and learned basically nothing about whether LLMs think, other than "it's not impossible". That's the actual "fundamental possibilities" we derived from knowing the architecture. One step above worthless. Oh fun. (If thinking requires hypercomputation, then, nope. LLMs are out. Good luck proving that it does though.) | | |
| ▲ | f_klem 10 days ago | parent [-] | | > Your references that "back that claim", which are in "books you mentioned", which you "mentioned" who knows where.
Yeah, no. I'm not walking that chain. If you want to, do it, but for now, I'm filing it as "has no evidence and knows it". You are free not to believe me and dismiss the whole point. I do have evidence and I know it, no need to prove that (to begin with, the references are there. Read them if you want to expand your knowledge). > By now, there's plenty of works, up to and including direct neural interfaces. Utah arrays, Michigan arrays. Stab the brain, dump the spike trains, decode. You crack the manifold open by correlating to known stimuli using ML, and generalize from there to unknown stimuli. There is no need to "know the exact configuration", and few bother - you put your hardware into the part of the brain you want (top level map is consistent enough brain to brain), gather a set of reference points, and use them to anchor the rest of the decoding process. I am familiar with those works. Seeing the stimuli/activation correlation does not imply causal representation of the stimuli. It implies the causal activation of neural structures, at most. > What follows is: if you can represent "thinking" as a computational process, you can implement it with a Turing machine, and thus, an LLM can be made to think. That proves LLMs can think. But not that the existing ones do or don't! Because that's the entire thing about upper bounds! Alas! assumption spotted. IF you can represent "thinking" as a computational process, then you could implement a thinking machine. You need to prove first that thinking _is_ a computational process, _then_ you could go and try to implement such machine, and because you proved that thinking is a computational process, you are certain that theoretically such a machine can be built. But until you prove your assumption right, you are just trying blindfolded. The harm in the actual field/society regarding AI is that _we don't even know if thinking can be modeled as a computational process_. And no, this does not have anything to do with science. (By the way, I would not regard AI research as science since it is strictly studying an engineered artifact, but that's another story). | | |
| ▲ | ACCount37 9 days ago | parent [-] | | Knowing what exact algorithm "thinking" is isn't a requirement. Automata class is enough to say "a Turing machine can implement it". There are exactly two possibilities: thinking can be expressed as computation, or thinking requires hypercomputation. I did acknowledge both, explicitly. Which one? I'm betting hard against the second one, by the way. Because it requires hypercomputational magic fairy dust to: 1) exist - physical Church-Turing thesis has to be proven wrong empirically 2) be so involved in the functioning of human brain that it cannot be substituted for anything else Wishful thinking, in my eyes. But that's the name of the game, isn't it? Anything but admitting that your mind is a glorified math construct implemented in wet meat. | | |
| ▲ | f_klem 8 days ago | parent [-] | | > Knowing what exact algorithm "thinking" is isn't a requirement. Automata class is enough to say "a Turing machine can implement it". I don't know what you are referring to by the word 'thinking'. But in any case, if you declare that it is not necessary to know the algorithm about thinking, how can you say then that a Turing machine can implement it? How can you say you implemented something you don't know how it works and how it is constituted? The only option I see then is that you implement something that is phenomenically identical to human intelligence, provided that you exhaust all possible combinations of human intelligence phenomena in a descriptive, extensional way (which, if you assume a finite extension of such phenomena, in any case, and most probably, gets you in the trouble of counting uncountable finite sets). > There are exactly two possibilities: thinking can be expressed as computation, or thinking requires hypercomputation. Again, if you do not define what 'thinking' is and how and on what assumptions it can be described as a computational process, this claim is empty. So as far as I see it, you are still trapped by the assumption that the brain or mind are fundamentally similar to the kind of machines we can build. > But that's the name of the game, isn't it? Anything but admitting that your mind is a glorified math construct implemented in wet meat. Again here some assumptions operate, that tell you that the brain is some kind of hardware. And again: there is no real evidence that the body/consciousness 'construct' has any relation or analogy to the hardware/software/machine idea. Quite the contrary. Since the science that occupies itself on these topics is on the very frontier of knowledge and experimentation, reading science literature only will not clarify your thoughts. You will need additional guidance, and that guidance is called philosophy. I recognize that the references I posted in my original comment are hard to read. But that's the point with the AI/mind debate: it is a tough, bitter topic. Just reading AI research won't bring anyone to the level this research space needs in order to discuss these topics. |
|
|
|
|
|
|
|
|
|
|
|
| |
| ▲ | galangalalgol 11 days ago | parent | prev [-] | | 10T is about a crows worth. The mythos count doesn't include any diffusion model. But the crows count includes all its visual processing. And tactile. Touch uses up enough that they use skin surface area to normalize across animals when doing comparisons. It is one of the reasons suggested to explain how crows exhibit tool use and language with only 10T. We have a lot more skin than crows, and indeed far more than mythos. |
|
| |
| ▲ | f_klem 11 days ago | parent | prev [-] | | I agree, we can miss the forest for the trees. 1) This definition could actually be expanded (for example, with definitions from Mumford or Reuleaux). But still this definition cannot be applied directly to living organisms.
2) This is in my opinion one of the sources of misunderstanding. We mainly operate on analogies and metaphors, so we have build this 'analogy space' around the idea that living organisms are machines. But it is only when we say 'alike' that we can truly gather some meaning out of it all, going beyond the 'behaves like' or 'is conceptualized as' when it gets messy. > With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!" This is exactly my point. There is a fallacy operating from "A is not B" to "A is C". And this fallacy is pervasive in the AI research field, the book from Dreyfus (What Computers can't still do) explains that in much detail. | | |
| ▲ | calepayson 11 days ago | parent [-] | | > 1) This definition could actually be expanded (for example, with definitions from Mumford or Reuleaux). But still this definition cannot be applied directly to living organisms. I'm not sure I understand this. Why not? | | |
| ▲ | f_klem 11 days ago | parent [-] | | It has to do with words and how we evolve words throughout history and across geographic boundaries. The term 'machine' comes, after some modifications, from the greek word mekhanos, which was used to describe something ingenious or a device made in some clever way or operating in a clever way. From there it went on to describe things like devices, to end up being the actual definition of what we might call 'a device' (a machine). The idea of 'mechanistic' is also related. Traditionally, things that are alive were described with different words and assigned a different set of properties and characteristics. Machine can break, living things die. And we still have those two semantic frames separated:
A living thing: can be harmed, it breathes, it nourishes, it reproduces, etc
A machine: can break, can be fixed, can be repurposed, etc. But because of a specific tradition in western philosophy, we started applying and analogy between 'inner mechanism (clever thing), that moves or provides a function, and seems to work in a causal way' and living things. So when we say 'a living cell is a wonderful, complex machine', we are not actually saying it is a machine, we are operating through an analogy. That's how far we can go. |
|
|
|
