| ▲ | f_klem 11 days ago |
| After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind.
What all these theories have in common is the underlying belief that our brain/mind works as the machines we build. Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. The idea of 'internal models' and 'control loops' inside us is a projection of the aforementioned assumption. There is also an epistemological assumption that prevails, and that is that we understand (or we think we understand) how our brain/mind works. But the truth is that we don't know. And there's even not a single clue that we actually know too much, and not a clue that our brain/mind and cells work 'as the machines we build'. Only by bypassing this epistemological problem, we can build 'theories of computational mind'. These assumptions are there for already long time, to the point that when Turing asked himself 'can machines think?', he already assumed our thinking could be modeled as a machine. I highly recommend people in the AI research space should read philosophy and modern linguistics. But not stopping at Descartes/Leibniz. Heidegger made contributions that cannot be avoided. |
|
| ▲ | codeflo 11 days ago | parent | next [-] |
| > These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. On the contrary, it's precisely this assumption, that there is a "subjective experience" that requires explanation beyond the material, that is axiomatically assumed without evidence. It falls apart quickly, any "subjective experience" is completely tied to neurons, knock out the neurons and the subjective experience disappears, or stimulate the neurons to cause the experience. |
| |
| ▲ | SlinkyOnStairs 11 days ago | parent | next [-] | | There's two meanings to "the body is a complex machine" and I think you're missing the forest for the trees here. 1) The abstract "dictionary" version: It'd be technically correct to say that the body is a machine under the definition of "A machine is a thermodynamic system that uses power to apply forces and control movement to perform an action.". 2) But there's also the less abstract/technical: "The body is alike the complex machines we have built", and this is much less true. Especially for the brain. The "neuron" analogy in machine learning is effective, but entirely wrong; We do not fully know how even a single neuron works, nevermind any complex system made out of multiple of them. With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!" Especially so by people who have a financial/legal interest in doing so. "AI is just like a brain, fire your employees and buy our LLM now!", "AI is just like a brain, so it's totally not copyright infringement!" | | |
| ▲ | aspenmartin 11 days ago | parent | next [-] | | > With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!" Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise? Does anyone seriously consider a brain to be something other than “a pile of molecules following the laws of physics”? If so that’s not science or philosophy, that’s religion. You have a virtually complete phenomenological model of the universe for all intents and purposes and yet somehow the onus is on the person being like “hey no laws of physics are being broken ==> the brain is simply following the laws of physics” How is it possible that people think of subjective experience and get rabbit holed into some mystical world where subjective experience is this special exception to everything else that is simply an emergent property of complex physical systems? “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness? It’s more: consciousness is not some mystical or religious thing outside of the realm of physics, it’s an emergent property of a complex system. AI is a relatively complex system. We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that | | |
| ▲ | SlinkyOnStairs 11 days ago | parent | next [-] | | > Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise? They would have to be similar because the argument is that they are similar; "AI is like the human brain" only holds true if it actually is like the human brain, not merely a superficial resemblance. What I'm describing in that prior comment is how a lot of people drastically simplify the resemblance in order to make it feel true; That the lack of a Jesus Christ coming down from the heavens to tell everyone they have immaterial souls that the computers don't makes the comparison more true. > We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that Therein lies the conflict: "You can't prove it's not conscious" is an unfalsifiable statement. You can't engage with the argument because it's proponents will always claim victory, often with their own interests at play. All concerns about "superintelligence" or the long term ethics of "when do our robots become sufficiently intelligent that they'd be slaves?" have been subsumed into the AI marketing machine. However sincere one might try to address the issue, they look like a Sam Altman stooge by association. It's like claiming the quantum fluctuations inside a pet rock as a "consciousness", even if observed directly any measurement of random noise can still be dismissed as "nuh-uh it just takes billions of years to have a thought". More practically with current AI systems, we can look inside them pretty well and there genuinely is nothing there. Standalone LLMs are purely feed-forward systems. Their failure modes show that they perform no meaningful thought or world modelling during inference. They're just language models. The reasoning and agentic systems are even easier to introspect. We know how they work, we can look at the full prompts & context they operate on. There is nothing there. This is what sets AI apart from animals, which are given the benefit of the doubt on their intelligence. | |
| ▲ | wredcoll 11 days ago | parent | prev | next [-] | | > “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness? It doesn't need to be true but a lot of people make it/assume it. There's a lot of, perhaps casual and uninformed, conversations that strongly imply a deeper understanding of the "physics" of brain chemistry than we actually have, mostly by comparing it to machines we've constructed. (I believe) We don't need to replicate human neurons and dendrites and whatever else is in there in order to create a sapient "machine", but whether or not we've actually done that isn't being helped by arguing that what we currently have is all that similar to a human brain. | |
| ▲ | calepayson 11 days ago | parent | prev [-] | | I think the point of the commentator above is that there are two extreme narratives that start each start with an uncontroversial assumption and then taking it to a pretty wild place. One narrative takes the assumption that brains are just matter so it should be possible to engineer consciousness and then argues that LLMs are conscious. The other takes the assumption that LLMs aren't conscious but then argues that because they aren't we won't ever be able to make anything conscious. I don't actually think the commentator you responded to is arguing for either of these narratives and I thought it was a pretty useful way to look at some of these arguments. |
| |
| ▲ | sixeyes 11 days ago | parent | prev | next [-] | | oh yeah! i recall a paper linked here not so long ago, where it was shown that the dendrites of a neuron do computations themselves. The "weight per neuron" is very simplistic then. At the very least, each actual neuron is a network of weights. https://www.quantamagazine.org/neural-dendrites-reveal-their... | | |
| ▲ | ACCount37 11 days ago | parent [-] | | I'm partial to "modern ML weights are much closer to 1:1 capacity mapping to synapse count than to neuron count". A single biological neuron is closer to 100 or even 1000 weights worth of ANN than to 1 weight worth of ANN. In which case: modern LLMs are still running in a capacity-starved regime! Even Mythos 5, the 10-trillion monster LLM, the scaling law boogeyman, the harbinger of Vera Rubin NVL72, doesn't quite rise to 100T-to-1000T of synapses. Anything the light of today's AI touches still lives in the shadow of what evolution has managed to cram into a single human skull. We're arguing about the limitations of AI while our best AIs are still very subhuman in the scale dimension. The one dimension we know how to push. And it's already this tight. | | |
| ▲ | SlinkyOnStairs 11 days ago | parent | next [-] | | > A single biological neuron is closer to 100 or even 1000 weights worth of ANN than to 1 weight worth of ANN. Even those comparisons need to be cautioned. The complexity of biology is enormous, and more importantly yet, it's simply not comparable. And doing so invited a bunch of bad assumptions. An ANN could quite probably model a single in vitro neuron with reasonable accuracy. Whether that requires a hundred or a hundred million nodes isn't terribly relevant. But the way neurons combine in vivo is completely unlike the way machine learning systems are built. Both "locally" in how neurons interface which is vastly more complex than a weighted sum of inputs, and the macro scale interactions of hormones and other chemicals. It's not even a given that large numbers of neurons will create the emergent behaviour of human intelligence; Elephants have significantly more neurons, but they're not the triple galaxy brains writing all our science papers. Other animal intelligence similarly is only loosely correlated with brain complexity. (Heck, not to be forgotten is the other end of the scale. Plenty of microscopic life that manages shockingly complex behaviour without any dedicated neurons) This also applies to ANNs. There's no reason to expect that stuffing enough matrix multiplications into a program will make it intelligent or turn out conscious. Really, the history of machine learning suggests the opposite; That the big gains are primarily had in architectural changes. In this regard, I find the talk of the "limits of AI" quite credible. LLMs have already hit the diminishing returns on their growth, and even reasoning/agentic models display failure modes that confirm they're not "thinking" in the ways that humans do. This is not to say that we've hit the final limits of what AI in the broad sense can do, it's just that the next advancement won't be "LLM but even bigger" | | |
| ▲ | ACCount37 11 days ago | parent [-] | | Not really. The history of "big gains" of machine learning is: put together a simple architecture that makes few assumptions but scales well. Then up the data and compute by 2 OOMs. By itself, the new architecture underperforms. Paired with the bitter lesson, however? Don't make assumptions. Make a setup where the gradient descent can make them for you. Empirically? LLMs are nowhere near "the wall". We've been hearing "the wall is nigh" since 2020. Six years in, we're still scaling LLMs, and the graveyards are full of "LLM killers". The system that kills the LLM is always a bigger, badder LLM, and never a new revolutionary architecture. The scaling doesn't just keep working - it works so well that it's seen as the only viable path forward at the frontier of reasoning and agentic work. Or even outside it. ChatGPT Images 2.0 is an image model with an agentic LLM at its core - generational gains in compositional capability. For just about every "failure mode that confirms they're not thinking", you see one of two things. The first is that a new LLM releases a few months after and the "fundamental" issue abruptly goes away. The second is that we take a good, long look at a human, and find that the human also fails like this - and thus, "not thinking". Often both! Always funny when it's both. One thing that's very biologically distinct is: local connectivity. In a GPU, global connectivity is cheap. In a brain, it's prohibitively expensive. The brain has no true backpropagation because it has no true global connectivity, and has to make do with local rules. A GPU is a strictly more expressive substrate connectivity-wise. So any point in the design of a computational substrate where you could remove complexity or increase performance by adding more connectivity? Silicon advantage. The brain isn't a "strictly better computational substrate" - it makes different tradeoffs. Which tradeoffs are better for attaining intelligence is an open question. And, sure. Having a substrate with a capacity for intelligence doesn't mean having intelligence. No elephant has ever learned to code. The problem is: LLMs already did! LLMs already compete with humans on just about every task that was once thought to "require human intelligence". They don't always win - but they perform significantly above chance, and often above an average non-expert human. So, my bet is on "LLM but even bigger". If there's a point where LLMs begin to lag behind and novel architectures get a sharp advantage, we are yet to hit it. | | |
| ▲ | f_klem 11 days ago | parent [-] | | > For just about every "failure mode that confirms they're not thinking", you see one of two things. The first is that a new LLM releases a few months after and the "fundamental" issue abruptly goes away. The second is that we take a good, long look at a human, and find that the human also fails like this - and thus, "not thinking". Often both! Always funny when it's both.
The way machines 'don't think' or 'fail' is fundamentally different from the way humans don't think or fail. In any case, the way LLMs learn and human beings learn is completely different. There is no actual clue that we are approaching any inflection point in machine 'learning'. > So, my bet is on "LLM but even bigger". If there's a point where LLMs begin to lag behind and novel architectures get a sharp advantage, we are yet to hit it.
We are already hearing this 'we are about to hit it' since the late 60s. The difference now is that the market is willingly investing insane amounts of money to make it possible. But again, there is no philosophical, theoretical, epistemological or biological clue that we are getting any closer to human intelligence level.
What we did observe in the last decade though, is that we can build enormous machines that can statistically mimic statistical human outputs. Language and images being some of them. But that is not thinking. | | |
| ▲ | ACCount37 11 days ago | parent [-] | | First, fix your formatting. It's a fucking mess. Second, what is the difference? Is it that one thing has an immortal soul, and thus Actual Intelligence and Actual Reasoning and Actual Learning, and the other has no soul, and a Pale Imitation of Intelligence, At Best? Because I've seen versions of this "it's not actually thinking" for actual fucking years, and the difference between "actually thinking" and "not actually thinking" always seems to boil down to "I don't want LLMs to be actually thinking, so I will bend the definitions and twist the qualifiers and move the goalposts until they aren't". No one ever made an ActualThinkingBenchmark on which humans score 100% and LLMs score 0%. Nothing but human insecurity, in my eyes. There was never a principled difference. Just a way to operationalize some "I'm unique and special and better than a matrix math machine" vibes. | | |
| ▲ | f_klem 11 days ago | parent [-] | | Agreed, formatting was kind of f, but there is no need to be rude. I wasn't saying there was any difference. All I'm saying is that the claimings the AI research field does are based on false assumptions. And from false assumptions, you cannot reach a proper conclusion. Whether an AI system can reason and think like if it where a human being, or not, I don't care. I'm fine with either: it is just technological advance. But making claims based on false assumptions, and then being fooled by how 'wonderful' or 'spectacular' the results are, is, at least, naive. > Nothing but human insecurity, in my eyes. There was never a principled difference. Just a way to operationalize some "I'm unique and special and better than a matrix math machine" vibes. This is just something I don't get. People ignorant of technique are insecure and afraid. People that know how technology works, and thus investigate and know how it works fundamentally*, were never afraid or insecure. | | |
| ▲ | ACCount37 10 days ago | parent [-] | | A lot of people who "know how technology works" just went looking for copium, and found some. Now, they "know" a comforting lie - something like "it's just next token prediction". Very comforting, that, but actively harmful to understanding. The understanding starts with: we don't actually know how LLMs do what they do. They're more grown than designed. And it only gets worse from there. Very little comfort to be found in modern AI. | | |
| ▲ | f_klem 10 days ago | parent [-] | | There are two things here: one is how an LLM is fundamentally structured and designed, the other is how an LLM distributes and 'lays out' the relationship between inputs and outputs through layers and weights. We might not know how the actual distribution works, but we do know how it i s fundamentally structured and designed -- because we did it. We also know that there is something like a representation system inside them. And we also know that human beings do not hold 'internal representations' like any AI system needs to. So there isn't any 'intrinsically magical' in modern AI systems. | | |
| ▲ | ACCount37 10 days ago | parent [-] | | And knowing that structure is about as meaningful as knowing "a PC consists of a keyboard, on which you type, a screen, at which you look, and a processor, which does things with binary logic". None of that helps you understand how exactly LLMs do what they do. Because it describes an interface, not a mechanism. The inner mechanisms of an LLM are more learned than designed. We know what an LLM does on a low level, but going from that to understanding how they work is like trying to understand how a web browser works by looking at netlists of a CPU. Low level understanding does not grant you high level understanding for free. But ignoring all of that lets you cling to a very comforting "we understand LLMs because we made them". Ha ha. As if. > And we also know that human beings do not hold 'internal representations' like any AI system needs to. Bold fucking claim. Got a source on that? Because neurobiology has been trying to crack neural representations - the very internal representations brains use - for as long as it existed, and with some success. Both reading and injecting internal representations into the brain is possible now, in narrow cases. The specifics vary region to region, but sparse population coding is a true staple. Today's SOTA for wrangling this mess is ML decoders, and not by a coincidence. | | |
| ▲ | f_klem 10 days ago | parent [-] | | We know how LLMs learn at the fundamental level. What we do not know is the actual dynamic process of encoding embeddings and their distributions. Your analogies about the PC and web browser are not correctly formulated, because in the case of the PC you talk about 'external components' (you should be talking about cpu arch, structure, digital components, interfaces, etc); in the case of the web browser, you should be talking about modules, code, etc. We do know how LLMs are laid out: layers, att heads, etc. So what we need to look at are the fundamental possibilities of the structure of LLMs, not how the weights are distributed. > > And we also know that human beings do not hold 'internal representations' like any AI system needs to. > Bold fucking claim. Got a source on that? Part of the sources are in the books I mentioned. Nonetheless, you can still fact-check and refute in an adult and serious manner, not in an disrespectful and arrogant way. If my claim sounded arrogant I apologize, but then as I already mentioned, my references back that claim. Regarding internal representations in the brain: I guess you are referring to areas of the brain being activated when a subject receives a stimuli, and this is tested through MRI. I would be cautious to causally relate stimuli to neuron activations, since you first need to know if the exact configuration of cell involved and their connections allow for such representation (which I think it is still not known -- again, AFAIK, the contrary seems to be the case). | | |
| ▲ | ACCount37 10 days ago | parent [-] | | Your references that "back that claim", which are in "books you mentioned", which you "mentioned" who knows where. Yeah, no. I'm not walking that chain. If you want to, do it, but for now, I'm filing it as "has no evidence and knows it". By now, there's plenty of works, up to and including direct neural interfaces. Utah arrays, Michigan arrays. Stab the brain, dump the spike trains, decode. You crack the manifold open by correlating to known stimuli using ML, and generalize from there to unknown stimuli. There is no need to "know the exact configuration", and few bother - you put your hardware into the part of the brain you want (top level map is consistent enough brain to brain), gather a set of reference points, and use them to anchor the rest of the decoding process. Why use ML? Because you need a very expressive correlator to bridge the gap between known inputs and the products of whatever transformations the brain subjects them to before they show up in spike trains. > So what we need to look at are the fundamental possibilities of the structure of LLMs, not how the weights are distributed. And the fundamental possibilities are... what exactly? We know the I/O planes, we know the possible flow of information, now, what does that give us? We know enough to prove that a transformer LLM can implement a Turing machine, the same way a CPU can implement a Turing machine. So an LLM is capable of performing arbitrary computation within its capacity. That's it. That's the upper bound. What follows is: if you can represent "thinking" as a computational process, you can implement it with a Turing machine, and thus, an LLM can be made to think. That proves LLMs can think. But not that the existing ones do or don't! Because that's the entire thing about upper bounds! We've looked at LLM architecture, and learned basically nothing about whether LLMs think, other than "it's not impossible". That's the actual "fundamental possibilities" we derived from knowing the architecture. One step above worthless. Oh fun. (If thinking requires hypercomputation, then, nope. LLMs are out. Good luck proving that it does though.) | | |
| ▲ | f_klem 10 days ago | parent [-] | | > Your references that "back that claim", which are in "books you mentioned", which you "mentioned" who knows where.
Yeah, no. I'm not walking that chain. If you want to, do it, but for now, I'm filing it as "has no evidence and knows it". You are free not to believe me and dismiss the whole point. I do have evidence and I know it, no need to prove that (to begin with, the references are there. Read them if you want to expand your knowledge). > By now, there's plenty of works, up to and including direct neural interfaces. Utah arrays, Michigan arrays. Stab the brain, dump the spike trains, decode. You crack the manifold open by correlating to known stimuli using ML, and generalize from there to unknown stimuli. There is no need to "know the exact configuration", and few bother - you put your hardware into the part of the brain you want (top level map is consistent enough brain to brain), gather a set of reference points, and use them to anchor the rest of the decoding process. I am familiar with those works. Seeing the stimuli/activation correlation does not imply causal representation of the stimuli. It implies the causal activation of neural structures, at most. > What follows is: if you can represent "thinking" as a computational process, you can implement it with a Turing machine, and thus, an LLM can be made to think. That proves LLMs can think. But not that the existing ones do or don't! Because that's the entire thing about upper bounds! Alas! assumption spotted. IF you can represent "thinking" as a computational process, then you could implement a thinking machine. You need to prove first that thinking _is_ a computational process, _then_ you could go and try to implement such machine, and because you proved that thinking is a computational process, you are certain that theoretically such a machine can be built. But until you prove your assumption right, you are just trying blindfolded. The harm in the actual field/society regarding AI is that _we don't even know if thinking can be modeled as a computational process_. And no, this does not have anything to do with science. (By the way, I would not regard AI research as science since it is strictly studying an engineered artifact, but that's another story). | | |
| ▲ | ACCount37 9 days ago | parent [-] | | Knowing what exact algorithm "thinking" is isn't a requirement. Automata class is enough to say "a Turing machine can implement it". There are exactly two possibilities: thinking can be expressed as computation, or thinking requires hypercomputation. I did acknowledge both, explicitly. Which one? I'm betting hard against the second one, by the way. Because it requires hypercomputational magic fairy dust to: 1) exist - physical Church-Turing thesis has to be proven wrong empirically 2) be so involved in the functioning of human brain that it cannot be substituted for anything else Wishful thinking, in my eyes. But that's the name of the game, isn't it? Anything but admitting that your mind is a glorified math construct implemented in wet meat. | | |
| ▲ | f_klem 8 days ago | parent [-] | | > Knowing what exact algorithm "thinking" is isn't a requirement. Automata class is enough to say "a Turing machine can implement it". I don't know what you are referring to by the word 'thinking'. But in any case, if you declare that it is not necessary to know the algorithm about thinking, how can you say then that a Turing machine can implement it? How can you say you implemented something you don't know how it works and how it is constituted? The only option I see then is that you implement something that is phenomenically identical to human intelligence, provided that you exhaust all possible combinations of human intelligence phenomena in a descriptive, extensional way (which, if you assume a finite extension of such phenomena, in any case, and most probably, gets you in the trouble of counting uncountable finite sets). > There are exactly two possibilities: thinking can be expressed as computation, or thinking requires hypercomputation. Again, if you do not define what 'thinking' is and how and on what assumptions it can be described as a computational process, this claim is empty. So as far as I see it, you are still trapped by the assumption that the brain or mind are fundamentally similar to the kind of machines we can build. > But that's the name of the game, isn't it? Anything but admitting that your mind is a glorified math construct implemented in wet meat. Again here some assumptions operate, that tell you that the brain is some kind of hardware. And again: there is no real evidence that the body/consciousness 'construct' has any relation or analogy to the hardware/software/machine idea. Quite the contrary. Since the science that occupies itself on these topics is on the very frontier of knowledge and experimentation, reading science literature only will not clarify your thoughts. You will need additional guidance, and that guidance is called philosophy. I recognize that the references I posted in my original comment are hard to read. But that's the point with the AI/mind debate: it is a tough, bitter topic. Just reading AI research won't bring anyone to the level this research space needs in order to discuss these topics. |
|
|
|
|
|
|
|
|
|
|
|
| |
| ▲ | galangalalgol 11 days ago | parent | prev [-] | | 10T is about a crows worth. The mythos count doesn't include any diffusion model. But the crows count includes all its visual processing. And tactile. Touch uses up enough that they use skin surface area to normalize across animals when doing comparisons. It is one of the reasons suggested to explain how crows exhibit tool use and language with only 10T. We have a lot more skin than crows, and indeed far more than mythos. |
|
| |
| ▲ | f_klem 11 days ago | parent | prev [-] | | I agree, we can miss the forest for the trees. 1) This definition could actually be expanded (for example, with definitions from Mumford or Reuleaux). But still this definition cannot be applied directly to living organisms.
2) This is in my opinion one of the sources of misunderstanding. We mainly operate on analogies and metaphors, so we have build this 'analogy space' around the idea that living organisms are machines. But it is only when we say 'alike' that we can truly gather some meaning out of it all, going beyond the 'behaves like' or 'is conceptualized as' when it gets messy. > With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!" This is exactly my point. There is a fallacy operating from "A is not B" to "A is C". And this fallacy is pervasive in the AI research field, the book from Dreyfus (What Computers can't still do) explains that in much detail. | | |
| ▲ | calepayson 11 days ago | parent [-] | | > 1) This definition could actually be expanded (for example, with definitions from Mumford or Reuleaux). But still this definition cannot be applied directly to living organisms. I'm not sure I understand this. Why not? | | |
| ▲ | f_klem 11 days ago | parent [-] | | It has to do with words and how we evolve words throughout history and across geographic boundaries. The term 'machine' comes, after some modifications, from the greek word mekhanos, which was used to describe something ingenious or a device made in some clever way or operating in a clever way. From there it went on to describe things like devices, to end up being the actual definition of what we might call 'a device' (a machine). The idea of 'mechanistic' is also related. Traditionally, things that are alive were described with different words and assigned a different set of properties and characteristics. Machine can break, living things die. And we still have those two semantic frames separated:
A living thing: can be harmed, it breathes, it nourishes, it reproduces, etc
A machine: can break, can be fixed, can be repurposed, etc. But because of a specific tradition in western philosophy, we started applying and analogy between 'inner mechanism (clever thing), that moves or provides a function, and seems to work in a causal way' and living things. So when we say 'a living cell is a wonderful, complex machine', we are not actually saying it is a machine, we are operating through an analogy. That's how far we can go. |
|
|
| |
| ▲ | JulianChastain 11 days ago | parent | prev | next [-] | | I find the claim subjective experience may be illusory absolutely baffling. I can only speak for myself with certainty, but I am entirely sure I have subjective experience. All the other propositions I believe could be false but I don't see how I can be wrong about experiencing something. I could be a brain in a vat (or weights on a GPU) and be specifically programmed to only come to false beliefs and still I can be sure that there is an experience I am the subject of. I cannot provide empirical evidence for my experience, that is why it is subjective. I cannot be entirely sure you are experiencing anything, and when I encounter people who don't share the same baseline intuition here I do begin to wonder if this is truly a universal across humanity. But I can't think of any other assumption which I would be more comfortable as a foundational axiom other than, "I am experiencing something." I do not require additional evidence for it because I experience the truth of it directly | | |
| ▲ | Self-Perfection 11 days ago | parent | next [-] | | My current understand that "subjective experience" is a post effect of memory forming in the process. "I experience X" ≈ "I remember that I just recently received [external stimulus / interpreted my current state as] X". And I am as well baffled why people make such a big deal out of "subjective experience" and "consciousness". I was joking that maybe I miss this properties, but now starting to really wonder if it might be the case. What if these phenomenons are present in humans to various extent? Check aphantasia. Only in XIX we discovered, that ability to visualize mental images is not universal, available to different people to various degree and some people completely miss it. My ability to visualize is weak. What if "consciousness" and "subjective experience" are similar? And I am slightly worried when I am writing this that it might turn to be truth and in ~20 years I will be treated as "inferior human" without complete set of human rights. | |
| ▲ | edbaskerville 11 days ago | parent | prev [-] | | Indeed. Even positing an illusion seems like a contradiction. If it's illusory, doesn't there need to be a subjective entity experiencing the illusion? | | |
| ▲ | MichaelZuo 11 days ago | parent [-] | | This proves too much. As it would imply parrots have sapiencee. | | |
| ▲ | jazzypants 11 days ago | parent [-] | | Why do you think that parrots do not have sapience? How would one even measure such a thing? | | |
| ▲ | beering 11 days ago | parent [-] | | Because by definition, sapience is something only humans have. Ergo, parrots are not sapient. More meta, all of the threads on this page are just people playing games with definitions. Eg, “qualia is something I have as a human but machines don’t have it. Therefore, LLMs do not have qualia.” | | |
| ▲ | jazzypants 11 days ago | parent | next [-] | | You're confused about the etymology. Homo Sapiens was coined in the 1800s. People have been saying "sapient" since the 1300s, and it is rooted from the Latin word "sapientem" which simply means "sensible; shrewd, knowing, discrete". Homo Sapiens just means "wise human", and we humbly bestowed the name upon ourselves. https://www.etymonline.com/word/sapient https://en.wikipedia.org/wiki/Human#Etymology_and_definition | |
| ▲ | f_klem 11 days ago | parent | prev [-] | | > More meta, all of the threads on this page are just people playing games with definitions. Eg, “qualia is something I have as a human but machines don’t have it. Therefore, LLMs do not have qualia.”
True. For me, the actual interesting debate is not if LLMs are intelligent or not (easy to dismiss) but to what extent LLMs embed into our socio-techno-economic reality. |
|
|
|
|
| |
| ▲ | kristov 11 days ago | parent | prev | next [-] | | I think the truth lies somewhere between these two extremes. An LLM is not a human brain, and does not try to emulate one. It should not be a surprise that an LLM does not behave like a human brain. So we can not infer things either way. The best we can say is that an LLM appears to exhibit very similar behavior to a human brain, under certain constraints. So maybe we can infer that the human brain has something in it that operates in a similar way to an LLM (like the human "unconscious", or "intuition" maybe). It seems obvious to me that a human brain and an LLM are not comparable things, for many reasons. So we can not make inferrences one way or the other. | | |
| ▲ | dnautics 11 days ago | parent [-] | | > An LLM is not a human brain, true > and does not try to emulate one citation please. something like the universal approximation theory comes to mind, transformer architecture clearly has the shape of a universal algorithm approximator | | |
| ▲ | calepayson 11 days ago | parent | next [-] | | I think you're right in that it has the shape but I think it's missing a pretty key piece. We still haven't been able to solve catastrophic forgetting, yet everything with a brain has. Basically LLMs seem good at approximating intelligence on a moment-to-moment basis, but feel quite far away when you chat with one over time. Like at some level, yes, transformers are trying to emulate a human brain but the second you ask folks if they do a good job of it, I think most rationale people would say no. | |
| ▲ | cootsnuck 11 days ago | parent | prev [-] | | > something like the universal approximation theory comes to mind, transformer architecture clearly has the shape of a universal algorithm approximator That says nothing about emulating a human brain. | | |
| ▲ | dnautics 11 days ago | parent [-] | | is human cognition not an algorithm? is it just woo? or vibes? | | |
| ▲ | kristov 11 days ago | parent [-] | | But what is the shape of the algorithm of the human brain? It has a complex physical structure. We know the folds on the surface are important, but why is that shape specifically important? The brain is made up of two hemispheres - why, what does that do? There are different "types" of brain inside the human skull. There are physical areas that perform specific tasks. There are different types of neurons. Then there chemicals that interact with the brain, changing how it function depending on things happening to the body. All that stuff and more is the "algorithm" of the human brain. It's not the same algorithm as an LLM. |
|
|
|
| |
| ▲ | llmssuck 11 days ago | parent | prev | next [-] | | Interesting, so you would say that your experience is .. illusory? In what medium exactly? Illusion requires a substrate of some kind. "Awareness"? What's that? Neurons are themselves things we experience (indirectly). Once seen through a microscope or known about in some fashion the only way they "exist" is by you having the experience of knowing them. It's not the other way around. One thing is more fundamental here. What is this experience? What are the atoms of this? "Atomic particles"? How would you even approach an answer if your building blocks are themselves part of what needs to be explained? The hard problem cannot even be touched if you start out like this. | |
| ▲ | jimbokun 11 days ago | parent | prev | next [-] | | It’s the opposite. Descartes made clear that subjective experience is the ONLY thing we know. Everything else is theories to explain the phenomena we subjectively experience. We theorize that there is a physical world and other beings like us having similar subjective experiences, because that seems the best explanation for our subjective experiences. But we might be living in the Matrix, with all the people we think we are interacting with and just sophisticated simulations. | | |
| ▲ | aspenmartin 11 days ago | parent [-] | | And this has some bearing on the debate about whether these systems do or could in the future exhibit something similar? | | |
| ▲ | bogdanoff_2 10 days ago | parent [-] | | If we go along with the though that our own subjective experience is the only thing we truly know, and that we cannot really know if any other humans are having the same experience (and any belief of that sorts is purely extrapolation), then there isn't a fundamental difference between LLMs and "other humans" in terms of whether or not they're "conscious". Sure, it appears more likely that "other humans" are real conscious beings, but there's no fundamental difference. >whether these systems do or could in the future exhibit something similar? I think the whole discussion is based on the idea that consciousness isn't something you can "exhibit". (Tell me, how can you "exhibit consciousness"?) |
|
| |
| ▲ | f_klem 11 days ago | parent | prev | next [-] | | Refuting the "subjective experience" axiom does not lead to 'any "subjective experience" is completely tied to neurons', you also need to explain why the subjective experience is tied to neurons. And that's precisely what computational theories of mind do not account for: the link between subjective experience and neurons.
I am not arguing that neurons (or the brain) are not a necessary condition for subjective experience. | |
| ▲ | stonogo 11 days ago | parent | prev | next [-] | | "that is axiomatically assumed" passive voice doing a hell of a lot of work in this phrase | |
| ▲ | tuyiown 11 days ago | parent | prev [-] | | > at requires explanation beyond the material, that is axiomatically assumed without evidence Nobody talked about anything out of neurons. The question is still open. |
|
|
| ▲ | housecarpenter 11 days ago | parent | prev | next [-] |
| Why is asking 'can machines think?' assuming our thinking could be modeled as a machine? It's raising the possibility that our thinking could be modeled as a machine. Given that, as you acknowledge, we don't have a clue how our mind works, it seems premature to rule out this possibility. Rather, I would say that ruling it out betrays an assumption that we understand how the mind works enough that we can say that it is definitely not replicable by a machine, and that assumption seems unjustified. |
| |
| ▲ | tim333 11 days ago | parent | next [-] | | Indeed, scientific type thinking like that where you ask a question that you can do experiments on to see if machines can pass the test is probably the way to progress. The philosophical waffling mostly just goes in circles. To investigate consciousness you'll probably get further trying to build conscious machines with agency and comparing them to biological ones than reading Heidegger. | |
| ▲ | f_klem 11 days ago | parent | prev [-] | | > It's raising the possibility that our thinking could be modeled as a machine. Given that, as you acknowledge, we don't have a clue how our mind works, it seems premature to rule out this possibility. In order to raise the possibility that our thinking can be modeled as a machine, there needs to be a previous question: 'can our thinking be modeled at all?'. And before that, already the idea of possibility: 'our thinking could be something that can be modeled'.
Since we know 'we can think', asking 'can machines think?' needs the assumption that machines and brains are alike. If there is no assumption, then we should ask 'if brains and machines are alike, then we could raise the possibility of thinking a machine could also think'. But when we say 'brains and machines are alike', we are implicitly saying 'brains are like machines'.
There is no problem asking 'can machines think?', but there is indeed a problem with implicitly assuming that brains and machines are alike when we do not known it yet. I am critiquing the idea of assumptions here, not the research. |
|
|
| ▲ | larodi 11 days ago | parent | prev | next [-] |
| If we agree weights producing text may emerge consciousness, given large enough, then DBs must've gained it long ago. The whole idea of sentience "on the other side of the wire" is wrong, as there is neither wire, nor other side. We think there is, but the DB just expounds the query and repurposes this information into views, that we give notion and meaning to. LLMs are the same, do not be mistaken. One other important thing to consider is that the human experience is thanks to the body, is in connection, and perhaps product of the body. The body is observable and perhaps humans may state that they feel the connection to it. LLMs have no notion of nothing, the machine does not know the result, and the result does not know the VM. Modern psychology more or less has settled around the idea that consciousness is a product of the body. Why and how does this construct come to realize a Self is then another mystery even if we know which parts of the hardware may be involved. Whether it is the Holy Spirit or Life Force animating the human body is a completely different question also. Besides, the realization, the experience we have now with all life in 2026 is not something that can be easily explained or attuned to life 200 years ago and its terms and notions. So is also wrong to even attempt to. |
| |
| ▲ | IanCal 11 days ago | parent | next [-] | | > If we agree weights producing text may emerge consciousness, given large enough, then DBs must've gained it long ago. If we agree that silicon can perform calculations, then beaches must have been working out log tables long ago. | | |
| ▲ | dmd 11 days ago | parent | next [-] | | Greg Egan wrote an entire (fabulous) book about exactly that, "Permutation City". | | |
| ▲ | larodi 11 days ago | parent [-] | | Diaspora is mind-blowing, yet highly improbable and speculative, even though carefully threaded to sound plausible on all levels. The whole introdus idea and simulation from VM perspective sounds incredible, but I don't think the body runs a simulation of anything. It is something else. |
| |
| ▲ | bobson381 11 days ago | parent | prev | next [-] | | a la https://xkcd.com/505/ | |
| ▲ | larodi 11 days ago | parent | prev [-] | | working out things,... is not calculating. |
| |
| ▲ | lxgr 11 days ago | parent | prev | next [-] | | > Modern psychology more or less has settled around the idea that consciousness is a product of the body. The kind of consciousness we know. Jumping to the conclusion that that's the only kind possible, or even stronger that the way ours evolved is the only way this could have happened, is completely invalid. | |
| ▲ | bondarchuk 11 days ago | parent | prev [-] | | [dead] |
|
|
| ▲ | lxgr 11 days ago | parent | prev | next [-] |
| > Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. Agency: What’s missing, in your view? Agency seems more of a property/function of a thinking system’s position in an environment than of the thinking itself. Subjective experience: That’s not a contradiction to “complex machines” either. I think the evidence that our minds are highly complex machines is, at this point, irrefutable. The question is really if they’re “only” that. |
|
| ▲ | LudwigNagasena 11 days ago | parent | prev | next [-] |
| > These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things I don't see how any of the works you referenced can account for that either? Since when is the problem of consciousness solved and we can definitely say what does or doesn't result in consciousness? |
| |
| ▲ | f_klem 11 days ago | parent [-] | | None of the works I mentioned solved or try to solve the mind/body problem or the issue of consciousness. They are a frame of reference for not stepping into the common fallacies that the AI research field is based on. | | |
| ▲ | LudwigNagasena 10 days ago | parent [-] | | What's exactly the fallacy? How do the works help avoid stepping into that "fallacy" if they don't try to solve the issue of consciousness. | | |
| ▲ | f_klem 10 days ago | parent [-] | | The issue of consciousness appears when you think of the world in a mechanistic way: since all there is are laws of physics and materiality, then how could we explain our though processes and our perceptual experience? If the world itself (in a general, existential way) is only made of laws of physics and matter, the consciousness needs to be an emergent characteristic of physical systems, and needs to follow the laws of physics. Now at this stage, you are already in trouble and you need to explain what consciousness is and how it manifestates. And that's the moment where things like the computational theory of mind appears. But you need to step back in order to detect the fallacy, one of which is: the brain/mind processes information like a computer, then we could build better computers that can think. This fallacy is assumed in the question 'can a machine think?'. There's another fallacy, which the author call the first step fallacy, which is common nowadays: we solved the language problem, then machines will be able to think in the near future. So it is not about solving the consciousness problem, it is about not claiming things based on assumptions that can be easily challenged. | | |
| ▲ | LudwigNagasena 7 days ago | parent [-] | | Of course any research programme requires some assumptions. But I don’t see any reason to call it a fallacy. Saying that something may be “challenged” or is problematic is just weasel wording. Either there are some serious issues that makes such theories ”flawed in the sense that they cannot account for subjective experience and agency, amongst other things”, or they are just normal theories. | | |
| ▲ | f_klem 7 days ago | parent [-] | | True, any research programme requires assumptions. The problem lies when those assumptions are either false or theoretical (unproved), and the community derives facts or claims from them. Behind the actual AI programme operate the following assumptions (at least): 1. A biological assumption, that states that the brain works similar to a digital computer. The reality is that we do not know. 2. An epistemological assumption, that states that we know how our brain works (or an even worse assumption, that states that we don't even need to know how it works, it is sufficient to replicate its observed behavior). This is rather simplified, the assumption in reality being (as stated by Dreyfus) that we think all intelligent behavior can be formalized as heuristic rules (Dreyfus' critique is based on GOFAI, since the book is pre-GAN/RL AI systems). But the assumption still applies: we think all intelligent behavior can be sampled, captured and formalized in (albeit complex) statistical systems. Dreyfus describes 4 or 5 in total, one of them is the psychological assumption, which states that the mind itself can be described as a digital computer (I think it might be outdated, since the actual debate is if something we could call 'mind' exists at all). There is also a fallacy called first-step fallacy, which states that if the first step towards intelligence is met, then the rest of the steps are of similar nature (technical). | | |
| ▲ | LudwigNagasena 6 days ago | parent [-] | | You say that "the community" derives facts or claims from unproved assumptions, yet at the same time you say that you "strongly tend to disagree" with those theories and that the theories are "flawed in the sense that they cannot account for subjective experience and agency, amongst other things" merely on account that they are neither confirmed nor unconfirmed. I am confused about your stance. You allow yourself to have strong opinions about something unknown yet criticize other people for the same. I think it is absolutely normal that the core of a theory is based on not directly testable assumptions. And it's normal that people push it forward if it bears fruits, that's not a fallacy in any way, that's normal inquiry that may or may not lead to successful results. | | |
| ▲ | f_klem 5 days ago | parent [-] | | > You say that "the community" derives facts or claims from unproved assumptions, yet at the same time you say that you "strongly tend to disagree" with those theories and that the theories are "flawed in the sense that they cannot account for subjective experience and agency, amongst other things" merely on account that they are neither confirmed nor unconfirmed. I am confused about your stance. You allow yourself to have strong opinions about something unknown yet criticize other people for the same. The assumptions I refer to are not only unproved, there is also increasing evidence that they are false. I do not criticize based on the assumption that there is subjective experience, but on the well developed idea that there must be something like 'subjective experience'. Here we enter the realm of philosophy, which by the way, is what science encounters when it runs out of answers. And this was precisely my point: AI research is based on assumptions that _need support or help_ from philosophy, not only neuroscience. But what is at stake here is the prevailing neurocentrism and scientificism characteristic of our era. > I think it is absolutely normal that the core of a theory is based on not directly testable assumptions. And it's normal that people push it forward if it bears fruits, that's not a fallacy in any way, that's normal inquiry that may or may not lead to successful results. That is correct and it is precisely why they are called 'theories': because the evidence points towards a specific direction but there is not yet enough evidence to call it a law. Yet different theories, based on different assumptions, demand that those assumptions be tested at the fundamental level: logical, epistemological, philosophical, etc. Regarding the theory that current LLM research could lead to human level intelligence, many people have the opinion that it can be discarded on fundamental grounds. Why? Because the assumptions that this theory stands on are flawed. An issue I repeatedly see in the community (about which Dreyfus already wrote in his 1972 book, confirmed in his 1992 book, and we still see today) is that challenging the fundamental flaws in which current AI research is based on immediately sparks outrage in the AI community, as if people challenging those assumptions are against AI or AI research at all. I think that is a really silly, childish and not very humble position, and ultimately slows down research. | | |
| ▲ | LudwigNagasena 5 days ago | parent [-] | | Now you again say that there is increasing evidence that the assumptions are wrong and that the foundation is flawed, but when you get to specifics you merely claim that something is unknown or unconfirmed. | | |
| ▲ | f_klem 5 days ago | parent [-] | | The books that I referenced at my first comment already contain an extensive explanation of why those assumptions are flawed, false, or ungrounded. I will not paste here parts of books. Nonetheless, I've been compiling references on different AI research assumptions and problems. I'll paste them here later on. | | |
| ▲ | LudwigNagasena 4 days ago | parent [-] | | If I take a text written on a tangential topic from a generation or two ago and try to imagine how it applies to the current state of AI that would be me putting words in your mouth and speculating on your interpretation. I don't want to waste time going over weak uncompelling critique like, if human ~~sound production~~ intelligence is analogue, then ~~digital speakers~~ artificial intelligence is unlikely to be possible (to paraphrase the critique of the Biological Assumption); but I am genuinely interested in the increasing evidence that shows that the foundation of modern AI research is flawed. | | |
| ▲ | ozgung 4 days ago | parent | next [-] | | After watching a lecture of Dreyfus on Heidegger and skimming his books I think I begin to see what's going on. Hubert Dreyfus teaches a specific set of philosophers beginning with Heidegger. His brother Stuart is an industrial engineer working on programming very early computers (50s) for Operations Research. They both worked at RAND corporation and involved in early "AI" research projects for military. Those AI projects of course were logic based problem solvers of 50s and 60s. But Hubert sees a problem with this work. What they do is incompatible with the philosophical tradition he belongs: "I began reading NSS's landmark papers with a mixture of excitement and fear. Perhaps Hobbes, Kant, and Husserl were right after all, and the human mind was an analytic engine. But then what about the seemingly plausible arguments of Merleau-Ponty, Heidegger, and Wittgenstein, which I had come to accept? As I read the RAND papers my excitement and fear turned to disappointment and relief." I think this was more like a cognitive dissonance than an actual contradiction. It was about choosing sides between Heidegger and maybe Descartes for him. That's why his objection sounds personal and dogmatic. So what was the big idea of Heidegger? Elephant in the room is the concept of "Dasein" (being-there), which Dreyfus think Heidegger is a genius for being the first philosopher noticing that. Dasein is an entity, with special mode of being. Only *human-beings* can be Dasein. So it's not an "object" like a table, that has properties (like in object oriented programming). It's not an equipment like a hammer (objects having methods). Human-beings, and only human-beings (definitely not LLMs, not dogs) can have this special way of being, or Existence. This idea of course has some roots in Christian Theology, as Heidegger himself. I think this is the reason of the strong opinions. A bit dogmatic. A belief that humans are fundamentally different beings. So an Equipment trying to mimic "Dasein" is categorically wrong (even impossible) in this belief system. The problem doesn't have to be Dasein itself, but you get the idea. It's either-or. If modern AI research is not flawed, their philosophy must be flawed. Since it can't be flawed, AI research must be flawed. Since research is trial and error, failures are the part of the process. But for them, each failure is an "increasing evidence". | |
| ▲ | f_klem 4 days ago | parent | prev [-] | | You can start by reading What Computers still can't do, by Hubert Dreyfus. Sorry to repeat myself: this books explains the assumptions the AI research programme is based on, and why they are problematic. It also references evidence. It also references claims from AI researchers (among others, Minsky), that were unfounded. Is the book still relevant today? yes, it is. Why? because the assumptions work at a fundamental level. You can then proceed with Metaphors we live by, by Lackoff and Johnson. The book shows how and why our understanding of the world is based on the fact that we are embodied beings. Then there is Being and Time, by Martin Heidegger. It shows how our understanding of the world is, again, based on the fact that we are embodied beings. Now, these are not newly edited books, and no, there is no real reason to think that because they are all +30 years old or even more, they are outdated. They are not. If you only look at the publication date, then Ramon y Cajal works would be totally crap (by the way, still one of the most cited works in neuroscience). It is from early 1900s. To complete the picture a bit, you could read: On the mode of existence of technical object, by Gilbert Simondon
Technics and Time, by Bernard Stiegler
Meditation on the technique, by Jose Ortega y Gasset
The question concerning technology, by Martin Heidegger These works will give an understanding of how technique in general (technology in particular) is completely anthropomorphisized, which is what ultimately leads to the assumptions present in the AI research programme. Also, A history of philosophy, by Frederick Copleston. Although extensive, reading volume I (greek and roman philosophy) is essential. More citations (again, if you really measure the quality or relevance of a philosophical/scientific work by its publication date, you are missing the picture): Arbib, M. A. (2025).* Artificial intelligence meets brain theory (again). Biological Cybernetics, 119, 16. https://doi.org/10.1007/s00422-025-01013-5 Farkaš, I., Vavrečka, M., & Wermter, S. (2025).* Will multimodal large language models ever achieve deep understanding of the world? Frontiers in Systems Neuroscience, 19, 1683133. https://doi.org/10.3389/fnsys.2025.1683133 Lin, Z. (2025).* Six fallacies in substituting large language models for human participants. Advances in Methods and Practices in Psychological Science, 8(3), 25152459251357566. https://doi.org/10.1177/25152459251357566 Seth, A. K. (2025).* Conscious artificial intelligence and biological naturalism. Behavioral and Brain Sciences, 1–42. https://doi.org/10.1017/S0140525X25000032 Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024).* Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011 Mitchell, M., & Krakauer, D. C. (2023).* The debate over understanding in AI's large language models. Proceedings of the National Academy of Sciences, 120(13), e2215907120. https://doi.org/10.1073/pnas.2215907120 Bowers, J. S. (2025).* The successes and failures of artificial neural networks (ANNs) highlight the importance of innate linguistic priors for human language acquisition. Psychological Review. Advance online publication. https://doi.org/10.1037/rev0000595 Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024).* Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011 Bolhuis, J. J., Crain, S., Fong, S., & Moro, A. (2024).* Three reasons why AI doesn't model human language. Nature, 627(8004), 489. https://doi.org/10.1038/d41586-024-00824-z Bender, E. M., & Koller, A. (2020).* Climbing towards NLU: On meaning, form, and understanding in the age of data. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5185–5198. https://doi.org/10.18653/v1/2020.acl-main.463 Everaert, M. B. H., Huybregts, M. A. C., Chomsky, N., Berwick, R. C., & Bolhuis, J. J. (2015).* Structures, not strings: Linguistics as part of the cognitive sciences. Trends in Cognitive Sciences, 19(12), 729–743. https://doi.org/10.1016/j.tics.2015.09.008 Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002).* The faculty of language: What is it, who has it, and how did it evolve? Science, 298(5598), 1569–1579. https://doi.org/10.1126/science.298.5598.1569 Johnson, M., & Lakoff, G. (2002). Why cognitive linguistics requires embodied realism. Cognitive Linguistics, 13(3), 245–263. https://doi.org/10.1515/cogl.2002.016 Lakoff, G. (2012). Explaining embodied cognition results. Topics in Cognitive Science, 4(4), 773–785. https://doi.org/10.1111/j.1756-8765.2012.01222.x Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1–3), 335–346. https://doi.org/10.1016/0167-2789(90)90087-6 Placani, A. (2024). Anthropomorphism in AI: Hype and fallacy. AI and Ethics, 4, 691–698. https://doi.org/10.1007/s43681-024-00419-4 Salles, A., Evers, K., & Farisco, M. (2020). Anthropomorphism in AI. AJOB Neuroscience, 11(2), 88–95. https://doi.org/10.1080/21507740.2020.1740350 Floridi, L. (2025). AI as agency without intelligence: On artificial intelligence as a new form of artificial agency and the multiple realisability of agency thesis. Philosophy & Technology, 38(1), 1–27. https://doi.org/10.1007/s13347-025-00858-9 <<< I am not convinced by his position, but it is nonetheless relevant since it splits the debate in two: intention and consciousness Dreyfus, H. L. (2007). Why Heideggerian AI failed and how fixing it would require making it more Heideggerian. Philosophical Psychology, 20(2), 247–268. https://doi.org/10.1080/09515080701239510 Bengio, Y., & Elmoznino, E. (2025). Illusions of AI consciousness. Science, 389(6765), 1090–1091. https://doi.org/10.1126/science.adn4935 Dotov, D. G., Nie, L., & Chemero, A. (2010). A demonstration of the transition from ready-to-hand to unready-to-hand. PLOS ONE, 5(3), e9433. https://doi.org/10.1371/journal.pone.0009433 Mitchell, M. (2021). Why AI is harder than we think. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2021). https://doi.org/10.1145/3449639.3465421 I haven't read all of them yet. Feel free to discuss. Now, there are two problems I see in the community regarding the critique of AI. One is the problem of the increasing capability of models. The other is the idea that GOFAI and ANN-based systems (like LLMs) are fundamentally different. Let me explain. 1) The increasing capability of models: it is difficult to engage in any meaningful discussion if the metrics are the capability of models. One should look at how models structurally encode information and what the learning process looks like from an epistemic point of view. As far as I know, and correct me if I'm wrong, these two issues have not changed and are likely not going to change. 2) The idea that GOFAI and ANN-based systems are fundamentally different: this, I recognize, is a controversial claim. But one should not look at how GOFAI and ANN-based systems encode knowledge (explicitly curated and written rules vs statistical learning), but at how the learning material is selected, curated and presented to the system, and the problem of 'closure' and self-reference in datasets. In this regard (which we could call epistemic) there should be no difference between these two technologies. Again, we should not look at how they are implemented, but at how we relate to them from an epistemic point of view. But going back to my initial comment, this whole thread feels like proving my point.
For those not wanting to get involved in philosophy despite willing to engage in AI research discussions, keep in mind that philosophy has always been a guiding light for science. As a final note: the whole discussion about AI is on whether computational theories of mind are actually solid or not. But it is really difficult to engage in this conversation without at least some background in philosophy, but preferably a strong background in it. I'm getting a bit tired of coming back to this thread. Reach me out if you want to discuss more. Glad to help and glad to learn. @federico_ricca
https://www.linkedin.com/in/federico-ricca | | |
| ▲ | LudwigNagasena 3 days ago | parent [-] | | > But going back to my initial comment, this whole thread feels like proving my point. Ok, but look at this thread from the POV of someone like me, who reads lots of philosophy daily and who asks you to simply elaborate on your point, and you just continuously keep deflecting from providing an actual argument. Even in this comment instead of providing an actual philosophically grounded argument based on all those literally great thinkers many of whom I’ve read with great passion you waste your energy on name dropping things that don’t directly support your initial thesis in any way and then you do some meta-commentary on the impossibility of discussing those issues that you can’t even clearly articulate for several days. |
|
|
|
|
|
|
|
|
|
|
|
|
|
| ▲ | derektank 11 days ago | parent | prev | next [-] |
| >Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things Do you disagree with the assumption that cells are machines? They seem pretty machine-like to me. I certainly don’t think individual cells have any subjective experience or sense of agency. I would be curious to know where your intuitions diverge here, because if the mind is an emergent phenomenon from machines (cells) then it seems quite likely that a mind could emerge from other, different machines. |
| |
| ▲ | frig57 11 days ago | parent | next [-] | | It's not a huge leap to imagine biological cells might have a rudimentary consciousness Pan-psychists might argue that your subjective consciousness is an aggregate of all the cells/molecules, etc in the system "While each biological cell operates largely on its own chemical cues, they all coordinate through complex nervous and chemical networks to create your unified, subjective experience." You might be a rigid materialist | | | |
| ▲ | f_klem 11 days ago | parent | prev | next [-] | | > Do you disagree with the assumption that cells are machines? They seem pretty machine-like to me.
You just said it: they are machine-like. No, cells aren't machines for two reasons:
1) machines, by definition, are artifacts created by human beings.
2) The nature of a living organism is completely different from that of 'machines'. (even if we a re able to replicate a cell in a lab, like the group from Craig Venter did). Autopoiesis being one very big difference, another being emergence-within-the-environment (life) vs design-conditioned-by-will (machines) > ... if the mind is an emergent phenomenon from machines (cells) then it seems quite likely that a mind could emerge from other, different machines.
Since cells cannot be defined as machines, the argument about mind emerging from machines does not hold. | |
| ▲ | cootsnuck 11 days ago | parent | prev | next [-] | | > I certainly don’t think individual cells have any subjective experience or sense of agency. There's definitely research and scholarship that would beg to disagree with you there. At least in terms of completely writing off the notion of "agency" when it comes to cells. Dr. Michael Levin's lab is doing some pretty cool work. https://drmichaellevin.org/ | |
| ▲ | gyomu 11 days ago | parent | prev [-] | | > Do you disagree with the assumption that cells are machines? Yes? Literally no machine ever built by humans is capable of (or even hinting at beginnings of capability for) replication or novel synthesis like cells are, let alone autonomously, it’s quite unconceivable that anyone would take this to be a reasonable assumption in the first place. | | |
| ▲ | derektank 11 days ago | parent | next [-] | | Hmmm, well let’s take it one step lower. What do you think of organelles such as ribosomes? Do you disagree with the assumption that those are machines? They seem directly analogous to the jacquard loom or a CNC machine to me. | | |
| ▲ | gyomu 11 days ago | parent [-] | | Yes again, ribosomes have nothing in common with machines, that are built and designed by humans. The ball is in your camp to provide solid reasons to believe why they should be grouped together, when one is a deeply complex interrelated dynamic system (in fact, arguably the most complex system we know of) evolved bottom up over billions of years that we only very partially understand and cannot fully explain or document, and the other something entirely planned, designed, and produced by humans in which every component is finite and accounted for. The argument boils down to “well the vibes kind of match to my taste, and it’s the best analogy I have in my analogy toolkit”, which is just not serious reasoning. | | |
| ▲ | derektank 11 days ago | parent [-] | | I think ribosomes have a lot in common with machines. They use energy to accomplish a task (assembles proteins). This would seem to put them in the same category as artificial molecular machines like rotaxanes. I don’t think there’s a huge gap in our understanding of how either systems identically functions independently. Yes, ribosomes exist as part of a larger context, but they can be removed from that context pretty easily and understood as individual molecules quite extensively. In your view, can machines even exist that haven’t been created by people, definitionally? I, personally, don’t see the relevance of intent but that seems to be the only distinguishing factor here. |
|
| |
| ▲ | wat10000 11 days ago | parent | prev | next [-] | | The assertion is not that cells are machines built by humans (obviously false), merely that they are machines. Which they pretty clearly are, unless you assume the supernatural. | | |
| ▲ | f_klem 11 days ago | parent [-] | | There is no need to assume the supernatural.
They are machines based on what?
By definition, a machine is something build by humans, or any intelligent life form (because we extend the definition from our own experience as self defined 'intelligent life forms').
Living organisms and machines do share traits under our current frame of reference in modern science. But that doesn't mean cells are machines. The right framing would be 'cells exhibit behavior and certain internal relations that we can conceptualize them as machines', which is different from 'cells are machines'. | | |
| ▲ | wat10000 11 days ago | parent [-] | | By what definition? Anyway, seems like an argument over said definitions rather than the underlying characteristics. The relevant question is whether they're purely physical objects behaving according to rules, which is being described as "machine," or whether there is something beyond that. Current understanding is contradictory: all indications are that cells and bodies are purely physical objects, except that there is this phenomenon of subjective experience which doesn't fit with that at all. | | |
| ▲ | f_klem 11 days ago | parent | next [-] | | > By what definition? We have this definition from the history of western languages, the history of western philosophy, and works by Lewis Mumford and Franz Reuleaux, among others authors. > The relevant question is whether they're purely physical objects behaving according to rules, which is being described as "machine," or whether there is something beyond that. Current understanding is contradictory: all indications are that cells and bodies are purely physical objects, except that there is this phenomenon of subjective experience which doesn't fit with that at all. Then you would say that you dog broke (died), or that the vet fixed your cat (cured). Which by all means we might speak that way, but surely you would notice that it is not accurate. Saying that a living organism is just a machine because it is a physical object behaving according to rules is like saying that a beautifully built house is just a bunch of bricks layed in rows and something on top. But assuming that we say 'purely physical objects behaving according to rules' are machines, then: 1) there is no difference between you, your dog, your fridge and the snail in you garden
2) It would be semantically valid in all languages to say 'my dog broke' instead of it died, 'i got fixed' instead of 'the doctor cured me'
3) Machines vary in complexity and we would still have 'degrees of complexity' regarding machines (human bodies being the most complex, perhaps, toasters being fairly simple), but fundamentally, all of them would follow the same rules for 'fixing', 'breaking' and 'repairing'. Which is not the case.
4) You would have to come up with some kind of theology regarding how we were built. There is no evidence that we have been 'built', quite the contrary. And most probably there are quite more reasons not to regard machines as living organisms nor living organisms as machines. | |
| ▲ | gyomu 10 days ago | parent | prev [-] | | If your definition of machine is "purely physical object behaving according to rules"[0], then one could argue that everything in the universe is a machine, which is not a very useful definition. [0] where I assume you mean rules = "laws of physics", because if we were to choose the more conventional definition of rule = "an accepted principle or instruction that states the way things are or should be done", then your own definition doesn't apply to cells or other biological entities | | |
| ▲ | wat10000 10 days ago | parent [-] | | I didn’t say that was my definition, I said that was the actual important question here. |
|
|
|
| |
| ▲ | NiloCK 11 days ago | parent | prev | next [-] | | replication: https://en.wikipedia.org/wiki/Quine_(computing) autonomous replication: https://en.wikipedia.org/wiki/Computer_worm nb that writing your own quine remains in general terms a fun and challenging exercise in many programming languages, but not python. | | |
| ▲ | gyomu 11 days ago | parent [-] | | We are talking about physical replication, please let me know when a computer worm can turn my laptop into 2 laptops Otherwise you’re just arguing that Sims are totally alive because Sims can make baby Sims. |
| |
| ▲ | Sharlin 11 days ago | parent | prev [-] | | What exactly is there about cells that's inconceivable to replicate or synthetize? "I cannot conceive it" is a fallacy, an argument from incredulity. Trying to disguise it by draping it in the passive voice does not change that fact. | | |
| ▲ | f_klem 11 days ago | parent [-] | | Nothing. It is just that they are so incredibly complex, that after +100 years of research we still don't know how they work or why.
Will there be a time when we finally understand them 100% and could replicate them? Could be. That does not make them machines in any case, and the fallacy of thinking that mind/consciousness can emerge from machines will still be there. |
|
|
|
|
| ▲ | phkahler 11 days ago | parent | prev | next [-] |
| >> and that is that we understand (or we think we understand) how our brain/mind works. But the truth is that we don't know. And there's even not a single clue that we actually know too much, and not a clue that our brain/mind and cells work 'as the machines we build'. >> I highly recommend people in the AI research space should read philosophy and modern linguistics. I highly recommend the philosophers read some neuroscience. The whole "model weights" thing in AI is modeled after the synaptic connections and between actual neurons. There is already quite a bit known about how the brain works at a low level. There is also a lot that is still unknown. There are also differences between discrete neuron firing and weights as signals, but there is enough similarity to make artificial neural nets useful and do things similar to what real one do. |
| |
| ▲ | dbspin 11 days ago | parent | next [-] | | Hard disagree. We only discovered the role that glial cells play in processing around 2014. We're still uncertain how patterns of activation consolidate through long term potentiation, let alone how signaling encodes information. We understand quite a bit about the role of the hippocampus and subiculum in encoding memories; but we don't understand the structural layout of engram complexes - which were themselves mapped for the first time only in 2022! Taking effective results in machine learning, and somehow assuming that they apply to cognition - simply because neural nets were inspired by our limited knowledge of neural signaling and structure - is like trying to apply aircraft engineering to studying ornithology. For a better articulation of this point (from the reverse direction) check out the paper 'Could a Neuroscientist Understand a Microprocessor?' from 2017 - https://journals.plos.org/ploscompbiol/article?id=10.1371/jo... | | |
| ▲ | phkahler 11 days ago | parent [-] | | >> We understand quite a bit about the role of the hippocampus and subiculum in encoding memories... Hard disagree ;-) You're talking about high level architecture of the brain. I don't think (not my area I may be wrong) we know how memories are encoded in a real brain. Is it weights or something else? If it's weights that's supporting my point (but we don't know what the weights represent in a brain, where in LLMs many weights are just token encodings). If brains store memories in something other than weights I'd really like to know as it's something I haven't read about yet. |
| |
| ▲ | californical 11 days ago | parent | prev | next [-] | | > There are also differences between discrete neuron firing and weights as signals, but there is enough similarity to make artificial neural nets useful and do things similar to what real one do. There is barely a surface-level similarity. The best example I can come up with is this… Imagine the most intricate and beautiful tall building that you can think of. Think like an older skyscraper in Chicago or a palace. There are water features and moving parts everywhere but also tiny little handmade carvings and materials throughout. Now imagine we have no reference designs and no blueprints - we hire an architect to attempt to study the building by looking at it from a distance and understand everything they possibly can about it. She can go into the building to check but every time she does, it stops functioning normally. That architect is a neuroscientist. Then the ML researcher is like a graphic designer who sees the work that the architect is doing and makes a napkin sketch of the building the architect has been studying, to use for a project later. Sure the designer has some of her representations. But the difference in complexity between the designer’s napkin sketch and the architect’s analysis is massive. Several orders of magnitude. Then another many orders of magnitude is the level of detail that the architect can understand about this strange building without being able to fully interact with it, versus the actual complexity of the building. So yeah, an AI is modeled after neurons in the sense that they represent a couple of surface level features of neurons. But the difference in complexity is about as much as a napkin drawing of a grand building represents the actual structure and details of the building, no matter the level of skill that the graphic designer has | |
| ▲ | f_klem 11 days ago | parent | prev [-] | | > I highly recommend the philosophers read some neuroscience
Philosophers do read neuroscience and technical reports on AI. > The whole "model weights" thing in AI is modeled after the synaptic connections and between actual neurons
In reality, it is a really poor and basic model of what is actually happening in a real brain Brains and modern AI systems (LLMs for example) are structurally different.
(Don't get confused by topology. A structure is more than topology: it is also what the structure is made of, thus the properties of the material contribute and define what can emerge atop the structure) |
|
|
| ▲ | truculent 11 days ago | parent | prev | next [-] |
| > These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. That agency or free will exists outside of our subjective experience is an assumption; does any given theory need to explain agency, or is it sufficient to explain that we feel we have agency? |
| |
| ▲ | f_klem 11 days ago | parent [-] | | Indeed, I know there is research going on placing agency and free will as illusions of our own subjective experience. Yet, there is a big gap from a purely mechanistic view of the mind to the idea/phenomena of subjective experience. |
|
|
| ▲ | bwfan123 11 days ago | parent | prev | next [-] |
| > should read philosophy subject/object dichotomy is a springboard to many schools of thought. 1) that subject emerges from objects - ie, anything has a material explanation, and everything is a machine. 2) that objects emerge out of a subject as a world model (platonic, descartes) 3) the subject and objects are one and the same representation of nature (spinoza) 4) subjects and objects emerge and disappear together (buddhist) Anything reduced to computation is a fixed function
from input to output, and is "dead" in the sense that it is unadaptable to its environment. Weights therefore is a dead machine. Another view of this is that any closed system has unanswerable questions within it. Therefore, there is no system that can encompass everything. Hence weights being a closed system doesnt encompass everything. |
|
| ▲ | ozgung 11 days ago | parent | prev | next [-] |
| > I highly recommend people in the AI research space should read philosophy and modern linguistics. On the contrary, I highly recommend people in Philosophy of mind and linguistics should start reading AI research papers because their theories and ideas are highly outdated, even ancient. Your books are from 1927 and 1972 respectively and Turing's article is from 1950s. And they are relatively new with respect to other works in Philosophy. If one doesn't adequately understand what we have in 2026, how can they theorize about it? As others they don't understand how the mind/brain work, BUT ALSO they don't understand how the AI works. Also with this mindset that we can't understand seemingly complicated things, there would be no advancement in science and technology. I think philosophy people and Linguist will catch up in a century, like they did with Turing. Philosophers of this century are not in humanities or literature. They are in science and engineering. Heidegger was trained on priesthood and Theology. You should read greater minds like Hinton, LeCun etc. if you want to think on these things. They are the real Philosophers. |
| |
| ▲ | Avicebron 11 days ago | parent | next [-] | | Yikes dude. You may need to expand your horizons a bit. | | |
| ▲ | ozgung 11 days ago | parent [-] | | That's my point. Everything from 1927 is already in plain sight and a part of the current public knowledge. Horizon can only be expanded at the cutting edges. | | |
| ▲ | jhonof 11 days ago | parent [-] | | > Horizon can only be expanded at the cutting edges. Ironically this was advice Ralph Waldo Emerson gave in 1840 so by your logic it's irrelevant just because it's old. |
|
| |
| ▲ | f_klem 11 days ago | parent | prev [-] | | > On the contrary, I highly recommend people in Philosophy of mind and linguistics should start reading AI research papers because their theories and ideas are highly outdated, even ancient. Your books are from 1927 and 1972 respectively and Turing's article is from 1950s. And they are relatively new with respect to other works in Philosophy. People in philosophy and cognitive linguistics do read AI research. Don't get fooled by the publishing dates: although Heidegger's work dates from 1927, the work is contemporary. The same happens with Dreyfus' work. Again, publishing dates don't mean anything here.
Maybe you can clarify why they are outdated. > If one doesn't adequately understand what we have in 2026, how can they theorize about it? As others they don't understand how the mind/brain work, BUT ALSO they don't understand how the AI works. I would say that people involved in the critique of AI do know how it works. But I've found that is normally the case that people in AI research does not have the framing provided by works in philosophy or cognitive linguistics. > I think philosophy people and Linguist will catch up in a century, like they did with Turing. Philosophers of this century are not in humanities or literature. They are in science and engineering.
What do you base your claims on? Plenty of philosophers work in humanities, literature, sociology as well as science and engineering. Philosophers not catching up? The critique on automation and AI already dates from the early 20s if not before. > Heidegger was trained on priesthood and Theology. You should read greater minds like Hinton, LeCun etc. if you want to think on these things. They are the real Philosophers. Sorry, but this does not make too much sense. Hinton and LeCun are great in their own fields. But seriously, they are not philosophers, they are inventors. | | |
| ▲ | ozgung 10 days ago | parent [-] | | > Maybe you can clarify why they are outdated. The First Edition (1995) of the classic textbook Artificial Intelligence: A Modern Approach by Russell and Norvig talks about the criticisms of Dreyfus quite extensively. In the second edition (2003) they conclude:
"In sum, many of the issues Dreyfus has focused on-background commonsense knowledge, the qualification problem, uncertainty, learning, compiled forms of decision making, the importance of considering situated agents rather than disembodied inference engines-have by now been incorporated into standard intelligent agent design. In our view, this is evidence of AI's progress, not of its impossibility." In the 4th edition (2020) Dreyfus reduces to a paragraph and Heidegger is just a reference in a footnote. | | |
| ▲ | f_klem 10 days ago | parent | next [-] | | I have that book at home and I'll check as soon as I get back from a trip. When Dreyfus' book appeared in 1972, it received really harsh criticism from the then AI community. Dreyfus actually comments on that criticism in the revised 1992 edition. I just don't see how Dreyfus critique of AI has been dealt with in modern AI: the critique is aimed at fundamental issues, not at the technical issues. It is true that the critique written by Dreyfus is based on GOFAI algorithms from the 60s, but it is also true that if you read the book today, you'll find lots of similar situations and a similar way of thinking about the possibilities of AI, as well as the same underlying assumptions. And as a side note, outdated means that it does not apply anymore, or that is not relevant anymore. Which is different from 'establishing a dialogue' with the text/author, in a way that 'seems' not to be relevant anymore. If you say that Dreyfus' book is outdated just because the 4th edition of Norvig's text only mentions it in a footnote, you are assuming that Norvig and Russell's opinion are definitive. They might be not. I have authors like Norvig, Russel, LeCun, Minsky and other in the field in high regard. But they are normally not trained in either modern linguistics nor philosophy. Let alone the rest, large amount of researchers in the field. AI research is a complex field, and maybe (in this we could follow Foucault) not even a science. Doing research in an area of study does not turn it into science. It is precisely philosophy, and even more contemporary philosophy, the discipline that focuses on how we build knowledge, and how we experience the world. Two really important, almost fundamental, topics that directly contribute to how AI is developed as a field of knowledge. | |
| ▲ | f_klem 7 days ago | parent | prev [-] | | I have the third edition, so I can only speak for it. Being the 3rd edition of the book, I assume that it is the 3rd time the text is revised, so I expect the other two editions (1st and 2nd) to adolesce from the same problem, which I state in the following paragraphs. The mention to Dreyfus in the 3rd edition of Artificial Intelligence, a Modern Approach, by Stuart Russell and Peter Norvig, is made in 4 different places of the book, referencing four different problems. The first mention is in page 279, effectively in the bibliographical notes, and it is about something called the 'frame problem'. Dreyfus presents this problem in the 1972 edition of the book, as a problem pertaining 'how to differentiate figure from ground', or 'how to account for what is important and what is not in a specific scenario'. But the solution to the problem that Norvig and Russel cite (Ray Reiter, 1991) is from a paper that _changes the conditions of the problem_, even _change the problem completely_ (by reductionism) to 'how to detect objects that do not change after an action'. They claim the problem solved, but they are actually not addressing Dreyfus criticisms, and misleading the reader to think that the problem is actually solved. The frame problem, by now, is still unsolved (and is one of the most difficult problems to solve). The second mention is in page 1024, under a section called 'Weak AI: Can machines act intelligently?', and subsection 'The argument from informality'. The section mentions the books What Computers can't do (1972) and What Computers still can't do (1992), as well as Mind over Machine (1986). Unfortunately, this section completely misunderstands the critique of AI that Dreyfus exposes in those books. The whole section is misleading, obfuscating or tergiversing the critique from Dreyfus to fit the purpose of Norvig and Russell (mainly, to show that advances in machine learning and AI can make a solid base for machines that 'act intelligently'). The third mention is in page 1049, and it tries to undermine the first-step fallacy (which is similar to the fallacy of composition). Again, they do it by completely dismissing Dreyfus' critique, not addressing the issue. Then they go on talking about 'rationality' (as explained in chapter 1), but with a trick: only in terms of machines, goal-oriented expectations, computing resources. Dreyfus' critique is about the overall AI enterprise and the search for 'artificial' intelligence, Russell and Norvig discourse in this section first reduce Dreyfus' critique to what they can handle, to their own terms. That is, they evade the issue. The fourth and final mention, in page 1072, is the bibliographical citation. Re-reading the non-technical, but more theoretical parts of the book just made me realize how poorly constructed the book is. For example, the definitions given about AI in page 2 are just laughable. Compare with an introductory text on Psychology [0]. [0] https://pressbooks.openeducationalberta.ca/saitintropsycholo... |
|
|
|
|
| ▲ | red75prime 11 days ago | parent | prev | next [-] |
| > There is also an epistemological assumption that prevails, and that is that we understand (or we think we understand) how our brain/mind works. It's good that it doesn't matter. Stochastic gradient descent works (or doesn't work) regardless of whether we know how the brain does its thing. |
| |
| ▲ | dnautics 11 days ago | parent [-] | | we know the brain does not use gradient descent. (i agree it doesn't matter) | | |
| ▲ | anthonyrstevens 11 days ago | parent [-] | | How do we know this? | | |
| ▲ | dnautics 11 days ago | parent [-] | | i recall a hinton lecture where he talks about the mechanism of memory formation in the brain and it's not fully sussed out but it's also not backprop, its some sort of forward prediction and immediate reinforcement loop. There's also no plausible biological/chemical mechanism to backpropagate. |
|
|
|
|
| ▲ | foolswisdom 11 days ago | parent | prev | next [-] |
| This reminds of this article which contends that this mistake has carried across time, in every era people described the workings of the brain similar to the machines they knew how to build, which is why we have "ridiculous" descriptions that changed over time. https://aeon.co/essays/your-brain-does-not-process-informati... |
| |
| ▲ | thewoodsman 11 days ago | parent [-] | | I like Leibniz's comparison of the brain to a mill, which he invokes in an argument[1] that seems to be the predecessor of the modern "hard problem:" > supposing that there were a mechanism so constructed as to think, feel and have perception, we might enter it as into a mill. And this granted, we should only find on visiting it, pieces which push one against another, but never anything by which to explain a perception. [1]: https://en.wikipedia.org/wiki/Leibniz%27s_gap |
|
|
| ▲ | runarberg 11 days ago | parent | prev | next [-] |
| It is also a story as old as time. We have been comparing our brains to our most recent information/communication technology since the invention of the telephone. After the telephone our brain was like computers, and when I did my bachelors in Psychology in 2009, we were comparing our brains to the internet. AI is simply the latest iteration of these comparison, just as wrong and unhelpful as when we compared it to the telephone. |
|
| ▲ | mahogany 10 days ago | parent | prev | next [-] |
| > What all these theories have in common is the underlying belief that our brain/mind works as the machines we build It’s interesting to note that this is not a new phenomenon. If you read writers from the past, they were always comparing the body or the brain to whatever the modern machine, or idea of a machine, was at the time, whether a steam engine or automata. |
|
| ▲ | ACCount37 11 days ago | parent | prev | next [-] |
| I don't think cognitive linguistics has that much to say about AI nowadays. Let alone philosophy. The biggest contributions from linguistics are probably "human languages mostly have statistical regularities rather than hard rules" and "the sum of data humans get from birth to language acquisition is insufficient to learn a language from scratch". Which LLMs already work with, and work around, respectively. From there, nothing. And philosophy just exists to be a distractor. "Subjective experience" is too subjective to matter in practice. "Task performance" is measurable, "consciousness" isn't. "Agency" is something an LLM in a tool calling loop, a rat in a maze and a human in an office tower may or may not have, depending on your favorite definition. Agentic LLMs are years in the making, and that's a product of engineering, not philosophy: "agentic" is whatever gets the job done. We are yet to discover any physical process whatsoever that can't be represented as mathematical operations and implemented by a Turing machine. So all of that "treating human mind as a machine is wrong" amounts to "human mind must be powered by magic fairy dust" paired with "a functionally similar magic-free replacement is impossible". I'm not about to give much weight to any hypothesis that requires undiscovered magic fairy dust. At least find the hyper-computational magic fairy dust first - not just assume it absolutely must be there because you want the human mind to be unique and special. Want to know why Turing did what he did? It's because he didn't want to engage with any of that "what is mind" bullshit either. So he proposed actual metrics - measurables that are harder to argue in circles about. Not that it stopped anyone. But at least he tried. |
| |
| ▲ | f_klem 11 days ago | parent [-] | | > I don't think cognitive linguistics has that much to say about AI nowadays. Let alone philosophy.
I would like to read those sources. > The biggest contributions from linguistics are probably "human languages mostly have statistical regularities rather than hard rules" and "the sum of data humans get from birth to language acquisition is insufficient to learn a language from scratch". Which LLMs already work with, and work around, respectively. From there, nothing. Again, what's the source for 'biggest contributions from linguistics are...'? It is a big contribution to the development of LLMs, but different cognitive linguistics authors already challenged this idea already 20-30 years ago. LLMs work with and around the problems you cite because of massive data/money, not at the fundamental level. It is all a game of statistics and data, which has been already challenged by cog. ling. > And philosophy just exists to be a distractor.
Well, this is just telling me that you either know too much about philosophy and reached that conclusion (which might make sense, know of some philosophers who also think that) or you just read too little. > So all of that "treating human mind as a machine is wrong" amounts to "human mind must be powered by magic fairy dust" This is the common fallacy people in AI/IT make . One of the benefits of reading philosophy is that you find your way out of them. > Want to know why Turing did what he did?
The actual tests Turing though about are themselves flawed (not that I discovered that, has been known for some time already) | | |
| ▲ | ACCount37 11 days ago | parent [-] | | Again: either stop using > quotes or learn to use them better. Fucking unreadable. I reiterate: philosophy is almost entirely worthless for AI design. We want to design systems that work, not systems that sound good on paper. If philosophy had a practical application in that, we'd stop calling it "philosophy" and start calling it "math", "science" or "engineering". | | |
| ▲ | f_klem 11 days ago | parent [-] | | Again: don't be rude. Not nice. As 'not nice' as my bad formatting. Philosophy is there precisely to critique and analyze what we build and how we behave in the world. Without it, by the way, there would be no science and no engineering. Dismissing philosophy in AI is like dismissing philosophy in any other applied, practical or creative field. It is precisely because there is a philosophical investigation on each practical, applied or creative field, that that field can actually make progress. There's philosophy in biology, which helps biology go further. Philosophy in engineering, which makes engineering go further. In architecture, film, photography, painting, literature, medicine, physics, linguistics, mathematics, etc, etc. In all those fields, there is also a philosophical investigation taking place. Right now. Maybe you confound practical AI design (you should be talking of 'building AI systems') with AI as a research field. I get that, because lots of people cannot make the distinction (calling yourself 'AI researcher' when you actually tinker a model is not scientific. It might follow a scientific method, but it is engineering research, not scientific research). |
|
|
|
|
| ▲ | raincole 11 days ago | parent | prev | next [-] |
| > subjective experience and agency, amongst other things In other words, souls? I'm sorry if that sounds accusing, but to me it sounds like you're talking about souls that are independent to the physical world, just with more 'scientific-y' wording. (I fully understand that some people believe souls exist.) |
| |
| ▲ | f_klem 11 days ago | parent | next [-] | | As other commenters pointed out, I'm not getting into the mind/body problem.
But that's what people normally do: you challenge the assumptions, and all of a sudden they assume you are already coming to a conclusion.
And no, just because I challenge the assumptions, I don't need to provide a counter conclusion. | |
| ▲ | sarchertech 11 days ago | parent | prev [-] | | I don’t think the OP is discussing mind-body dualism. There are many materialists who believe that consciousness maybe a non-computational process. But it is worth pointing out that something like 80% of the world (it fluctuates depending on the survey but its around that) believes in some non-physical spirit, life force, or soul. It’s a very HN bubble thing to start a discussion with the assumption that everyone must be a materialist. | | |
| ▲ | raincole 11 days ago | parent [-] | | I acknowledge and understand that many people believe in the existence of souls. But I hope they just say it rather than try to wrap the concepts in layers of jargons. | | |
| ▲ | sarchertech 10 days ago | parent [-] | | Again I don’t think the person you were talking to was doing that. But one of the reasons that people might do that is because on a forum like hacker news they’ll be attacked as a lunatic. Just look at some of the comments on this article. I saw one where someone said that dualism is so ridiculous that it can’t even be considered philosophy. |
|
|
|
|
| ▲ | paulluuk 11 days ago | parent | prev | next [-] |
| > These theories are flawed in the sense that they cannot account for subjective experience and agency So, I work in AI research (as a research engineer though, not a scientist). I've also studied philosophy and I'm a vegan. Yes yes, insert "they will tell you" joke here, but I promise it's actually relevant this time. First, while I studied philosophy one of the things that stuck with me the most, was the discussion of "souls": humans have souls, animals don't. For centuries the specifics of souls were discussed: people would be weighed while they died, in an effort to measure the approximate weight of a soul as it departed the body. Discussions about how many souls (or angels) could dance on the tip of a needle. Many people still believe in souls, but it's very hard to have a real discussion about them because by definition they do not "interact" with this world in a way that can be measured. When discussing whether it's okay to harm animals for food or sport, one other argument I hear quite often (other than having no souls) is that animals do not experience "qualia": basically the smallest unit of "subjective experience". People know that they themselves experience qualia: the sensation of touching a doorknob, the taste of fresh fruit, the sense of beauty watching a rainbow. Ironically, they would say that animals are like robots: just (biological) machines acting on instinct, and feeling any kind of compassion for them means you are anthropomorphizing. Subjective experience (or at least qualia) and souls both have one thing in common: they can not be measured externally in a meaningful way. You can simply state that an AI system no matter how advanced, has no soul and has no subjective experience. And that's pretty much that. There's no meaningful discussion to be had about it, because no matter what an AI might tell you: it has no way to prove it to you. In fact, you can't even be certain that anyone other than yourself has subjective experiences: you assume that because they are humans like you, and you experience them, that they probably do as well. They tell you that they do. But a human without subjective experience, someone on "autopilot", would be absolutely indistinguishable from a human who does have them. But perhaps I am conflating here whether experience can be "measured", with whether a system even allows experience in the first place regardless of whether it can be measured. I think that Dreyfus and others argue that in order to have any "experiences" at all, you simply must have a body in the real world, and you must care about that body. Please correct me if that's the wrong interpretation, I haven't actually read the book. That argument would be harder for me to discuss, because I personally believe that consciousness will "emerge" from a complex interaction of relatively simple systems - but that's also just a theory. I don't believe that experience is literally impossible to engineer, as consciousness has emerged from non-conscious being through evolution, so clearly there must be some kind of mechanism for it -- and if there is, then I believe it can be replicated, we just don't really understand it well enough yet to do so. And with how AI tech is going, I think that we're more likely to accidentally stumble upon it than we are to get these deliberately. |
| |
| ▲ | lobofta 11 days ago | parent | next [-] | | Vegan ML engineer here. In total agreement with you. People are just moving the goal post to keep themselves protected from the obvious conclusion: there is nothing really all that special about us humans. Perhaps subjective experience is simply the internal state of a self supervised continuous learning algorithm and we don't like that conclusion very much. | | |
| ▲ | foobarian 11 days ago | parent [-] | | It's too bad it's so hard to pin down a definition, but in practice I feel like most animals with brains experience degrees of qualia. Some mornings after a night of poor sleep when I wake up super-slow I wonder if that's how animals experience thinking. My biggest problem with "brains are machines" arguments is that there is a risk there is unknown physics at work that is not representable as a Turing machine. What if there is some quantum field effect powering everything? | | |
| ▲ | jcynix 11 days ago | parent | next [-] | | Quantum field effects? You don't need these, IMHO, if you look at how highly parallel things seem to work in brains. Marvin Minsky's theory of a "Society of Mind" describes a (highly) distributed model of the mind. Which BTW, always reminds me of the first Shrek movie, where the donkey jumps up and down, shouting "Take me! Take me!" to Shrek. That's similar to what I observe when I'm undecided but two instances of "sub-processes" (or agents as Minsky calls them) of my mind try to get attention. Daniel Dennett similarly gives a distributed model of consciousness. Where many parallel "processes" are at work, competing and "observing" each other. And this parallelism is happening with a much, much higher degree than any of our computers parallelism. | | |
| ▲ | foobarian 11 days ago | parent [-] | | Mostly I have a hard time accepting that a Turing machine can experience "consciousness"/awareness. Therefore I also have a hard time with simulatable chemical processes; it feels like there is some missing link there. All just feelings/vibes unfortunately. | | |
| ▲ | jcynix 11 days ago | parent [-] | | I can sympathise with you, but how could a "quantum" effect" doesn't make this easier? Maybe "Turing machine" is too abstract or simplistic as a concept? Both for real computers and brains? I can see that a computer is on some level just a lot of sand (silica and metal) but put together in a really complex way, it "suddenly" can add and compare numbers … if we observe the complexity levels from sand to computer and try to see the analogy when comparing cells / neurons to a structure of billions of them somehow interconnected on both a physical and chemical level, evolved during millions of years, I have no problem to accept that brains are still too complex to explain for us. |
|
| |
| ▲ | tim333 11 days ago | parent | prev [-] | | They know quite a lot about how neurons work to the extent they can replace bits of brains with artificial retinas or cochleas and interface with devices like neuralink. It's unlikely there is a quantum field effect of the type you mention powering things, although of course atoms and the like obey quantum mechanics in the normal way. |
|
| |
| ▲ | jodrellblank 11 days ago | parent | prev | next [-] | | “Reflections on trusting trust” is the paper that posits a compiler which is edited once so that when it compiles a program, it adds a security vulnerability to it, and when it compiles it’s own source code, it adds this edit into itself. Then it is used to compile its own source code once. Then this edit is removed. Now any study of the program or compiler source code will not show any vulnerability, but compiling the program will make a vulnerable program, and recompiling the compiler from its clean source code will not fix the situation. This carrying down of a pattern which is not written down anywhere, a flaming torch lighting a torch lighting a torch, is analogous to four billion years of life on Earth. We talk like DNA is an everything-code that defines a human and a human brain, but it’s the implicit behaviour of cells (‘compiler’) and the mechanisms inside them which interpret DNA. The unbroken chain of life getting more and more complex and never being restarted from scratch, with the behaviours not written down anywhere for us to study. How does DNA arrange for x, y, z to happen? Maybe it doesn’t at all. Accidentally stumbling on a mechanism that is simple enough to be recreated with every human birth might be possible, accidentally stumbling on a mechanism that took evolution billions of years to find and which it has hung onto by copying it and has never recreated it from scratch, could be much less likely, in a much bigger search space. | | |
| ▲ | gpderetta 11 days ago | parent [-] | | > Accidentally stumbling on a mechanism that is simple enough to be recreated with every human birth might be possible, accidentally stumbling on a mechanism that took evolution billions of years to find and which it has hung onto by copying it and has never recreated it from scratch, could be much less likely, in a much bigger search space. Maybe, but you could make the same argument about anything artificial. | | |
| ▲ | jodrellblank 11 days ago | parent [-] | | I don't know what point you're making; I'm making the point that it might be harder to discover what makes humans human, than is often suggested. You can't make the same claim about "anything artificial", we know how to dry muddy clay into clay bricks and stack them into a brick wall, and that can be taught from scratch to new people in hours. You can make a similar argument with a company like ASML where their secret sauce is the organisational ability to fine-tune 100,000 components into a precision Silicon-wafer etching machine. You're far more likely to accidentally stumble upon "how to recreate a mud hut" than "how to recreate ASML". Okay, and...? |
|
| |
| ▲ | f_klem 10 days ago | parent | prev [-] | | Dreyfus builds his arguments based on Heidegger's work. You expressed it correctly: for Heidegger, in order to experience the world, we need a body that is always present in the world. It is like (but not the same) a continuum from the world, through our body, to our experience, and back.
As far as I as understand it, Heidegger does not states explicitly that there is a mind or consciousness, and I think it is irrelevant in the discussion. > but that's also just a theory. I don't believe that experience is literally impossible to engineer, as consciousness has emerged from non-conscious being through evolution, so clearly there must be some kind of mechanism for it -- and if there is, then I believe it can be replicated, we just don't really understand it well enough yet to do so Something can be understood but no replicated. The structure and inner workings of the Sun or a complete galaxy can be understood, but not replicated. In order to replicate things, knowledge is not enough, one also needs the material capacity to do it. So we might understand at some point how experience emerges from complex systems, but nonetheless we also may be unable to replicate it. I do believe that experience comes from this 'emergence'... but specifically from the combination of factors that make complex mammals and human beings the way they are. But it is a belief: I simply cannot base a whole field of study on it on the base that is a proved fact. |
|
|
| ▲ | jordigh 11 days ago | parent | prev | next [-] |
| You actually read Being and Time? What the heck do you get out of it? Heidegger just seems to play word games with German words without actually saying anything. "Time is the ripening of temporality." tf do phrases like that mean?? |
| |
| ▲ | lmf4lol 11 days ago | parent | next [-] | | Not OP, but I have read Being and Time on and off over the years, and every time, I am blown away how precise Heidegger describes things and the being of things.
Granted, I am German, so I can directly read the source without needing a translator. That might change things. Regardless, I think Heidegger gave one of the fullest metaphysic-free accounts of the human experience and what Being is. And he starts from scratch. You essentially can read him without having to first study the whole western continental philosophy and he will construct the whole
system by himself. Tremendous work | | |
| ▲ | jordigh 11 days ago | parent [-] | | I keep hearing "it's better in German" which really reinforces my point that he's just playing word games with etymologies without saying anything. | | |
| ▲ | lmf4lol 11 days ago | parent [-] | | i didnt say that at all :-) Heidegger uses very specific German words to build a very specific vocabulary. This vocabulary then allows him to express very complicated sentiments very quickly and he can use this to express more and more complex
structures. Obviously this requires the reader to first learn the vocabulary and - granted - that is hard and challenging. I have a notebook here , which I consult and modify evertine i dive into Being and Time. But as I said. It keeps on giving. I often try to convey arguments and descriptions to others without falling back into Heideggers jargon, and its sometimes very hard and requires a lot
of bloating . So you can argue that it was even necessary for him to invent the vocabulary, because otherwise the book would have been 10x in size Your comment strikes me as a bit ignorant i must say. You accuse the work
of being non-sensical word play but you’ve obviously not
invested any time in learning the vocabulary. Because otherwise you wouldnt have made that comment. Id suggest to give it a chance. Its a wonderful piece of work and mind blowing in its own way. That a single mind can think something like that up. I’d argue its on the level of Hegel in terms of system building. | | |
| ▲ | jordigh 11 days ago | parent [-] | | Yes, I'm ignorant, because Heidegger isn't saying anything. He didn't teach me anything. Thus I remain ignorant, unknowing. You can't even explain what he said, you just said, "go learn his words". That's not knowledge, that's not insight. That's just "the wordplay is great". But it's not content. It's merely form, it's sophistry, it's useless and meaningless. I asked a very specific question originally. What does "time is the ripening of temporality" mean? That's one way to translate one of the things he says, using different words for time and ripening in German. He's playing word games because those words sound similar in German and people like you confuse it for profundity. | | |
| ▲ | lmf4lol 11 days ago | parent [-] | | Phew, you are quite something. I hope one day your mind will open up. All that is left to say for me is that you are really missing out on some truly great work that is a masterpiece of human thought. Your loss | | |
|
|
|
| |
| ▲ | f_klem 11 days ago | parent | prev [-] | | Yes. I read the spanish translation (with occasional reading of the german original). A lot is lost in translation (I do not regard english translations too high). So yes, it is a difficult reading, on a difficult topic.
He is not playing with words, but defining concepts in a very specific way.
What I would recommend is reading Heidegger through other authors (Being-in-the-world, also by Dreyfus, would be a starting point). |
|
|
| ▲ | CuriouslyC 11 days ago | parent | prev | next [-] |
| The problem with the stuff you cite is that it is coming from consciousness via an anthropomorphic perspective. The priors baked in are poisoning the logic. The question we have to answer is "Why do we think we're magical matter uniquely blessed with consciousness?" If you go far enough down the rabbit hole on that question, the answer you will come to is either "we're not" or "because god" (with a lot of pseudo-scientific bullshit wrapped around the "because god" to make it palatable for the nonreligious). Panpsychism (or a deeper form, such as idealism) is actually the solution favored strongly by Occam's Razor over the variants of "because god" (such as magical emergence). Given panpsychism, AI is already conscious, like everything else, though no claims are made about the correlation between the internal experience of that consciousness and the tokens that are being printed on the screen. |
| |
| ▲ | f_klem 11 days ago | parent [-] | | There is no way of dealing with these topics via a non-anthropomorphic perspective. That has been already (in my view) proved by Heidegger and then Derrida. Last time I read about panpsychism, it was deeply flawed. But I can't remember the sources (sorry). | | |
| ▲ | CuriouslyC 11 days ago | parent [-] | | If it's so deeply flawed, should be easy to state why in an ELI5 manner. For example: The emergence hypothesis requires physical processes to effectively create new dimensions of information (700nm light -> "red" qualia) which we're somehow incapable of directly measuring with all the tools of modern physics. | | |
| ▲ | f_klem 10 days ago | parent [-] | | I am not an expert in panpsychism, but for what I know: 1) the idea that everything has a degree of consciousness proportional to its complexity, introduces the problem of compound consciousness. How do they compose, how is each consciousness contributing to the overall, upper-level one? how is experience explained at the different complexity levels? 2) it is impossible to test whether something is conscious or not 3) the theory is more a philosophical framework for dealing with the mind/body problem, but it actually moves the problem forward on the assumption that 'because it is something physical, it has consciousness. Then complex things have complex consciousness' | | |
| ▲ | CuriouslyC 10 days ago | parent [-] | | 1 is an open question, but it's an easier question than the ones related to emergence, since it reduces from "how do we detect emergence of consciousness" (to which people have settled for behavioral tests, hence the whole AI consciousness kerfuffle) to "how do we correlate physical coupling with assumed (human) consciousness." This framing admits study via sleep/brainwave/dreaming/sedation experiments, though there are still things like large scale quantum coupling we can't easily measure. 2 is a problem for any theory of consciousness. Regarding 3, I'm not certain "complex things have complex consciousness" is assumed, at least not for all definitions of complex. A crystal might be very complex from a structural standpoint but simple from a temporal evolution standpoint. I don't think there's a unified panpsychist perspective there. From my perspective, it's more a parsimonious rejection of materialist emergence hypotheses than a definitive statement of knowledge. | | |
| ▲ | f_klem 10 days ago | parent [-] | | 1) I think that is precisely the flaw: it is reductionist. 2) how do you actually measure if a rock has consciousness? if you redefine consciousness as simply some kind of physical manifestation, like radiance or quantum fluctuation (measured as compound at the macro-level), then you will have to redefine what we understand as 'human consciousness' is, otherwise it will be a characteristic so generic that won't be useful at all. Then this new characteristic will suffer from the same original interpretation problem... So at least the panspychism approach is just evading the main problem. 3) I would argue that the only way of thinking about a complex consciousness emerging from a diverse and vast set of things, is considering complexity. If it is not complexity (like, structurally complex things can have simple consciousness and the other way around) then there must be something, material or not, that provides for the emergence of such complex consciousness. This is rather tricky: you will need to postulate some kind of non-physical (or not discovered yet) characteristic that generates consciousness, or you will have to come up with some causal relation between non-related-to-complexity but still-related-to-physicality and complex consciousness, which from our current physics framework might not be possible. |
|
|
|
|
|
|
| ▲ | ordu 11 days ago | parent | prev [-] |
| > These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. It would be a valid argument if you could explain us what subjective experience and agency are. No one can explain this, so the arguments sounds like "AI doesn't have something I don't really know what, but they have to miss something, sure". > But the truth is that we don't know. Yes! This is the point. If we don't know how our minds work, how can we be sure that a machine doesn't work like our minds? > I highly recommend people in the AI research space should read philosophy and modern linguistics. Linguists are linguists, they don't know about consciousness, they specialists in language. The main teaching of philosophy is to open your mind wide, and then still wider. So yes, we all should read more philosophy, but we should remember while reading, that philosophy is what we do, when we can't turn a question to a scientific one. I'd say that philosophy is more about finding the right question, than the right answer. Until there are no science branches like Subjectology or Consciousiology, and all you can find are TL;DRs written by philosophers and scientists from unrelated branches of science (like linguists or neurologists) you can be sure that the right question is not found yet. Therefore it is better to keep yourself uncertain. BTW it is the main lesson of philosophy: to open your mind wide and then open it wider still. I believe, that all this philosophy is... well... philosophy. Meta-physics. It doesn't matter. What does matter is how we should deal with machines? Do we have to treat them as human beings? Should we accept that they have "human rights"? Can a machine be held accountable for its mistakes? Can we talk about "intentional" and "unintentional" mistakes of a machine? Answers to these questions are important, they are imperative answers, they govern how we live our lives. But when people try to answer them, they somehow jump to talking about consciousness, and "are they really like us or not", and... etc. I personally keep myself uncertain. I'm pretty sure that current models do not deserve human rights. I cannot say about future models. You see, there were times in history when smart and well-intentioned people were certain that some people deserve less moral considerations than others. We are smart and well-intentioned and we are certain that AI deserve no moral considerations. How it will play out in a future? Will my descendants think bad of me because I used slave labor of a local model on a GPU? I don't know, we don't know. Right now, I'm exploiting LLMs and I believe it is ok, but I'm not going to fall into this trap and to stick to my belief because of some philosophical or lingustic or neurological argument. I'm choosing epistemological humility, I clearly state that I don't know the right answer and I'm keeping an eye to it so I wouldn't miss it. |
| |
| ▲ | f_klem 11 days ago | parent [-] | | > It would be a valid argument if you could explain us what subjective experience and agency are. No one can explain this, so the arguments sounds like "AI doesn't have something I don't really know what, but they have to miss something, sure". Subjective experience is what you and only you, different from other beings, experience in and about the world. > Yes! This is the point. If we don't know how our minds work, how can we be sure that a machine doesn't work like our minds? You can't be sure that machine do not work like our minds or brains. But you cannot say the opposite either, so saying 'machines could think' base on a false assumption (because you cannot say it is true. It doesn't matter if you could be true). > Linguists are linguists, they don't know about consciousness, they specialists in language. That's a very narrow understanding of what language is, what linguists do/research, and the contributions made in the field.
Linguists are (since already 2 o 3 decades) focusing more and more on psychological/cognitive matters. The intertwined topics of language, mind, body and though has a long way in the western philosophical tradition. > I believe, that all this philosophy is... well... philosophy. Meta-physics. It doesn't matter. What does matter is how we should deal with machines? Do we have to treat them as human beings? Should we accept that they have "human rights"? Can a machine be held accountable for its mistakes? Can we talk about "intentional" and "unintentional" mistakes of a machine? Exactly because of this. And is this what I am talking about... the topics you mention here are already settled in the philosophy space, but the AI research space keeps going 'round them... | | |
| ▲ | ordu 10 days ago | parent [-] | | > Subjective experience is what you and only you, different from other beings, experience in and about the world. So subjective experience is subjective experience? Do rocks have subjective experience? The main goal of any definition is to say clearly what is doesn't cover. It is not enough to say what is subjective experience, you should say it in a way that excludes everything else. Like what is objective experience? How it differs from subjective one? Do LLM experience objectively or not experience at all? How can you say? > But you cannot say the opposite either, so saying 'machines could think' base on a false assumption I can say that they could think. The thought process implies a measurable product. Yes, there are similar situation with it, it can be hard to say sometimes if we observe a product of a thought or something else. But science has some success with this, like claiming that bees can think and solve problems. > That's a very narrow understanding of what language is, what linguists do/research, and the contributions made in the field. Linguists are (since already 2 o 3 decades) focusing more and more on psychological/cognitive matters. I'm telling you, psychologists (who specialize on mind research) do not know what consciousness is and they do not have a definition of a subjective experience (well, if we treat your proposed definition as a valid definition, then we should say they have plenty). And my claim still stands: until you heard about new science "Subjectology" you can be sure that no one knows what subjective experience is. Including linguists. > the topics you mention here are already settled in the philosophy space You shouldn't believe that anything can be settled in the philosophy space. Philosophers can think they have settled things, but until science agreed and started empirical studies, it is just philosophers believing that they settled things. > Exactly because of this. And is this what I am talking about... Exactly because of this I'm vary of any "settled things". You see, people have all the reasons for motivated reasoning. People are historically very wary of any attempts to extend the group of beings covered by "human rights" or whatever it was actual at their time. People would fight to the death to not include another group of beings to the list of "sentient", "conscious" or what it is today is euphemism to "morally equal to a human". And I do not trust anyone to overcome this deep psychological bias. I do not trust myself, I do not trust philosophers, I do not trust linguists, I do not trust neurologists and psychologists. Well, psychologists are probably the most prepared to it, but I do not trust them still. Just think about it, lets suppose someone proved beyond any doubt that LLM has conscious experience. What happens then? Try to imagine that counterfactual world. It will be a very troubled world, won't it? Any sentient human feels it, they feel the level of push back they would receive if they claimed that LLM is conscious. So any sentient person unconsciously feel the urge to join the side claiming that LLMs are not conscious: it is safe to argue they are not, you can publish papers on it and you won't face a strong condemnation from angry scientists, politicians, large corporations hoping to make a shitton of money with LLMs, various religious groups, etc. Our world just have no place for a sentient LLMs. If they are sentient we have to either ban them, or to do sweeping changes to make room for LLMs with human rights. No philosopher/linguist/neuroscientist/psychologist is safe from this unconscious bias, and they do not speak math, they say things like your definition of subjective experience above. Things that can mean anything if you try hard enough. It means you can't just take logic and check their reasoning, you have to _feel_ that they are right, but your feelings a subject to your psychological biases too. You can trust your subjective experiences because they are subjective. If philosophy "settled things" then I even more wary of it. It has even more biases to keep status quo. Wanna me to draw you a picture how the philosopher opinion would evolve if it turns out that LLM are conscious? History shows us how these things unfold. It won't happen overnight, people working with LLMs will notice some patterns and they will ask new questions and slightly modified versions of "already settled questions". The latter will be forced into "already settled answers" (Thomas Kuhn, normal science phase). The former might be answered in the current paradigm (and become settled) or just thrown away as "meaningless" questions. It will be a long process: people noticing things, world class thinkers forcing them into the paradigm. With each step it will become more and more ridiculous, until everyone will see that paradigm doesn't hold. After that politics will take over and it will be defending "traditional paradigm"... well, I can't predict the next phase, but it doesn't matter. What does matter is the alternate Universe where LLMs are sentient would look to you just like ours. Matters settled already, all questions have answers. Well, some don't, but those are nonsensical questions. etc. etc. | | |
| ▲ | f_klem 10 days ago | parent [-] | | I think the conversation derailed a bit. I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing. My original comment was that it seems (and it is actually documented in the books I referenced) that the AI research space builds its claims on assumptions, not on facts, and that those assumptions are flawed. So a nice discussion, to begin with, would consider: 1) why I make the claim that the AI research space builds its claims on assumptions instead of facts, why we could say that there are actually no assumptions but facts, or why the assumptions are correct. 2) instead of strictly and directly dismissing readings on philosophy, I would expect intelligent and curious people to embrace new references. Particularly if those references are highly regarded and a solid contribution during the last 120 years Regarding point 1), I can barely count a single comment in this thread that tries to engage in the idea of the assumptions (except for some comments that agree with the premise). Then regarding point 2), I can barely count research papers, books or contributions in the space of AI research that references (either to built upon or dismiss) philosophy that is pertinent to AI, pertinent to philosophy of technique or cognitive linguistics. This is strange. It looks like if the space revived during the 2000s with the invention of neural nets (RL, GAN, etc), and then became isolated from contributions about human intelligence, even though it continually tries to explain intelligence in its own terms. The reference to What Computers Can't Still do is precisely relevant because it narrates exactly this same discussion (false assumptions, claims built upon assumptions instead of facts, dismissal of evidence from psychology, dismissal of frameworks from philosophy, fallacies about progress), but it was written in 1972. Still, you read the book today, and it is totally relevant. Now, regarding your comments: > Do LLM experience objectively or not experience at all? How can you say? The world cannot be experienced 'objectively'. If they experience the world most probably you won't notice. Given that the only way of interacting with an LLM is through a process initiated solely by a human actor, it would be difficult to assess whether an LLM experiences anything at all. > I can say that they could think. The thought process implies a measurable product. Yes, there are similar situation with it, it can be hard to say sometimes if we observe a product of a thought or something else. But science has some success with this, like claiming that bees can think and solve problems. The moment you say 'they could think', that implies an assumption about the actual possibility of thinking as a process that can be modeled and executed by a machine. There is, as far as I know, no current evidence that human beings process information the same way a computer does it, nor that though processes necessarily imply a measurable outcome. > I'm telling you, psychologists (who specialize on mind research) do not know what consciousness is and they do not have a definition of a subjective experience (well, if we treat your proposed definition as a valid definition, then we should say they have plenty). And my claim still stands: until you heard about new science "Subjectology" you can be sure that no one knows what subjective experience is. Including linguists. Psychology is a very broad field with lights and shadows through its short history. Here, and in my original comment, I am not talking about linguists in general, but specifically about cognitive linguistics. The contributions made be the field are significant and mostly lacking in AI research (for example, the idea of embodiment, the rebuttal of generative grammars, prototype theory, frame semantics, among others). What you mention as 'subjectology', would be just psychology. Foucault explains more or less clearly why this cannot be a science, and that's just fine (in The Order of Things). > You shouldn't believe that anything can be settled in the philosophy space. Philosophers can think they have settled things, but until science agreed and started empirical studies, it is just philosophers believing that they settled things. Well... certainly nothing can be 'settled' (not even in science, btw), but my point is: there is already enough convincing arguments in the field of philosophy so as to say that current LLM systems do not posses agency or experience, and that they do not behave like us. Again, read the sources, what are you people afraid of? Just read the sources, and then engage in the conversation. | | |
| ▲ | ordu 10 days ago | parent [-] | | > I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing. I can probably to clarify it. 1. I don't want to end in a situation when AI deserves human rights but I deny it. There are moral reasons for that, and they are important. 2. I employ a systemic view. Not just arguments for consciousness or against it. I look at the people generating these arguments, how their minds work? I look at social institutions while they try to find some consensus. I'm very interested in their inner processes of generating truths. IN particular I'm interested in their failings and how they can generate untruths instead. 3. To understand a system I rely on historic data. How people and social institutions (including science) dealt with similar questions before. The issue is, that the problem of LLM agency has potentially extremely wide implications, I expect people to be afraid of them, I expect social institutions to be afraid of them, so I cannot trust science in this regard like I trust it when it talks about physics or biochemistry. > 2) instead of strictly and directly dismissing readings on philosophy I think I have a good idea what philosophy thinks now, and it doesn't seem convincing. It looks like a normal philosophy, not like an established science, so you should take it with a grain of salt. > The world cannot be experienced 'objectively'. So why we call it subjective experience then? Probably it is irrelevant, and the reasons are purely historic... or maybe not. How about the idea of computers experiencing things, just not "subjectively" but rather "digitally"? Or choose any other adjective you like. You are arguing against assumptions, but why you just accept the idea of experience with the assumption that human way to experience things is the only possible way? > I can barely count research papers, books or contributions in the space of AI research that references (either to built upon or dismiss) philosophy that is pertinent to AI, pertinent to philosophy of technique or cognitive linguistics. This is strange. I believe it is an expected outcome. AI is evolving fast, there are plenty of things to research without establishing connections with other branches of science. No sane AI researcher would stop researching AI to get PhD in linguistics to build a bridge between AI research and linguistics. Probably in an ideal world this shouldn't happen, maybe it is short-sighted behavior of a system, but it is just how things work in our real world. BTW it is a good example of what happens with all the philosophy when shit hits the fan. When possibility of empirical studies arrives, no one bothers themselves with philosophy of things. > I am not talking about linguists in general, but specifically about cognitive linguistics. I studied psychology, I've read some linguists (cognitive ones, because they are in an adjacent field), and you see, I don't have trust in either. They do their research, they find some interesting facts and devise interesting theories, but it is all looks more like a chemistry in the first half of a XIX century, than a chemistry after periodic table was created. They can't find their building blocks to create a sound theory. > The moment you say 'they could think', that implies an assumption about the actual possibility of thinking as a process that can be modeled and executed by a machine. No, I'm not implying "modeling a thinking process". We don't know what thinking process is. What we observe in our minds is not thinking by itself, it is some kind of a mirror process in our consciousness. The real thinking is hidden from us, but it creates echoes in our consciousness we can observe. If we don't know how thinking works, we can't model it. BTW the reverse is also true: if we can't model thinking, we don't know how it works. I'm defining thinking more in terms of a problem solving ability. Like psychologists do. Science still doesn't have a good enough definition for thinking, but it has some definitions that a) operational; b) good enough for some limited tasks. "Operational" means that they are defined in terms how to measure what you define, not in terms of modeling some process. > there is already enough convincing arguments in the field of philosophy so as to say that current LLM systems do not posses agency or experience, and that they do not behave like us. Well, I don't argue that current LLM systems do not possess agency or experience, I argue that we should not trust philosophy to be the first who claims that LLM systems got agency, if they really got it. There is a possibility that they will fight to the death against it even if it is true. You see, until philosophy methods successfully proved that something is conscious despite it was deemed unconscious before, we can't really know that their methods really work. Maybe they work, or maybe they just mirror our biases and heuristics. > Again, read the sources, what are you people afraid of? Just read the sources, and then engage in the conversation. I hadn't read books you mentioned, but in your words I see nothing that can hint that those books have something I don't considered already. So maybe I'll read them in a future, but I wouldn't postpone my engagement in the conversation till I read them. | | |
| ▲ | f_klem 10 days ago | parent [-] | | > So why we call it subjective experience then? Probably it is irrelevant, and the reasons are purely historic... or maybe not. How about the idea of computers experiencing things, just not "subjectively" but rather "digitally"? Or choose any other adjective you like. You are arguing against assumptions, but why you just accept the idea of experience with the assumption that human way to experience things is the only possible way? We call it subjective because is 'we, ourselves' and not the objects we perceive there in the world, where the 'experience' is manifested, as perception. We do not always codify dichotomies in language. > No sane AI researcher would stop researching AI to get PhD in linguistics to build a bridge between AI research and linguistics. Probably in an ideal world this shouldn't happen, maybe it is short-sighted behavior of a system, but it is just how things work in our real world. In the real world, anyone doing a serious PhD thesis will read whatever is necessary to build a proper, sound theory or body of work. This dismissal just makes me think that you don't know how a PhD thesis is done. > BTW it is a good example of what happens with all the philosophy when shit hits the fan. When possibility of empirical studies arrives, no one bothers themselves with philosophy of things. From what I see, you never took philosophy seriously. I don´t know how you can then seriously engage in a conversation about philosophy. > I studied psychology, I've read some linguists (cognitive ones, because they are in an adjacent field), and you see, I don't have trust in either. They do their research, they find some interesting facts and devise interesting theories, but it is all looks more like a chemistry in the first half of a XIX century, than a chemistry after periodic table was created. They can't find their building blocks to create a sound theory. This is true, cognitive linguistics do not represent a unified theory. Not yet, at least, and maybe it will not come to that. The same happens in psychology, but nobody is dismissing psychology all at once just because of that. > No, I'm not implying "modeling a thinking process". We don't know what thinking process is. What we observe in our minds is not thinking by itself, it is some kind of a mirror process in our consciousness. The real thinking is hidden from us, but it creates echoes in our consciousness we can observe. If we don't know how thinking works, we can't model it. BTW the reverse is also true: if we can't model thinking, we don't know how it works. This is exactly what the assumptions I challenge are about. The AI space already declared that they 'know' how such processes work. Or at least, they pretend they do. > I'm defining thinking more in terms of a problem solving ability. Like psychologists do. Science still doesn't have a good enough definition for thinking, but it has some definitions that a) operational; b) good enough for some limited tasks. "Operational" means that they are defined in terms how to measure what you define, not in terms of modeling some process. You would agree that 'problem solving' is just a small portion of what 'thinking' constitutes. > You see, until philosophy methods successfully proved that something is conscious despite it was deemed unconscious before, we can't really know that their methods really work. Maybe they work, or maybe they just mirror our biases and heuristics. You talk about philosophy as if argumentative biases were completely strange to philosophers. Not the case, and a great deal of XX century philosophy is exactly about that. But if you read the sources on AI research through its own history, you will see how the AI research space is full of such biases and assumptions. Again, my original argument is about that. > I hadn't read books you mentioned, but in your words I see nothing that can hint that those books have something I don't considered already. So maybe I'll read them in a future, but I wouldn't postpone my engagement in the conversation till I read them. You don't have to trust me. Just pick them up and think for yourself, do some research. Regarding postponing the conversation, I really appreciate that, but it is really difficult to argue about books you haven't read, especially if they are complex ones. Unfortunately I don't have that much time to explain the books in detail. | | |
| ▲ | ordu 10 days ago | parent [-] | | Let me start with:
> if you read the sources on AI research through its own history, you will see how the AI research space is full of such biases and assumptions. You are repeating it in different forms all the time, so it seems really important to you. I should state, that I don't believe that AI research has some special insights into human nature or into what agency is. I'm sure that they have some biases and assumptions, and maybe it is full of them. You don't need to prove it to me, but if you want to, you could list some of them. It would be interesting to read. OTOH I really like what AI research does right now. I believe it is beneficial for science and philosophy. "Throw away all that garbage from XX century and build something new." It will be something new, because they are creating things, not just daydreaming philosophically. It may be that the result of their research would be laughable, but it doesn't matter in the long term. It would be another way to look at things, which is the very important for pushing human knowledge forward. I expect AI research would create a heap of knowledge that would be mined for decades then to be incorporated into existing knowledge frameworks. The existing frameworks would break and people would create new ones. And it is an unhappy path, assuming that AI research will not stumble onto a good enough framework to consume all (or some of) others. Ah... There is one more assumption in there. The assumption that AI wouldn't change the world enough for my intuition of how things go becomes wrong, because it was trained on data from the old world where there was no AI. > We call it subjective because is 'we, ourselves' and not the objects we perceive there in the world, where the 'experience' is manifested, as perception. We do not always codify dichotomies in language. So you assume that this naming normalized due to stupid historic reasons, and reject the possibility that it reflects something deeper? Well, I don't. I'm consciously keep myself in an uncertain state, in other words I keep my mind wide open. Psychology teaches us to keep an eye on the exact phrasing, it reflects the real thinking process. It may be hard or even impossible to decipher these cues, but we should try at least. > In the real world, anyone doing a serious PhD thesis will read whatever is necessary to build a proper, sound theory or body of work. This dismissal just makes me think that you don't know how a PhD thesis is done. You are talking about about an established science, AI research regarding LLMs is not an established science. Things happens too fast for science to be established. If they slow down, then would be the great time to learn linguistics and to build bridges. > From what I see, you never took philosophy seriously. I don´t know how you can then seriously engage in a conversation about philosophy. I took it seriously enough to go from standard philosophic meta level to meta-meta level. I'm seeing not just what philosophy says it is doing, I see what it is really doing de facto. It is like POSIWID principle: the system purpose is what it does. > The AI space already declared that they 'know' how such processes work. Or at least, they pretend they do. I'd ask "did they really declared that", but you phrasing suggests that they didn't. I do not exactly on the topic, I do not read all they say, so I may be wrong, but I suppose what they really say, is more like "our machines can think". It is not the same "we know how thinking processes work" in general. They claim that they created some thinking processes. Am I right? > You would agree that 'problem solving' is just a small portion of what 'thinking' constitutes. I'm not sure really. You see, I don't know how thinking works. Especially I don't know how human thinking works. It may be that it involves more than just problem solving, or may be not. I can agree that when I reflect my own thinking it looks like something more than just problem solving. Or maybe something less: I have a strong suspicion that human thinking is tuned evolutionary to solve a specific kind of problems. Cultural training repurpose it for some other kinds, but it is like to drive in screws with a hammer, or to hammer nails with a screwdriver. And yes, it is the reasons I do not trust philosophers and scientists to judge if AI has agency already. The space of possible answers includes very scary ones, and in such cases human thinking strongly prefers safe options and creates very strong arguments in favor of them. Wow. I really glad I spent time arguing with you. I don't know if it was beneficial for you in some way, but it was for me, so thank you for your time. I've just found that I doubt that human thinking is up to the problem. Yeah... Probably the problem will be solved in a chaotic way, when society would just picks some "random" opinion after the dust settles. Yes! It is very likely outcome, we'll end with "they're made out of weights", so the text is prophetic. > You don't have to trust me. Just pick them up and think for yourself I'd like to, but you failed to spark a real interest in those books in me. I still do not see what fundamental shifts in my beliefs they may bring. They look like reiteration of thoughts and ideas I know already. Or maybe I know these ideas from reiterations of these books, which are the original sources for the ideas. I took notice of them, and I'll try to read them. But not right now. | | |
| ▲ | f_klem 10 days ago | parent [-] | | I'm not trying to convince anyone. I am just baffled that a large part of the tech/software and AI research community do not question their own assumptions, when those assumptions are being actively questioned in other fields (namely, philosophy and linguistics). Reaching AGI or human-level intelligence might be possible, but not on the basis of dismissing what other fields already said something about. That is arrogant, and does not help. Even more, this has already happened in the 60s/70s. And I say 'might be possible' precisely because I pay attention to what other fields have to say. | | |
| ▲ | ordu 8 days ago | parent [-] | | > I am just baffled that a large part of the tech/software and AI research community do not question their own assumptions Why are you so concerned about it? If philosophers and linguists are right, then what the sequence of evens should we expect? AI developments will slow down and stop, AI bubble will burst, and AI researchers will be humiliated and forced to accept their failings. > That is arrogant, and does not help. Even more, this has already happened in the 60s/70s. Exactly like that time. Or maybe we should say "those times". Doesn't matter really. |
|
|
|
|
|
|
|
|
