| |
| ▲ | hliyan 6 days ago | parent | next [-] | | In early 2023, I remember someone breathlessly explaining that there are signs that LLMs that are seemingly good at chess/checkers moves may have a rudimentary model of the board within them, somehow magically encoded into the model weights through the training. I was stupid enough to briefly entertain the possibility until I actually bothered to develop a high level understanding of the transformer architecture. It's surprising how much mysticism this field seems to attract. Perhaps it being a non-deterministic, linguistically invoked black box, triggers the same internal impulses that draw some people to magic and spellcasting. | | |
| ▲ | pegasus 6 days ago | parent | next [-] | | Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function, or that there can be no form of world model that they are developing. Recently there has been more evidence for that particular idea [1]. The feats of apparent intelligence LLMs sometimes display have taken even their creators by surprise. Sure, there's a lot of hype too, that's part and parcel of any new technology today, but we are far from understanding what makes them perform so well. In that sense, yeah you could say they are a bit "magical". [1] https://the-decoder.com/new-othello-experiment-supports-the-... | | |
| ▲ | ath3nd 6 days ago | parent [-] | | > Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function Mumbo jumbo magical thinking. They perform so well because they are trained on probabilistic token matching. Where they perform terribly, e.g math, reasoning, they are delegating to other approaches, and that's how you get the illusion that there is actually something there. But it's not. Faking intelligence is not intelligence. It's just text generation. > In that sense, yeah you could say they are a bit "magical" Nobody but the most unhinged hype pushers are calling it "magical". The LLM can never ever be AGI. Guessing the next word is not intelligence. > there can be no form of world model that they are developing Kind of impossible to form a world model if your foundation is probabilistic token guessing which is what LLMs are. LLMs are a dead end in achieving "intelligence", something novel as an approach needs to be discovered (or not) to go into the intelligence direction. But hey, at least we can generate text fast now! | | |
| ▲ | whalee 6 days ago | parent [-] | | > LLMs are a dead end in achieving "intelligence" There is no evidence to indicate this is the case. To the contrary, all evidence we have points to these models, over time, being able to perform a wider range of tasks at a higher rate of success. Whether it's GPQA, ARC-AGI or tool usage. > they are delegating to other approaches
> Faking intelligence is not intelligence. It's just text generation. It seems like you know something about what intelligence actually is that you're not sharing. If it walks, talks and quacks like a duck, I have to assume it's a duck[1]. Though, maybe it quacks a bit weird. [1] https://en.wikipedia.org/wiki/Solipsism | | |
| ▲ | ath3nd 5 days ago | parent [-] | | > There is no evidence to indicate this is the case Burden of proof is on those trying to convince us to buy into the idea of LLMs as being "intelligence". There is no evidence of the Flying Spaghetti monster or Zeus or God not existing either, but we don't take seriously the people who claim they do exist (and there isn't proof because these concepts are made up). Why should we take seriously the tolks claiming LLMs are intelligence without proof (there can't be proof, of course, because LLMs are not intelligence)? |
|
|
| |
| ▲ | ericmcer 6 days ago | parent | prev | next [-] | | Is there something we are all missing? Using Claude feels like magic sometimes, but can't everyone see the limitation now that we are 4 years and 100s of billions down the road? Are they still really hoping that they are gonna tweak a model and feed it an even bigger dataset and it will be AGI? | |
| ▲ | momojo 6 days ago | parent | prev [-] | | I'm not a fan of mysticism. I'm also with you that these are simply statistical machines. But I don't understand what happened when understood transformers at a high-level. If you're saying the magic disappeared after looking at a single transformer, did the magic of human intelligence disappear after you understood human neurons at a high level? |
| |
| ▲ | stuaxo 6 days ago | parent | prev | next [-] | | Its insane really, anyone who has worked with LLMs for a bit and has an idea of how they work shouldn't think its going to lead to "AGI". Hopefully some big players, like FB bankrupt themselves. | | |
| ▲ | IanCal 6 days ago | parent | next [-] | | Tbh I find this view odd, and I wonder what people view as agi now. It used to be that we had extremely narrow pieces of AI and I remember being on a research project about architectures and just very basic “what’s going on?” was advanced. Understanding that someone asked a question, that would be solved by getting a book and being able to then go and navigate to the place the book was likely to be was fancy. Most systems could solve literally one type of problem. They weren’t just bad at other things they were fundamentally incapable of anything but an extremely narrow use case. I can throw wide ranging problems at things like gpt5 and get what seem like dramatically better answers than if I asked a random person. The amount of common sense is so far beyond what we had it’s hard to express. It used to be always pointed out that the things we had were below basic insect level. Now I have something that can research a charity, find grants and make coherent arguments for them, read matrix specs and debug error messages, and understand sarcasm. To me, it’s clear that agi is here. But then what I always pictured from it may be very different to you. What’s your image of it? | | |
| ▲ | whizzter 6 days ago | parent | next [-] | | It's more that "random" people are dumb as bricks (but we've in the name of equality and historic measurement errors decided to forgo that), add to it that AI's have a phenomenal (internet sized) memory makes them far more capable than many people. However, even "dumb" people can often make judgements structures in a way that AI's cannot, it's just that many have such a bad knowledge-base that they cannot build the structures coherently whereas AI's succeed thanks to their knowledge. I wouldn't be surprised if the top AI firms today spend an inordinate amount of time to build "manual" appendages into the LLM systems to cater to tasks such as debugging to uphold the facade that the system is really smart, while in reality it's mostly papering up a leaky model to avoid losing the enormous investments they need to stay alive with a hope that someone on their staff comes up a real solution to self-learning. https://magazine.sebastianraschka.com/p/understanding-reason... | |
| ▲ | adwn 6 days ago | parent | prev | next [-] | | I think the discrepancy between different views on the matter mainly stems from the fact that state-of-the-art LLMs are better (sometimes extremely better) at some tasks, and worse (sometimes extremely worse) at other tasks, compared to average humans. For example, they're better at retrieving information from huge amounts of unstructured data. But they're also terrible at learning: any "experience" which falls out of the context window is lost forever, and the model can't learn from its mistakes. To actually make it learn something requires very many examples and a lot of compute, whereas a human can permanently learn from a single example. | | |
| ▲ | andsoitis 6 days ago | parent [-] | | > human can permanently learn from a single example This, to me at least, seems like an important ingredient to satisfying a practical definition / implementation of AGI. Another might be curiosity, and I think perhaps also agency. |
| |
| ▲ | Yoric 6 days ago | parent | prev | next [-] | | I think it's clear that nobody agrees what AGI is. OpenAI describes it in terms of revenue. Other people/orgs in terms of, essentially, magic. If I had to pick a name, I'd probably describe ChatGPT & co as advanced proof of concepts for general purpose agents, rather than AGI. | | |
| ▲ | delecti 6 days ago | parent [-] | | > I think it's clear that nobody agrees what AGI is People selling AI products are incentivized to push misleading definitions of AGI. |
| |
| ▲ | boppo1 6 days ago | parent | prev | next [-] | | Human-level intelligence. Being able to know what it doesn't know. Having a practical grasp on the idea of truth. Doing math correctly, every time. I give it a high-res photo of a kitchen and ask it to calculate the volume of a pot in the image. | | |
| ▲ | tomaskafka 6 days ago | parent | next [-] | | You discover truth by doing stuff in real world and observing the results. Current LLM have enough intelligence, but all the inputs they have are the “he said she said” by us monkeys, including all omissions and biases. | |
| ▲ | snapcaster 6 days ago | parent | prev | next [-] | | But many humans can't do a lot of those things and we still consider them "generally intelligent" | |
| ▲ | 293984j29384 6 days ago | parent | prev [-] | | None of what you describe would I label within the realm of 'average' | | |
| ▲ | swiftcoder 6 days ago | parent [-] | | It's not about what the average human can do - it's about what humans as a category are capable of. There will always be outliers (in both directions), but you can, in general, teach a human a variety of tasks, such as performing arithmetic deterministically, that you cannot teach to, for example, a parrot. |
|
| |
| ▲ | audunw 5 days ago | parent | prev | next [-] | | I don’t have a very high expectation of AGI at all. Just an algorithm or system you can put onto a robot dog, and get a dog level general intelligence. You should be able to live with that robot dog for 10 years and it should be just as capable as a dog throughout that timespan. Hell, I’d even say we have AGI if you could emulate something like a hamster. LLMs are way more impressive in certain ways than such a hypothetical AGI. But that has been true of computers for a long time. Computers have been much better at Chess than humans for decades. Dogs can’t do that. But that doesn’t mean that a chess engine is an AGI. I would also say we have a special form of AGI if the AI can pass an extended Turing test. We’ve had chat bots that can fool a human for a minute for a long time. Doesn’t mean we had AGI. So time and knowledge was always a factor in a realistic Turing test. If an AGI can fool someone who knows how to properly probe an LLM, for a month or so, while solving a bunch of different real world tasks that require stable long term memory and planning, then I’d day we’re in AGI territory for language specifically. I think we have to distinguish between language AGI and multi-modal AGI. So this test wouldn’t prove what we could call “full” AGI. These are some of the missing components for full AGI:
- Being able to act as a stable agent with a stable personality over long timespans
- Capable of dealing with uncertainties. Having a understanding of what it doesn’t know
- One-shot learning, with long term retention, for a large number of things
- Fully integrated multi-modality across sound, vision, and other inputs/outputs we may throw at it. The last one is where we may be able to get at the root of the algorithm we’re missing. A blind person can learn to “see” by making clicks and using their ears to see. Animals can do similar “tricks”. I think this is where we truly see the full extent of the generality and adaptability of the biological brain. Imagine trying to make a robot that can exhibit this kind of adaptability. It doesn’t fit into the model we have for AI right now. | |
| ▲ | homarp 6 days ago | parent | prev | next [-] | | my picture of AGI is
1) autonomous improvement
2) ability to say 'i don't know/can't be done' | | |
| ▲ | dmboyd 6 days ago | parent [-] | | I wonder if 2) is a result of published bias for positive results in the training set. An “I don’t know” response is probably ranked unsatisfactory by human feedback and most published scientific literature are biased towards positive results and factual explanations. | | |
| ▲ | InitialLastName 6 days ago | parent [-] | | In my experience, the willingness to say "I don't know" instead of confabulate is also down-rated as a human attribute, so it's not surprising that even an AGI trained on the "best" of humanity would avoid it. |
|
| |
| ▲ | AlienRobot 6 days ago | parent | prev [-] | | Nobody is saying that LLM's don't work like magic. I know how neural networks work and they still feel like voodoo to me. What we are saying is that LLM's can't become AGI. I don't know what AGI will look like, but it won't look like an LLM. There is a difference between being able to melt iron and being able to melt tungsten. |
| |
| ▲ | thaawyy33432434 6 days ago | parent | prev | next [-] | | Recently I realized that US are very close to a centrally planned economy. Meta wasted 50B on metaverse, which like how much Texas spends on healthcare. Now the "AI" investments seems dubious. You could fund 1000+ projects with this kinds of money. This is not an effective capital allocation. | |
| ▲ | amelius 6 days ago | parent | prev | next [-] | | The only way we'll have AGI is if people get dumber. Using modern tech like LLMs makes people dumber. Ergo, we might see AGI sooner than expected. | |
| ▲ | menaerus 6 days ago | parent | prev | next [-] | | > ... and has an idea of how they work shouldn't think its going to lead to "AGI" Not sure what level of understanding are you referring to but having learned and researched about the pretty much all LLM internals I think this has led me exactly to the opposite line of thinking. To me it's unbelievable what we have today. | |
| ▲ | janalsncm 6 days ago | parent | prev | next [-] | | I think AI research is like anything else really. The smartest people are heads down working on their problems. The people going on podcasts are less connected to day to day work. It’s also pretty useless to talk about whether something is AGI without defining intelligence in the first place. | |
| ▲ | foobarian 6 days ago | parent | prev | next [-] | | I think something like we saw in the show "Devs" is much more likely, although what the developers did with it in the show was bonkers unrealistic. But some kind of big enough quantum device basically. | |
| ▲ | guardian5x 6 days ago | parent | prev | next [-] | | Just scaling them up might not leat to "AGI", but they can still lead to AGI as a bridge. | |
| ▲ | meowface 6 days ago | parent | prev | next [-] | | This is not and has not been the consensus opinion. If you're not an AI researcher you shouldn't write as if you've set your confidence parameter to 0.95. Of course it might be the case, but it's not a thing that should be expressed with such confidence. | |
| ▲ | blackhaz 6 days ago | parent | prev [-] | | Is it widely accepted that LLMs won't lead to AGI? I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly: 1) LLMs as simple "next token predictors" so they just mimicry thinking: But can it be argued that current models operate on layers of multiple depth and are able to actually understand by building concepts and making connections on abstract levels? Also, don't we all mimicry? 2) Grounding problem: Yes, models build their world models on text data, but we have models operating on non-textual data already, so this appears to be a technical obstacle rather than fundamental. 3) Lack of World Model. But can anyone really claim they have a coherent model of reality? There are flat-earthers, yet I still wouldn't deny them having AGI. People hallucinate and make mistakes all the time. I'd argue hallucinations is in fact the sign of an emerging intelligence. 4) Fixed learning data sets. Looks like this is now being actively solved with self-improving models? I just couldn't find a strong argument supporting this claim. What am I missing? | | |
| ▲ | globnomulous 6 days ago | parent | next [-] | | Why on earth would you copy and paste an LLM's output into a comment? What does that accomplish or provide that just a simply stated argument doesn't accomplish more succinctly? If you don't know something, simply don't comment on it -- or ask a question. | | |
| ▲ | blackhaz 6 days ago | parent [-] | | None of the above is AI. | | |
| ▲ | globnomulous 4 days ago | parent [-] | | > I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly: This line means, and literally says, that everything that follows is a summary or direct quotation from an LLM's output. There's a more charitable but unintuitive interpretation, in which "commenting on them briefly" is intended to mean "I will comment on them briefly:". But this isn't a natural interpretation. It's one I could be expected to reach only after seeing your statement that 'none of the above is AI.' But even this more charitable interpretation actually contradicts your claim that it's not AI. So now I'm even less sure I know what you meant to communicate. Either I'm missing something really obvious or the writing doesn't communicate what you intended. |
|
| |
| ▲ | welferkj 6 days ago | parent | prev [-] | | Fur future reference, pasting llm slop feels exactly as patronizing as back when people pasted links to google searches in response to questions they considered beneath their dignity to answer. Except in this case, no-one asked to begin with. |
|
| |
| ▲ | qcnguy 6 days ago | parent | prev | next [-] | | > I don't know what he was talking about There's a bunch of ways AI is improving itself, depending on how you want to interpret that. But it's been true since the start. 1. AI is used to train AI. RLHF uses this, curriculum learning is full of it, video model training pipelines are overflowing with it. AI gets used in pipelines to clean and upgrade training data a lot. 2. There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents. There isn't exponential growth. There might be if you throw enough compute at it, but this tactic is very compute hungry. At current prices it's cheaper to pay an AI expert to implement your agent than use this. | | |
| ▲ | Eggpants 6 days ago | parent | next [-] | | So have an AI with a 40% error rate judge an AI with an 40% error rate… AGI is a complete no go until a model can adjust its own weights on the fly, which requires some kind of negative feedback loop, which requires a means to determine a failure. Humans have pain receptors to provide negative feedback and we can imagine events that would be painful such as driving into a parked car would be painful without having to experience it. If current models could adjust its own weights to fix the famous “how many r’s in strawberry” then I would say we are on the right path. However, the current solution is to detect the question and forward it to a function to determine the right answer. Or attempt to add more training data the next time the model is generated ($$$). Aka cheat the test. | |
| ▲ | mitjam 6 days ago | parent | prev | next [-] | | I think LLM as a toolsmith like demonstrated in the Voyager paper (1) is another interesting approach to creating a system that can learn to do a task better over time. (1) https://arxiv.org/abs/2305.16291 | |
| ▲ | Yoric 6 days ago | parent | prev | next [-] | | > There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents. Interesting. Do you have links? | | | |
| ▲ | franktankbank 6 days ago | parent | prev [-] | | I'm skeptical that RLHF really works. Doesn't it just patch the obvious holes so it looks better on paper? If it can't reason then it will continue to get 2nd and 3rd order difficulty problems wrong. |
| |
| ▲ | abraxas 6 days ago | parent | prev | next [-] | | > it's pretty clear at this point that LLMs are never going to lead to general intelligence. It is far from clear. There may well be emergent hierarchies of more abstract thought at much higher numbers of weights. We just don't know how a transformer will behave if one is built with 100T connections - something that would finally approach the connectome level of a human brain. Perhaps nothing interesting but we just do not know this and the current limitation in building such a beast is likely not software but hardware. At these scales the use of silicon transistors to approximate analog curve switching models just doesn't make sense. True neuromorphic chips may be needed to approach the numbers of weights necessary for general intelligence to emerge. I don't think there is anything in production at the moment that could rival the efficiency of biological neurons. Most likely we do not need that level of efficiency. But it's almost certain that stringing together a bunch of H100s isn't a path to the scale we should be aiming for. | |
| ▲ | epolanski 6 days ago | parent | prev | next [-] | | I don't get it, I really don't. Even assuming a company gets to AGI first this doesn't mean another one will follow. Suppose that FooAI gets to it first:
- competitors may get there too in a different or more efficient way
- Some FooAI staff can leave and found their own company
- Some FooAI staff can join a competitor
- FooAI "secret sauce" can be figured out, or simply stolen, by a competitor At the end of the day, it really doesn't matter, the equation AI === commodity just does not change. There is no way to make money by going into this never ending frontier model war, price of training keeps getting higher and higher, but your competitors few months later can achieve your own results for a fraction of your $. | | |
| ▲ | cedws 6 days ago | parent [-] | | Some would say that the race to AGI is like the race to nuclear weapons and that the first to get there will hold all the cards (and be potentially able to stop others getting there.) It's a bit too sci-fi for me. | | |
| ▲ | Yossarrian22 6 days ago | parent [-] | | If AGI is reached it would be trivial for the competing superpowers to completely quarantine themselves network wise by cutting undersea cables long enough to develop competing AGI |
|
| |
| ▲ | CrossVR 6 days ago | parent | prev [-] | | I don't know if AGI will emerge from LLM, but I'm always reminded of the Chinese room thought experiment. With billions thrown at the idea it will certainly be the ultimate answer as to whether true understanding can emerge from a large enough dictionary. | | |
| ▲ | torginus 6 days ago | parent [-] | | Please stop refering to the Chinese Room - it's just magical/deist thinking in disguise. It postulates that humans have way of 'understanding' things that is impossible to replicate mechanically. The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there. (That doesn't mean LLMs are or will be AGI, its just this argument is tautological and meaningless) | | |
| ▲ | armada651 6 days ago | parent | next [-] | | That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thought experiment itself. I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. If the answer is no, that shouldn't lead to a deist conclusion. It can just as easily lead to the conclusion that a non-deterministic Turing machine is required. | | |
| ▲ | torginus 6 days ago | parent [-] | | I'd appreciate if you tried to explain why instead of resorting to ad hominem. > I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The original 'Chinese Room' experiment describes a book of static rules of Chinese - which is probably not the way to go, and AI does not work like that. It's probabilistic in its training and evaluation. What you are arguing is that constructing an artificial consciousness lies beyond our current computational ability(probably), and understanding of physics (possibly), but that does not rule out that we might solve these issues at some point, and there's no fundamental roadblock to artificial consciousness. I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. Which is an argument for the existence of the supernatural and deist thinking. | | |
| ▲ | CrossVR 6 days ago | parent | next [-] | | > I'd appreciate if you tried to explain why instead of resorting to ad hominem. It is not meant as an ad hominem. If someone thinks our current computers can't emulate human thinking and draws the conclusion that therefore humans have special powers given to them by a deity, then that probably means that person is quite religious. I'm not saying you personally believe that and therefore your arguments are invalid. > Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The idea that a sufficiently complex pseudo-random number generator can emulate real-world non-determinism enough to fully simulate the human brain is quite an assumption. It could be true, but it's not something I would accept as a matter of fact. > I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. In that same Wikipedia article Searle denies he's arguing for that. And even if he did secretly believe that, it doesn't really matter, because we can draw our own conclusions. Disregarding his arguments because you feel he holds a hidden agenda, isn't that itself an ad hominem? (Also, I apologize for using two accounts, I'm not attempting to sock puppet) | | |
| ▲ | torginus 6 days ago | parent [-] | | What are his arguments then? >Searle argues that, without "understanding" (or "intentionality"), we cannot describe what the machine is doing as "thinking" and, since it does not think, it does not have a "mind" in the normal sense of the word. This is the only sentence that seems to be pointing to what constitutes the specialness of humans, and the terms of 'understanding' and 'intentionality' are in air quotes so who knows? This sounds like the archetypical no true scotsman fallacy. In mathematical analysis, if we conclude that the difference between 2 numbers is smaller than any arbitrary number we can pick, those 2 numbers must be the same. In engineering, we can reduce the claim to 'any difference large about to care about' Likewise if the difference between a real human brain and an arbitrarily sophisticated Chinese Room brain is arbitrarily small, they are the same. If our limited understanding of physics and engineering makes the practical difference not zero, this essentially becomes a bit of a somewhat magical 'superscience' argument claiming we can't simulate the real world to a good enough resolution that the meaningful differences between our 'consciousness simulator' and the thing itself disappear - which is an extraordinary claim. | | |
| ▲ | CrossVR 6 days ago | parent [-] | | > What are his arguments then? They're in the "Complete Argument" section of the article. > This sounds like the archetypical no true scotsman fallacy. I get what you're trying to say, but he is not arguing only a true Scotsman is capable of thought. He is arguing that our current machines lack the required "causal powers" for thought. Powers that he doesn't prescribe to only a true Scotsman, though maybe we should try adding bagpipes to our AI just to be sure... | | |
| ▲ | torginus 6 days ago | parent [-] | | Thanks, but that makes his arguments even less valid. He argues that computer programs only manipulate symbols and thus have no semantic understanding. But that's not true - many programs, like compilers that existed back when the argument was made, had semantic understanding of the code (in a limited way, but they did have some understanding about what the program did). LLMs in contrast have a very rich semantic understanding of the text they parse - their tensor representations encode a lot about each token, or you can just ask them about anything - they might not be human level at reading subtext, but they're not horrible either. | | |
| ▲ | CrossVR 6 days ago | parent [-] | | Now you're getting to the heart of the thought experiment. Because does it really understand the code or subtext, or is it just really good at fooling us that it does? When it makes a mistake, did it just have a too limited understanding or did it simply not get lucky with its prediction of the next word? Is there even a difference between the two? I would like to agree with you that there's no special "causal power" that Turing machines can't emulate. But I remain skeptical, not out of chauvinism, but out of caution. Because I think it's dangerous to assume an AI understands a problem simply because it said the right words. |
|
|
|
| |
| ▲ | dahart 6 days ago | parent | prev [-] | | > I cannot help but conclude that Searle argues that ‘understanding’ is only something that humans can do, which means… Regardless of whether Searle is right or wrong, you’ve jumped to conclusions and are misunderstanding his argument and making further assumptions based on your misunderstanding. Your argument is also ad-hominem by accusing people of believing things they don’t believe. Maybe it would be prudent to read some of the good critiques of Searle before trying to litigate it rapidly and sloppily on HN. The randomness stuff is very straw man, definitely not a good argument, best to drop it. Today’s LLMs are deterministic, not random. Pseudorandom sequences come in different varieties, but they model some properties of randomness, not all of them. The functioning of today’s neural networks, both training and inference, is exactly a book of static rules, despite their use of pseudorandom sequences. In case you missed it in the WP article, most of the field of cognitive science thinks Searle is wrong. However, they’re largely not critiquing him for using metaphysics, because that’s not his argument. He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. That much is certainly true. Whether there’s a difference in theory is unproven. But today currently there absolutely is a difference in practice, nobody has ever simulated the real world or a human brain using deterministic computation. | | |
| ▲ | torginus 6 days ago | parent [-] | | If scientific consensus is that he's wrong why is he being constantly brought up and defended - am I not right to call them out then? Nobody brings up that light travels through the aether, that diseases are caused by bad humors etc. - is it not right to call out people for stating theory that's believed to be false? >The randomness stuff is very straw man, And a direct response to what armada651 wrote: >I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. > He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. Once again the argument here changed from 'computers which only manipulate symbols cannot create consciousness' to 'we don't have the algorithm for consiousness yet'. He might have successfully argued against the expert systems of his time - and true, mechanistic attempts at language translation have largely failed - but that doesn't extend to modern LLMs (and pre LLM AI) or even statistical methods. | | |
| ▲ | dahart 6 days ago | parent [-] | | You’re making more assumptions. There’s no “scientific consensus” that he’s wrong, there are just opinions. Unlike the straw man examples you bring up, nobody has proven the claims you’re making. If they had, then the argument would go away like the others you mentioned. Where did the argument change? Searle’s argument that you quoted is not arguing that we don’t have the algorithm yet. He’s arguing that the algorithm doesn’t run on electrical computers. I’m not defending his argument, just pointing out that yours isn’t compelling because you don't seem to fully understand his, at least your restatement of it isn’t a good faith interpretation. Make his argument the strongest possible argument, and then show why it doesn’t work. IMO modern LLMs don’t prove anything here. They don’t understand anything. LLMs aren’t evidence that computers can successfully think, they only prove that humans are prone to either anthropomorphic hyperbole, or to gullibility. That doesn’t mean computers can’t think, but I don’t think we’ve seen it yet, and I’m certainly not alone there. | | |
| ▲ | torginus 5 days ago | parent [-] | | >most of the field of cognitive science thinks Searle is wrong. >There’s no “scientific consensus” that he’s wrong, there are just opinions. | | |
| ▲ | dahart 5 days ago | parent [-] | | And? Are you imagining that these aren’t both true at the same time? If so, I’m happy to explain. Since nothing has been proven, there’s nothing “scientific”. And since there’s some disagreement, “consensus” has not been achieved yet. This is why your presumptive use of “scientific consensus” was not correct, and why the term “scientific consensus” is not the same thing as “most people think”. A split of 60/40 or 75/25 or even 90/10 counts as “most” but does not count as “consensus”. So I guess maybe be careful about assuming what something means, it seems like this thread was limited by several incorrect assumptions. |
|
|
|
|
|
| |
| ▲ | globnomulous 6 days ago | parent | prev | next [-] | | > The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there. That's one possibility. The other is that your pomposity and dismissiveness towards the entire field of philosophy speaks volumes on how little you know about either philosophical arguments in general or this philosophical argument in particular. | | |
| ▲ | torginus 6 days ago | parent [-] | | Another ad hominem, I'd like you to refute my claim that Searle's argument is essentially 100% magical thinking. And yes, if for example, medicine would be no worse at curing cancer than it is today, yet doctors asserted that crystal healing is a serious study, that would reflect badly on the field at large, despite most of it being sound. | | |
| ▲ | dahart 6 days ago | parent | next [-] | | Searle refutes your claim that there’s magical thinking. “Searle does not disagree with the notion that machines can have consciousness and understanding, because, as he writes, "we are precisely such machines". Searle holds that the brain is, in fact, a machine, but that the brain gives rise to consciousness and understanding using specific machinery.” | | |
| ▲ | torginus 6 days ago | parent [-] | | But the core of the original argument is that programs only manipulate symbols and consciousness can never arise just through symbol manipulation - which here then becomes 'we have not discovered the algorithms' for consciousness yet. It's just a contradiction. | | |
| ▲ | dahart 6 days ago | parent [-] | | When you say something that contradicts his statements, it doesn’t mean he’s wrong, it most likely means you haven’t understood or interpreted his argument correctly. The Wikipedia page you linked to doesn’t use the word “algorithm”, so the source of the contradiction you imagine might be you. Searle says he thinks humans are biological machines, so your argument should withstand that hypothesis rather than dismiss it. |
|
| |
| ▲ | globnomulous 6 days ago | parent | prev | next [-] | | Why on earth do you take it as an ad hominem attack? Do you really think your comment isn't dismissive or pompous? | |
| ▲ | malfist 6 days ago | parent | prev [-] | | Another ad hominem, just like you calling anyone who talks about the chinese room thought experiment a deist? |
|
| |
| ▲ | simianparrot 6 days ago | parent | prev | next [-] | | It is still relevant because it hasn’t been disproven yet. So far all computer programs are Chinese Rooms, LLM’s included. | | |
| ▲ | IanCal 6 days ago | parent [-] | | If you’re talking about it being proven or disproven you’re misunderstanding the point of the thought experiment. |
| |
| ▲ | herculity275 6 days ago | parent | prev | next [-] | | "Please stop referring to this thought experiment because it has possible interpretations I don't personally agree with" | | |
| ▲ | torginus 6 days ago | parent [-] | | Please give me an interpretation that is both correct an meaningful (as in possible to disprove) |
| |
| ▲ | welferkj 6 days ago | parent | prev [-] | | The human way of understanding things can be replicated mechanically, because it is mechanical in nature. The contents of your skull are an existence proof of AGI. | | |
| ▲ | hiatus 6 days ago | parent | next [-] | | The A stands for artificial. | |
| ▲ | armada651 6 days ago | parent | prev [-] | | The contents of my skull are only a proof for AGI if your mechanical machine replicates all its processes. It's not a question about whether a machine can reproduce that, it's a question about whether we have given our current machines all the tools it needs to do that. | | |
| ▲ | torginus 6 days ago | parent [-] | | The theory of special relativity does not say 'you can't exceed the speed of light(unless you have a really big rocket)'. It presents a theoretical limit. Likewise the Chinese room doesn't state that consciousness is an intractable engineering problem, but an impossibility. But the way Searle formulates his argument, by not defining what consciousness is, he essentially gives himself enough wiggle room to be always right - he's essentially making the 'No True Scotsman' fallacy. |
|
|
|
|
|
| |
| ▲ | empiko 6 days ago | parent | next [-] | | I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service. | | |
| ▲ | lelanthran 6 days ago | parent | next [-] | | > I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service. It is. It's wild to me that all these VCs pouring money into AI companies don't know what a value-chain is. Tokens are the bottom of the value-chain; it's where the lowest margins exist because the product at that level is a widely available commodity. I wrote about this already (shameless plug: https://www.rundata.co.za/blog/index.html?the-ai-value-chain ) | |
| ▲ | physicsguy 6 days ago | parent | prev [-] | | On top of that, the on-device models have got stronger and stronger as the base models + RL has got better. You can do on your laptop now what 2 years ago was state of the art. |
| |
| ▲ | gnfargbl 6 days ago | parent | prev | next [-] | | Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality. I tend personally to stick with ChatGPT most of the time, but only because I prefer the "tone" of the thing somehow. If you forced me to move to Gemini tomorrow I wouldn't be particularly upset. | | |
| ▲ | motorest 6 days ago | parent [-] | | > Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality. Gemini holds indeed the top spot, but I feel you framed your response quite well: they are all broadly comparable. The difference in the synthetic benchmark from the top spot and the 20th spot was something like 57 points on a scale of 0-1500 |
| |
| ▲ | Keyframe 6 days ago | parent | prev | next [-] | | " in many dimensions they lag behind GPT-5 class " - such as? Outside of computer, "the moat" is also data to train on. That's an even wider moat. Now, google has all the data. Data no one else has or ever will have. If anything, I'd expect them to outclass everyone by a fat margin. I think we're seeing that on video however. | | |
| ▲ | ivape 6 days ago | parent | next [-] | | You think Chinese companies are short on data and people? Google doesn’t have an advantage there until the CCP takes on a more hands on approach. Tin foil hat time: - If you were a God and you wanted to create an ideal situation for the arrival of AI - It would make sense to precede it with a social media phenomena that introduces mass scale normalization of sharing of personal information Yes, that would be ideal … People can’t stop sharing and creating data on anything, for awhile now. It’s a perfect situation for AI as an independent, uncontrollable force. | | |
| ▲ | rusk 6 days ago | parent [-] | | > People can’t stop sharing and creating data on anything Garbage in. Garbage out. There has never been a better time to produce an AI that mimics a racist uneducated teenager. | | |
| |
| ▲ | willvarfar 6 days ago | parent | prev | next [-] | | not according to google: “We have no moat, and neither does OpenAI”: the big memo and the big HN thread on same https://news.ycombinator.com/item?id=35813322 | | |
| ▲ | Keyframe 6 days ago | parent | next [-] | | a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything.. | | |
| ▲ | lelanthran 6 days ago | parent | next [-] | | > a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything.. Yeah, Google totally has a moat. Them saying that they have no moat doesn't magically make that moat go away. They also own the entire vertical which none of the competitors do - all their competitors have to buy compute from someone who makes a profit just on compute (Nvidia, for example). Google owns the entire vertical, from silicon to end-user. It would be crazy if they can't make this work. | |
| ▲ | rvba 6 days ago | parent | prev | next [-] | | That's why robots make so much traffic now. Those other companies are trying to get data. Google theoretically has reddit access. I wonder if they have sort of an internet archive - data unpolutted by LLMs On a side note, funny how all the companies seem to train on book archivr which they just downloaded from the internet | |
| ▲ | lrem 6 days ago | parent | prev [-] | | > all of the videos, [...], all of the user's search interest, ads, everything.. And privacy policies that are actually limiting what information gets used in what. | | |
| |
| ▲ | IncreasePosts 6 days ago | parent | prev [-] | | That's one person's opinion that works for Google. |
| |
| ▲ | seunosewa 6 days ago | parent | prev [-] | | counterpoint: with their aggressive crawlers, most AI companies can have as much data as google... |
| |
| ▲ | motorest 6 days ago | parent | prev | next [-] | | > This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given). I don't know what you are talking about. I use Gemini on a daily basis and I honestly can't tell a difference. We are at a point where training corpus and hallucinations makes more of a difference than "model class". | |
| ▲ | jorisboris 6 days ago | parent | prev | next [-] | | Yes, or Apple who with all the talent don’t manage to pull off anything useful in AI xAI seems to be the exception, not the rule | | |
| ▲ | rusk 6 days ago | parent [-] | | Given Apple’s moat is their devices, their particular spin on AI is very much edge focussed, which isn’t as spectacular as the current wave of cloud based LLM. Apple’s cloud stuff is laughably poor. |
| |
| ▲ | jeanloolz 6 days ago | parent | prev | next [-] | | Depending on how you look at it I suppose but I believe Gemini surpasses OpenAI on many levels now. Better photo and video models. The leaderboard for text and embeddings are also putting Google on top of Openai. | |
| ▲ | ebonnafoux 6 days ago | parent | prev | next [-] | | gemini-2.5-pro is ranked number 1 in llmarena (https://lmarena.ai/leaderboard) before gpt-5-high. In the Text-to-Video and Image-to-video, google also have the highest places, OpenAI is nowhere. | | |
| ▲ | IX-103 6 days ago | parent [-] | | Yes, but they're also slower. As LLMs start to be used for more general purpose things, they are becoming a productivity bottle-neck. If I get a mostly right answer in a few seconds that's much better than a perfect answer in 5 minutes. Right now the delay for Google's AI coding assistant is high enough for humans to context switch and do something else while waiting. Particularly since one of the main features of AI code assistants is rapid iteration. | | |
| ▲ | janalsncm 6 days ago | parent [-] | | Anecdotally, Gemini pro is way faster than GPT 5 thinking. Flash is even faster. I have no numbers though. |
|
| |
| ▲ | paulddraper 6 days ago | parent | prev [-] | | It doesn’t guarantee success, but the point stands about X and Deepseek |
|
