| ▲ | Why I don't think AGI is imminent(dlants.me) |
| 34 points by anonymid 3 hours ago | 65 comments |
| |
|
| ▲ | NiloCK an hour ago | parent | next [-] |
| > The transformer architectures powering current LLMs are strictly feed-forward. This is true in a specific contextual sense (each token that an LLM produces is from a feed-forward pass). But untrue for more than a year with reasoning models, who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully. Heck, it was untrue before that as well, any time an LLM responded with more than one token. > A [March] 2025 survey by the Association for the Advancement of Artificial Intelligence (AAAI), surveying 475 AI researchers, found that 76% believe scaling up current AI approaches to achieve AGI is "unlikely" or "very unlikely" to succeed. I dunno. This survey publication was from nearly a year ago, so the survey itself is probably more than a year old. That puts us at Sonnet 3.7. The gap between that and present day is tremendous. I am not skilled enough to say this tactfully, but: expert opinions can be the slowest to update on the news that their specific domain may have, in hindsight, have been the wrong horse. It's the quote about it being difficult to believe something that your income requires to be false, but instead of income it can be your whole legacy or self concept. Way worse. > My take is that research taste is going to rely heavily on the short-duration cognitive primitives that the ARC highlights but the METR metric does not capture. I don't have an opinion on this, but I'd like to hear more about this take. |
|
| ▲ | hi_hi 42 minutes ago | parent | prev | next [-] |
| Here's a thought. Lets all arbitrarily agree AGI is here. I can't even be bothered discussing what the definition of AGI is. It's just here, accept it. Or vice versa. Now what....? Whats happening right now that should make me care that AGI is here (or not). Whats the magic thing thats happening with AGI that wasn't happening before? <looks out of window>
<checks news websites>
<checks social media...briefly>
<asks wife> Right, so, not much has changed from 1-2 years ago that I can tell. The job markets a bit shit if you're in software...is that what we get for billions of dollars spent? |
|
| ▲ | Lerc 2 hours ago | parent | prev | next [-] |
| I don't really understand the argument that AGI cannot be achieved just by scaling current methods. I too believe that (for any sane level of scaling anyway), but this-year's LLMs are not using entirely last-year's methods. And they, in turn, are using methods that weren't used the year before. It seems like a prediction like "Bob won't become a formula one driver in a minivan". It's true, but not very interesting. If Bob turned up a couple of years later in Formula one, you'd probably be right in saying that what he is driving is not a mini van. The same is true for AGI anyone who says it can't be done with current methods can point to any advancement along the way and say that's the difference. A better way to frame it would be, is there any fundimental, quantifiable ability that is blocking AGI? I would not be surprised if the breakthrough technique has been created, but the research has not described the problem that it solves well enough for us to know that it is the breakthrough. I realise that, for some the notion of AGI is relatively new, but some of us have been considering the matter for some time. I suspect my first essay on the topic was around 1993. It's been quite weird watching people fall into all of the same philosophical potholes that were pointed out to us at university. |
| |
| ▲ | hunterpayne 8 minutes ago | parent | next [-] | | Then you don't understand Machine Learning in any real way. Literally the 3rd or 4th thing you learn about ML is that for any given problem, there is an ideal model size. Just making the model bigger doesn't work because of something called the curse of dimensionality. This is something we have discovered about every single problem and type of learning algorithm used in ML. For LLMs, we probably moved past the ideal model size about 18 months ago. From the POV of something who actually learned ML in school (from the person who coined the term), I see no real reason to think that AGI will happen based upon the current techniques. Maybe someday. Probably not anytime soon. PS The first thing you learn about ML is to compare your models to random to make sure the model didn't degenerate during training. | |
| ▲ | trial3 2 hours ago | parent | prev [-] | | i think the minivan analogy is flawed, and that AGI is moving from "bob driving a minivan" to "bob literally becoming the thing that is formula one" | | |
| ▲ | Lerc an hour ago | parent [-] | | What would that even mean though? Who is making claims of that sort? I feel like it's such a bending of the idea,that it's not really making a prediction of anything at all. |
|
|
|
| ▲ | Animats 2 hours ago | parent | prev | next [-] |
| Now that understanding video and projecting what happens next indicates we're getting past the LLM problem of lacking a world model. That's encouraging. There's more than one way to do intelligence. Basic intelligence has evolved independently three times that we know of - mammals, corvids, and octopuses. All three show at least ape-level intelligence, but the species split before intelligence developed, and the brain architectures are quite different. Corvids get more done with less brain mass than mammals, and don't have a mammalian-type cortex. Octopuses have a distributed brain architecture, and have a more efficient eye design than mammals. |
|
| ▲ | nsainsbury 2 hours ago | parent | prev | next [-] |
| I used to also believe along these lines but lately I'm not so sure. I'm honestly shocked by the latest results we're seeing with Gemini 3 Deep Think, Opus 4.6, and Codex 5.3 in math, coding, abstract reasoning, etc. Deep Think just scored 84.6% on ARC-AGI-2 (https://deepmind.google/models/gemini/)! And these benchmarks are supported by my own experimentation and testing with these models ~ specifically most recently with Opus 4.6 doing things I would have never thought possible in codebases I'm working in. These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population. And then combine that with the latest video output we're seeing from Seedance 2.0, etc showing an incredible level of image/video understanding and generation capability. I was previously deeply skeptical that the architecture we have would be sufficient to get us to AGI. But my belief in that has been strongly rattled lately. Honestly I think the greatest gap now is simply one of orchestration, data presentation, and work around in-context memory representations - that is, converting work done into real world into formats/representations, etc. amenable for AI to run on (text conversion, etc.) and keeping new trained/taught information in context to support continual learning. |
| |
| ▲ | 9x39 an hour ago | parent | next [-] | | >These models are demonstrating an incredible capacity for logical abstract reasoning of a level far greater than 99.9% of the world's population. This is the key I think that Altman and Amodei see, but get buried in hype accusations. The frontier models absolutely blow away the majority of people on simple general tasks and reasoning. Run the last 50 decisions I've seen locally through Opus 4.6 or ChatGPT 5.2 and I might conclude I'd rather work with an AI than the human intelligence. It's a soft threshold where I think people saw it spit out some answers during the chat-to-LLM first hype wave and missed that the majority of white collar work (I mean it all, not just the top software industry architects and senior SWEs) seems to come out better when a human is pushed further out of the loop. Humans are useful for spreading out responsibility and accountability, for now, thankfully. | |
| ▲ | lostmsu an hour ago | parent | prev [-] | | While I think 99.9% is overstating it, I can believe that number is strictly more than 1% at this point. |
|
|
| ▲ | parpfish 2 hours ago | parent | prev | next [-] |
| I’d love to see one of the AI behemoths put their money where their mouth is and replace their C-suite with their SOTA chatbot. |
|
| ▲ | 9x39 an hour ago | parent | prev | next [-] |
| There was a meme going around that said the fall of Rome was an unannounced anticlimactic event where one day someone went out and the bridge wasn't ever repaired. Maybe AGI's arrival is when one day someone is given an AI to supervise instead of a new employee. Just a user who's followed the whole mess, not a researcher. I wonder if the scaffolding and bolt-ons like reasoning will sufficiently be an asymptote to 'true AGI'. I kept reading about the limits of transformers around GPT-4 and Opus 3 time, and then those seem basic compared to today. I gave up trying to guess when the diminishing returns will truly hit, if ever, but I do think some threshold has been passed where the frontier models are doing "white collar work as an API" and basic reasoning better than the humans in many cases, and once capital familiarizes themselves with this idea more, it's going to get interesting. |
| |
| ▲ | esafak an hour ago | parent | next [-] | | But it's already like that; models are better than many workers, and I'm supervising agents. I'd rather have the model than numerous juniors; esp. the kind that can't identify mistakes in the model. | | |
| ▲ | causal an hour ago | parent [-] | | This is my greatest cause for alarm regarding LLM adoption. I am not yet sure AI will ever be good enough to use without experts watching them carefully; but they are certainly good enough that non-experts cannot tell the difference. |
| |
| ▲ | an hour ago | parent | prev | next [-] | | [deleted] | |
| ▲ | beej71 an hour ago | parent | prev [-] | | I'd always imagined that AGI meant an AI was given other AIs to manage. |
|
|
| ▲ | xutopia an hour ago | parent | prev | next [-] |
| I’m under the same impression. I don’t think LLMs are the path to AGI. The “intelligence” we see is mostly illusory. It’s statistical repetition of the mediocre minds who wrote content online. The intelligence we think we recognize is simply an electronic parrot finding the right words in its model to make itself useful. |
| |
| ▲ | causal an hour ago | parent [-] | | I fear that AI will be intelligent enough to negate human general intelligence before it is itself generally intelligent. |
|
|
| ▲ | famouswaffles an hour ago | parent | prev | next [-] |
| State of the Art Large Language Models are already Generally Intelligent, in so far as the term has any useful meaning. Their biggest weakness are long horizon planning competency, and spatial reasoning and navigation, both of which continue to improve steadily and are leaps and bounds above where they were a few years ago. I don't think there's any magic wall. Eventually they will simply get good enough, just like everything else. |
|
| ▲ | mikewarot 2 hours ago | parent | prev | next [-] |
| I think that AGI has already happened, but it's not well understood, nor well distributed yet. OpenClaw, et al, are one thing that got me nudged a little bit, but it was Sammy Jankis[1,2] that pushed me over the edge, with force. It's janky as all get out, but it'll learn to build it's own memory system on top of an LLM which definitely forgets. [1] https://sammyjankis.com/ [2] https://news.ycombinator.com/item?id=47018100 |
| |
| ▲ | hermitShell 2 hours ago | parent [-] | | The Sammy Jankis link was certainly interesting. Thanks for sharing. Whether or not AGI is imminent, and whether or not Sammy Jankis is or will be conscious... it's going to become so close that for most people, there will be no difference except to philosophers. Is AGI 'right around the corner' or currently already achieved? I agree with the author, no, we have something like 10 years to go IMO. At the end of the post he points to the last 30 years of research, and I would accept that as an upper bound. In 10 to 30 years, 99% of people won't be able to distinguish between an 'AGI' and another person when not in meatspace. |
|
|
| ▲ | hi_hi 2 hours ago | parent | prev | next [-] |
| How will we know if its AGI/Not AGI? (I don't think a simple app is gonna cut it here haha) What is the benchmark now that the Turing test has been blown out of the water? |
| |
| ▲ | jltsiren an hour ago | parent | next [-] | | Until recently, philosophy of artificial intelligence seemed to be mostly about arguments why the Turing test was not a useful benchmark for intelligence. Pretty much everyone who had ever thought about the problem seriously had come to the same conclusion. The fundamental issue was the assumption that general intelligence is an objective property that can be determined experimentally. It's better to consider intelligence an abstraction that may help us to understand the behavior of a system. A system where a fixed LLM provides answers to prompts is little more than a Chinese room. If we give the system agency to interact with external systems on its own initiative, we get qualitatively different behavior. The same happens if we add memory that lets the system scale beyond the fixed context window. Now we definitely have some aspects of general intelligence, but something still seems to be missing. Current AIs are essentially symbolic reasoning systems that rely on a fixed model to provide intuition. But the system never learns. It can't update its intuition based on its experiences. Maybe the ability to learn in a useful way is the final obstacle on the way towards AGI. Or maybe once again, once we start thinking we are close to solving intelligence, we realize that there is more to intelligence than what we had thought so far. | |
| ▲ | beej71 an hour ago | parent | prev | next [-] | | I like the line of thinking from an earlier commenter: when an AI company no longer has any humans working, we'll know we're there. | |
| ▲ | pixl97 an hour ago | parent | prev | next [-] | | There is a different way I look at this. Humans will never accept we created AI, they'll go so far as to say we were not intelligent in the first place. That is the true power of the AI effect. | |
| ▲ | jobs_throwaway 2 hours ago | parent | prev | next [-] | | Supranormal GDP growth is my bar. When its actually able to get around bottlenecks and produce value on a societal level | |
| ▲ | lostmsu an hour ago | parent | prev [-] | | To my knowledge Turing test has not been blown out of the water. The forms I saw were time limited and participants were not pushed hard to interrogate. |
|
|
| ▲ | est31 2 hours ago | parent | prev | next [-] |
| Our brains evolved to hunt prey, find mates, and avoid becoming hunted ourselves. Those three tasks were the main factors for the vast majority of evolutionary history. We didn't evolve our brains to do math, write code, write letters in the right registers to government institutions, or get an intuition on how to fold proteins. For us, these are hard tasks. That's why you get AI competing at IMO level but unable to clean toilets or drive cars in all of the settings that humans do. |
| |
| ▲ | dd8601fn 2 hours ago | parent | next [-] | | I'm not excited about a future where the division of labor is something like: AI does all of the interesting stuff and the humans clean the toilets. Especially now that I'm older and my joints won't tolerate it. | | |
| ▲ | beloch 2 hours ago | parent | next [-] | | It's not that AI is intrinsically better at software engineering, writing, or art than it is at learning how to clean toilets. It's not. The real issue is that cleaning toilets using humans is cheap. That, sadly, is the incentive driving the current wave of AI innovation. Your job will be automated long before your household chores are. | |
| ▲ | martin-t 2 hours ago | parent | prev [-] | | Don't be ridiculous, AI will create robots that do all the work and the only use for humans will be as amusement for the rich who own everything. Probably not sarcasm, I don't even know. |
| |
| ▲ | nozzlegear 28 minutes ago | parent | prev | next [-] | | > Our brains evolved to hunt prey, find mates, and avoid becoming hunted ourselves. Those three tasks were the main factors for the vast majority of evolutionary history. That seems like a massive oversimplification of the things our brains evolved to do. | |
| ▲ | andsoitis 2 hours ago | parent | prev [-] | | > We didn't evolve our brains to do math, write code, write letters in the right registers to government institutions, or get an intuition on how to fold proteins. For us, these are hard tasks. Humans discovered or invented all of those. | | |
| ▲ | pixl97 2 hours ago | parent | next [-] | | And it took a massively long time for that to happen after we gained that capability. Human ingenuity really only took off after we put a lot of the work on writing and tools. It wasn't so much that humans created many of these, but the super human organism that uses language and writing to express ideas. Now think about what we just created. | |
| ▲ | alex43578 2 hours ago | parent | prev [-] | | Only in small ways and very recently, evolutionarily speaking, were those things rewarded by natural selection (and even that has stopped nowadays). |
|
|
|
| ▲ | AngryData 2 hours ago | parent | prev | next [-] |
| Until I can get a robot wife maid im not worried about or even confident I will ever see actual AGI. People have been predicting it for as long as fusion power and while progress has been made, we might still be like Romans dreaming of flight. |
| |
| ▲ | pixl97 2 hours ago | parent [-] | | Dear sir, what does embodiment actually have to do with agi? Not much different than saying someone that is paralyzed is not intelligence. More so, our recent advances in AI have massively accelerated robotics evolution. They are becoming smarter, faster, and more capable at an ever increasing rate. |
|
|
| ▲ | ed_mercer 2 hours ago | parent | prev | next [-] |
| As far as I'm concerned, it's already here. |
|
| ▲ | tananaev 2 hours ago | parent | prev | next [-] |
| I think it's really poor argument that AGI won't happen because model doesn't understand physical world. That can be trained the same way everything else is. I think the biggest issue we currently have is with proper memory. But even that is because it's not feasible to post-train an individual model on its experiences at scale. It's not a fundamental architectural limitation. |
| |
| ▲ | stagezerowil 2 hours ago | parent [-] | | When people move the goal posts for AGI toward a physical state, they are usually doing it so they can continue to raise more funding rounds at a higher valuation. Not saying the author is doing that. |
|
|
| ▲ | simbleau 2 hours ago | parent | prev | next [-] |
| AGI is a messy term, so to be concise, we have the models that can do work. What we lack is orchestration, management, and workflows to use models effectively. Give it 5 years and those will be built and they could be built using the models we have today (Opus 4.6 at the time of this message). |
|
| ▲ | nickjj 2 hours ago | parent | prev | next [-] |
| I'm certainly not holding my breath. In a handful of prompts I got the paid version of ChatGPT to say it's possible for dogs to lay eggs under the right circumstances. |
| |
| ▲ | SoftTalker 2 hours ago | parent [-] | | Do you believe you could not find humans who would do this? | | |
| ▲ | raddan 2 hours ago | parent | next [-] | | That's not really the point. If our definition of AGI does not include "being able to reliably do logic" then what are we even talking about? We don't really need computers with human abilities--we have plenty of humans. We need computers with _better_ abilities. | | |
| ▲ | AnimalMuppet 2 hours ago | parent [-] | | OK, but "what we need" is not the question. If the definition of AGI is "as smart as the average human in all areas", then it doesn't matter if the average human is pretty useless at a lot of tasks, that's still the definition of AGI. But I'd like to think that, even though you could find exceptions, the average human is never confused about whether dogs can lay eggs or not. | | |
| ▲ | pixl97 an hour ago | parent [-] | | I reached your view the day my grandma told me I was wrong and a hummingbird was a type of insect... Like, it's in the name. |
|
| |
| ▲ | 2 hours ago | parent | prev [-] | | [deleted] |
|
|
|
| ▲ | TMWNN 2 hours ago | parent | prev | next [-] |
| If AGI can be defined as meeting the general intelligence of a Redditor, we hit ASI a while ago. Highly relevant comment <https://www.reddit.com/r/singularity/comments/1jh9c90/why_do...> by /u/Pyros-SD-Models: >Imagine you had a frozen [large language] model that is a 1:1 copy of the average person, let’s say, an average Redditor. Literally nobody would use that model because it can’t do anything. It can’t code, can’t do math, isn’t particularly creative at writing stories. It generalizes when it’s wrong and has biases that not even fine-tuning with facts can eliminate. And it hallucinates like crazy often stating opinions as facts, or thinking it is correct when it isn't. >The only things it can do are basic tasks nobody needs a model for, because everyone can already do them. If you are lucky you get one that is pretty good in a singular narrow task. But that's the best it can get. >and somehow this model won't shut up and tell everyone how smart and special it is also it claims consciousness. ridiculous. |
| |
|
| ▲ | Legend2440 2 hours ago | parent | prev | next [-] |
| I've said it before and I'll say it again, all AI discussion feels like a waste of effort. “yes it will”, “no it won’t” - nobody really knows, it's just a bunch of extremely opinionated people rehashing the same tired arguments across 800 comments per thread. There’s no point in talking about it anymore, just wait to see how it all turns out. |
| |
| ▲ | barfiure an hour ago | parent [-] | | Nope. Not good enough. Your approach won’t drive engagement. We need the same tired arguments across 1600 comments per thread. |
|
|
| ▲ | ryanSrich 2 hours ago | parent | prev [-] |
| AGI is here. 90%+ of white collar work _can_ be done by an LLM. We are simply missing a tested orchestration layer. Speaking broadly about knowledge work here, there is almost nothing that a human is better at than Opus 4.6. Especially if you're a typical office worker whose job is done primarily on a computer, if that's all AGI is, then yeah, it's here. |
| |
| ▲ | causal 2 hours ago | parent | next [-] | | Opus is the very best and I still throw away most of what it produces. If I did not carefully vet its work I would degrade my code bases so quickly.
To accurately measure the value of AI you must include the negative in your sum. | |
| ▲ | loloquwowndueo 2 hours ago | parent | prev | next [-] | | > there is almost nothing that a human is better at than Opus 4.6. Lolwut. I keep having to correct Claude at trivial code organization tasks. The code it writes is correct; it’s just ham-fisted and violates DRY in unholy ways. And I’m not even a great coder… | | |
| ▲ | causal 2 hours ago | parent | next [-] | | > violates DRY in unholy ways Well said | |
| ▲ | danenania an hour ago | parent | prev [-] | | I’m very pro AI coding and use it all day long, but I also wouldn’t say “the code it writes is correct”. It will produce all kinds of bugs, vulnerabilities, performance problems, memory leaks, etc unless carefully guided. |
| |
| ▲ | JSDave 2 hours ago | parent | prev | next [-] | | AGI is when it can do all intellectual work that can be done by humans. It can improve its own intelligence and create a feedback loop because it is as smart as the humans who created it. | | |
| ▲ | pixl97 an hour ago | parent | next [-] | | No, that is ASI. No human can do all intellectual work themselves. You have millions of different human models based on roughly the same architecture to do that. When you have a single model that can do all you require, you are looking at something that can run billions of copies of itself and cause an intelligence explosion or an apocalypse. | | |
| ▲ | JSDave 17 minutes ago | parent [-] | | "Artificial general intelligence (AGI) is a type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks." |
| |
| ▲ | 9x39 2 hours ago | parent | prev [-] | | Why the super-high bar? What's unsatisfying is that aren't the 'dumbest' humans still a general intelligence that we're nearly past, depending how you squint and measure? It feels like an arbitrary bar to perhaps make sure we aren't putting AIs over humans, which they are most certainly in the superhuman category on a rapidly growing number of tasks. |
| |
| ▲ | lysace 2 hours ago | parent | prev [-] | | That "simple orchestration layer" (paraphrased) is what I consider the AGI. But yeah, I suspect LLM:s may actually get close enough. "Just" add more reasoning loops and corresponding compute. It is objectively grotesquely wasteful (a human brain operates on 12 to 25 watts and would vastly outperform something like that), but it would still be cataclysmic. /layperson, in case that wasn't obvious | | |
| ▲ | pixl97 an hour ago | parent | next [-] | | If we can get AI down to this power requirement then it's over for humans. Just think of how many copies of itself thinking at the levels of the smartest humans it could run at once. Also where all the hardware could hide itself and keep itself powered around the world. | |
| ▲ | jonas21 2 hours ago | parent | prev | next [-] | | > a human brain operates on 12 to 25 watts Yeah, but a human brain without the human attached to it is pretty useless. In the US, it averages out to around 2 kW per person for residential energy usage, or 9 kW if you include transportation and other primary energy usage too. | | |
| ▲ | lysace 2 hours ago | parent [-] | | Fair. Maybe the Matrix (1999) with the human battery farms were on to something. :) |
| |
| ▲ | ryanSrich 2 hours ago | parent | prev [-] | | I think "tested" is the hard part. The simple part seems to be there already, loops, crons, and computer use is getting pretty close. |
|
|