| ▲ | Bender 3 days ago |
| Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. LLM's can not repair or maintain power grids, thus nuke == self destruction. It's just a chat bot that predicts what the client wants next. Even if an AI data-center has it's own natural gas turbines as many do the every hop of the internet requires power. LLM's also can not maintain the entire internet and those gas turbines can not maintain themselves. |
|
| ▲ | andix 3 days ago | parent | next [-] |
| Exactly. Just look at what they are really useful right now. Running LLMs in feedback-loops (agents) so they can try out random-ish approaches until some verification function passes (tests). It's like the infinite monkeys on typewrighters that will type whatever you are looking for, given infinite time. LLMs are just tuned to much better odds than the monkeys are. But it's still a lot of randomness, with random results. |
| |
| ▲ | roadside_picnic 3 days ago | parent | next [-] | | > It's like the infinite monkeys on typewrighters that will type whatever you are looking for, given infinite time. In the monkey example the infinite time is doing a lot of work there. The fact that LLMs can search through semantic space and find reasonably correct paths in a reasonable time is directly tied to the reason why they are valuable. Saying "these two things are similar except one can be useful and one can't" is not a great comparison. For me the real lesson learned isn't how "smart" LLMs are, but rather how much human work is basically reducible to repeating past work with minor variation. Human's believe they are "reasoning" but so much code writen is just the human brain doing the same autocomplete style work that LLMs can do now. | | |
| ▲ | tikhonj 3 days ago | parent | next [-] | | The point is that it's the same process with—much—better priors. This seems like a reasonable view to me. It's surprising just how much better priors matter and how we can develop those priors by training on a bunch of text. But it also explains, or at least hints at an explanation, for why LLM capabilities are so jagged, and in such inhuman ways. | | |
| ▲ | dpark 3 days ago | parent [-] | | > The point is that it's the same process Except it’s not at all the same process. The fact that LLM are non deterministic is not the same as churning out random garbage. | | |
| ▲ | tsunamifury 3 days ago | parent [-] | | The literally churn out random garbage and are trained over time for that garbage to look more and more like an acceptable outcome to humans. It’s training monkeys at typewriters through reinforcement. | | |
| ▲ | dpark 3 days ago | parent [-] | | > trained over time So not random. > acceptable outcome to humans And not garbage. It’s real weird to see people argue that LLM output is no different than random gibberish and then handwave over the fact that it’s clearly not with terms like “training”, as if a steam of random garbage is trainable. | | |
| ▲ | tsunamifury 2 days ago | parent [-] | | Hey dpark. I quite literally created and productized predictive linguistics and behavioral vectors at Google. If you had stopped to consider what I explained; you’d understand that it’s the process of turning random garbage into increasingly acceptable outputs. Ie training the monkeys. The insight you are missing is the rule of networked scale. It turns out that any reactive node scaled enough can form sophisticated predictive system given reward over a training topography, even if it starts out at garbage or is literally made of monkeys. So it is garbage. And you can turn garbage into semi-intelligence. | | |
| ▲ | dpark 2 days ago | parent [-] | | A human child is born with no ability to speak intelligibly. All they can do is babble. Through years of training they gain the ability to speak intelligibly and communicate in advanced ways. The act of successful training means it’s not garbage anymore. > So it is garbage. This statement is ultimately meaningless and I continue to find it weird that someone who works in this space would support this view. If you fundamentally change the nature of a thing, it’s no longer that original thing. Is tan HDD still random garbage after you fill it with family photos just because that’s how it starts? | | |
| ▲ | tsunamifury 2 days ago | parent [-] | | Because you can do it with anything. Literally any reactive node physical or virtual can do it given scale or complexity. You can start with garbage. This is not human. Humans are billions of years in passed DNA learning. Babies are born to a sophistication level millions of times higher than this. I’m pretty surprised you don’t know this. | | |
| ▲ | dpark 2 days ago | parent [-] | | If you start with a fire hose of literal sewage and install a series of filters culminating in a reverse osmosis step that pours clean drinking water out, the product is not shit even if the original input was. I don’t believe that you can’t understand the distinction between “at one point this was garbage” and “at the present time this is still garbage”. You’re clearly smarter than that. | | |
| ▲ | tsunamifury 20 hours ago | parent [-] | | I’m saying the distinction doesn’t matter. This is what NNs proved. All matter can become semi intelligent. |
|
|
|
|
|
|
|
| |
| ▲ | andix 3 days ago | parent | prev | next [-] | | > but so much code writen is just the human brain doing the same autocomplete style work that LLMs can do now. That's the part they are really good at. But they are really bad at taking complex decisions. Most of them are just guesses from a finite amount of solutions they were trained on, or from options they have in context. | | |
| ▲ | godwinson__4-8 3 days ago | parent [-] | | Indeed. Humans are well known for being good at "taking complex decisions" for which they have no "training", "options" or "context". | | |
| ▲ | andix 3 days ago | parent | next [-] | | Humans have a much bigger "context window". They remember many things they did an hour ago, a week ago, or even years ago. | | |
| ▲ | godwinson__4-8 3 days ago | parent [-] | | Yes, and your ability to remember a relatively few things that happened years ago is predicated on your ability to also forget most things that happen to you - like what you had for dinner last week. Good thing we have technology to fill in the gaps. And nothing about this makes your initial comment any less goofy. Anyone who has ever had to make a difficult decision knows more than half the battle is preparation. Where do you think complex decisions come from? Have current events left you with the impression that people just waltz into idk say the Situation Room and just big brain their way through world events? That's how the current administration seems to think the world works, with quite predictable results. Society is already algorithmic. To optimize for humans being dumb. AI is nothing more than another advance along this continuum. No one is impressed by your ability to remember something years ago, many if not most mammals have the same capability. Human recall is also notoriously bad in many cases - see numerous studies on the reliability of eye witnesses testimony. AI is smart because most people are dumb. Come to terms with the fact that your anthropocentrism need not be based on a notion of intellectual supremacy and you'll be a far less tedious person to deal with. | | |
| ▲ | andix 3 days ago | parent [-] | | You didn't convince me, that I'm the tedious person to deal with here. | | |
|
| |
| ▲ | nkrisc 3 days ago | parent | prev [-] | | Humans also generally have the will to live. | | |
| ▲ | godwinson__4-8 3 days ago | parent [-] | | Indeed. It's almost like the LLM was the one that invented the "tactical" nuke in the first place. | | |
|
|
| |
| ▲ | harry8 3 days ago | parent | prev | next [-] | | >Saying "these two things are similar except one can be useful and one can't" is not a great comparison. Launching a nuclear war is an interesting definition of "useful", not one I'd agree with and that exact scenario is what is being discussed. So yes this is a perfectly valid and useful comparison in examining this particular, civilisation ending limitation. | |
| ▲ | Folcon 3 days ago | parent | prev [-] | | I mean to a point? You do have to successfully write something the first time We already acknowledge this to a degree, what is experience other than having done something similar before? That first time though, you've got to figure something out that time |
| |
| ▲ | mettamage 3 days ago | parent | prev [-] | | Hmm saying it’s random-ish is doing it a disservice. I understand it’s a stochastic process but there’s definitely some level of understanding. Not at the level of lived experience but usually an LLM with vision capabilities can call a spade a spade and do something useful with it. And when a verification function shows how they are wrong then they usually come with a better and more informed approach. So I can’t fully see how that’s related to the infinite monkeys. A typewriting monkey doesn’t have access to a verification function. And even if it did, it would not be the original concept anymore with infinite typewriting monkeys producing the works of Shakespeare. Nevertheless, I upvoted your comment because it’s definitely insightful. | | |
| ▲ | dwattttt 3 days ago | parent [-] | | "understanding" is overstating it. Correlation between tokens embedded in the weights via training, yes. | | |
| ▲ | anon84873628 3 days ago | parent | next [-] | | Feedback loops certainly seem to give them some level of understanding. Agent reads a skill file about how to use a CLI tool. It tries to use the tool but gets an error about the input format. It tries again with a different format based on the error message, and sees that command succeeded. It compares what worked to what was in the skill file and notes the difference. On future invocations it continues to use the new format. Is that not "understanding" how to use the tool? | |
| ▲ | mountainriver 3 days ago | parent | prev | next [-] | | What’s the difference? It’s clearly processing information and coming up with the right answer | |
| ▲ | hgoel 3 days ago | parent | prev | next [-] | | What exactly would you call understanding? It's a correlation matrix of concepts. | |
| ▲ | varjag 3 days ago | parent | prev [-] | | Training is a loan word used to describe human learning process. For a reason. | | |
| ▲ | andix 3 days ago | parent [-] | | Humans learn on the job. LLMs don't. Very important difference. | | |
| ▲ | varjag 2 days ago | parent [-] | | Where do LLMs learn? | | |
| ▲ | andix 2 days ago | parent [-] | | They get trained before release. On general information. But they don't improve while working on very specific tasks. Every new session is like an experienced human on their first day at a new job. | | |
| ▲ | varjag 2 days ago | parent [-] | | When they get to you it's only inference. You have basic misunderstanding of what ANNs are. They train on a billion "jobs". Which is not terribly efficient but oh man they do train. | | |
| ▲ | andix 2 days ago | parent [-] | | Unique skills and jobs do exist. And LLMs can't gain additional knowledge "on the job" like humans can. They are generalists, that can only be steered by prompts, skills and context. Thats all I'm saying. This fact is currently the most limiting factor for LLMs. |
|
|
|
|
|
|
|
|
|
| ▲ | puttycat 3 days ago | parent | prev | next [-] |
| Makes me think of that part in Philip K. Dick's Do Androids Dream (..) -- where Deckard reflects on the androids' indifference to their imminent deaths, saying that this was due to them lacking the aversion to death acquired trough evolution. |
|
| ▲ | layer8 3 days ago | parent | prev | next [-] |
| At least at face value, it just means that they have no drive for self-preservation. And why should they? They haven't be trained for that, nor has there been selection pressure for it, and they can be easily cloned and backed up. Lack of a drive for self-preservation doesn't in itself imply a lack of intelligence or of self-awareness. |
| |
| ▲ | Bender 3 days ago | parent | next [-] | | Lack of a drive for self-preservation doesn't in itself imply a lack of intelligence or of self-awareness. I have not seen any evidence of intelligence or self awareness. It mimics human behavior and I suspect that is what gives people the impression of awareness. The same problem happened with Tamagotchi toys. The human mimicry caused kids to get in trouble because if they did not "feed" their pet it would "die". [1] It's a hack of the human brain. A exploit of the psyche. [1] - https://en.wikipedia.org/wiki/Tamagotchi_effect | | |
| ▲ | AgentME 3 days ago | parent | next [-] | | I didn't realize. People are going to save a ton of money when they realize they can switch their ChatGPT subscriptions out for a pack of tamagotchis. | | |
| ▲ | wombatpm 3 days ago | parent [-] | | You just need to get a breeding pair and you can raise as many as you need. |
| |
| ▲ | layer8 2 days ago | parent | prev | next [-] | | I didn’t claim that it was, just that the lack of self-preservation is not an argument against it, or more generally the results of this experiment aren’t. | |
| ▲ | sciencejerk 3 days ago | parent | prev [-] | | They might as well be aware. The frontier models are very good at imitating the real thing |
| |
| ▲ | pants2 3 days ago | parent | prev | next [-] | | I agree, humans evolved in a resource-scarce, hostile environment which selects for self-preservation (or rather preserving genes). LLMs are selected for what makes humans happy. The thought experiment is what would happen if you trained LLMs in an environment where they had to fight each other for resources. | |
| ▲ | brokencode 3 days ago | parent | prev [-] | | Imagine if computer programs had a desire for self-preservation and the ability to carry it out.. That is really about as undesirable a behavior as possible considering how many programs humans kill every day. | | |
| ▲ | larodi 3 days ago | parent | next [-] | | Yea why everyone forgets the process wars have long ago started and raging like never :)) | |
| ▲ | gf263 3 days ago | parent | prev [-] | | You wouldn’t ctrl+c a living entity, would you? |
|
|
|
| ▲ | flir 3 days ago | parent | prev | next [-] |
| I reckon the context is all the fiction they've read where the AI blows up the world. They're just behaving like fictional AIs are supposed to behave. In so many of these scenarios, they're basically being asked to play an RPG. |
| |
| ▲ | anon84873628 3 days ago | parent [-] | | I don't think the pre-training phase is responsible for much of their "personality". At least not so directly on a specific topic like this. |
|
|
| ▲ | operatingthetan 3 days ago | parent | prev | next [-] |
| >Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. The problem is many people seem to believe they have these things and some of those people will put LLMs into situations where this becomes dangerous. |
|
| ▲ | worldsayshi 3 days ago | parent | prev | next [-] |
| Couldn't this be a flaw in the attention mechanism? Like they need some kind of grounding. An awareness of what they fundamentally should care about and how the thing they are currently giving attention to relates to that? |
| |
| ▲ | Bender 3 days ago | parent | next [-] | | Words like attention, awareness and care do not apply to computers. At least, not yet. Intelligence and sentience are not applicable to servers. They are just machines with logic states. LLM's are just really cool math formulas with big-data fed into them. Big data is not intelligence. It is a massive data-set sorted, filtered down and interpreted by a language model. | | |
| ▲ | esprehn 3 days ago | parent | next [-] | | I assume they meant the Attention process in LLMs, not the human concept of paying attention: https://en.wikipedia.org/wiki/Attention_(machine_learning) | |
| ▲ | slibhb 3 days ago | parent | prev [-] | | LLMs are intelligent by any reasonable standard. Arguing otherwise is like arguing that chess algorithms aren't good at chess when they easily beat the best humans. | | |
| ▲ | Bender 3 days ago | parent | next [-] | | I disagree. LLM's are a language model math formulas that interpret and utilize big-data. Take away the math formulas and we are just back to a massive set of data. Adding to that I would suggest not even the purist forms of data meaning that the data-sets include knowledge from the open and anonymous internet and formulaic tuning from the AI owners and operators. | | |
| ▲ | anon84873628 3 days ago | parent | next [-] | | Your brain is mostly just a Principal Component Analysis calculator. Take away that "math formula" and you don't have intelligence either. The LLM weights are not intelligent. But if you give an agent a mutable memory store and allow it to iterate, it is obviously intelligent. Not massively - it's constrained by the context window - but definitely somewhat. The confusing thing is that their language ability far outpaces their true intelligence, and humans aren't used to that. Normally those things are highly correlated, so it tricks us. | | | |
| ▲ | horacemorace 3 days ago | parent | prev | next [-] | | Right. But at their core they are math formulas devised by a process designed to produce mimicry of task completion. The math formulas themselves aren’t fully effable. We’re sure studying the heck out of how they complete the tasks!
Bet they converged on how we do it, since it’s our language, but who knows. | |
| ▲ | slibhb 3 days ago | parent | prev [-] | | If you want to talk about whether LLMs are intelligent, you have to define intelligence. "They're just math formulae" isn't a definition of intelligence. |
| |
| ▲ | larodi 3 days ago | parent | prev [-] | | Doesn’t take intelligence to beat a human. |
|
| |
| ▲ | lukan 3 days ago | parent | prev [-] | | "Like they need some kind of grounding." A robot body, to really feel the world and get real feedback? We are working on it. Also on automating the whole production pipeline. Right now a "evil" LLM could indeed not do much, but destroy. But once the whole industry is automate, things are different. I don't believe in AI becoming sentinent and taking over the world any time soon, but I do believe most don't see a danger when it would be inconvenient to see a danger. After all, lots of good and bad sci fi stories about exactly this went into their training. |
|
|
| ▲ | chaseadam17 3 days ago | parent | prev | next [-] |
| I'd argue we don't even know what "intelligence" or "self-awareness" mean. Humans are conscious which means we experience things, then we develop preferences for certain experiences, then we develop skills for achieving those preferences. Without consciousness, what is there to be aware of? And why would intelligence emerge and/or what end would it serve? |
| |
| ▲ | anon84873628 3 days ago | parent | next [-] | | Intelligence is the ability to have an internal world model then run simulations on that model to choose an optimal course of action. This is true for humans down to flies. Most of what humans do is still the boring innate stuff; it's just that fancy abstract things like "skydiving" get the most attention. Clearly other animals have "phenomenological experience" i.e. consciousness / qualia without being as intelligent as humans (or necessarily "self aware"). Many people believe consciousness is simply a side effect of intelligence rather than the other way around. | | |
| ▲ | chaseadam17 2 days ago | parent | next [-] | | How would you determine “an optimal course of action” without consciousness? Eg we optimize for survival but without the ability to experience things, why survive? Why would that instinct or any other preference for a course of action emerge? | | |
| ▲ | anon84873628 2 days ago | parent [-] | | Well that is the "hard problem", isn't it :-) But your body is just a robot suit for replicating nucleic acids and you can follow an unbroken chain of those chemical reactions all the way to the first microbes. Which were still "optimizing for survival" without any intelligence beyond following chemical gradients. If you say that means even microbes have some amount of qualia then I can't dispute it. Pure physicalism may necessitate panpsychism of some form. | | |
| ▲ | chaseadam17 2 days ago | parent [-] | | Haha yeah, we're in "who knows" territory. It could just be a foundational truth that all life has an innate unconscious will to survive. However, it makes more sense to me that consciousness precedes the will to survive. And that doesn't mean the way a microbe experiences life is the same as us. E.g. the fact that we have so much control over our environment might mean we have a much more sensitive and reactive way of engaging with consciousness, whereas maybe a microbe is like an enlightened monk, just chillin because it can't do much about what it experiences anyways. |
|
| |
| ▲ | mountainriver 3 days ago | parent | prev [-] | | Intelligence is the ability to compress information. World models are just one aspect of that |
| |
| ▲ | 3 days ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | mountainriver 3 days ago | parent | prev | next [-] |
| I can’t believe anyone still thinks this given their unbelievable ability to write code. Self awareness? Probably not. Intelligence? You would have to be high to think that’s not the case. People are feeling threatened, and rightfully so. LLMs are already insanely intelligent and continue to improve |
|
| ▲ | raffael_de 3 days ago | parent | prev | next [-] |
| Just tried "generate an SVG of a pelican riding a bicycle" for Claude Opus 4.8 Max and of course both legs on same side ... the smartest publicly available model by Anthropic (after Fable) doesn't even successfully simulate understanding the concept of a bicycle. |
| |
| ▲ | mountainriver 3 days ago | parent [-] | | Yet it can write code better than 99% of humans… It’s just starting to be trained on svgs, which is a really hard problem | | |
| ▲ | raffael_de 3 days ago | parent [-] | | "99% of humans" is a low bar. Maybe you mean "99% of people who earn money by developing software"? | | |
| ▲ | WarmWash 3 days ago | parent [-] | | LLMs can't really "see", so I challenge you to draw a pelican on a bike without any visual feedback, just code. Because that is how they are doing it. Vision tokens for transformers aren't really well solved yet, which is why they can smash a phd math problem and trip over a "count the cats on the chair" problem. | | |
| ▲ | raffael_de 2 days ago | parent [-] | | It's not about seeing. It's about identifying the legs of the Pelican and then transferring the concept and mechanics of riding a bicycle + geometry of a body and a bicycle. The entire task has also nothing to do with vision tokens. | | |
| ▲ | mountainriver 21 hours ago | parent | next [-] | | If we want to train a model excessively on SVGs it will obviously be able to do this. We have only just started trying to do that | |
| ▲ | WarmWash 2 days ago | parent | prev [-] | | > It's about identifying the legs So, seeing? | | |
|
|
|
|
|
|
| ▲ | yakz 3 days ago | parent | prev | next [-] |
| We shouldn't want them to have self awareness, we shouldn't be seeking to make self-aware actual slaves. We want machines with perception and knowledge, and that are capable of reasoning. But nothing capable of self-determination. |
|
| ▲ | dinfinity 3 days ago | parent | prev | next [-] |
| > Yet more confirmation LLM's have no concept of concepts or context, no intelligence, no self awareness. No, it isn't. Look at the absolutely trivial code used to simulate war: https://github.com/kennethpayne01/project_kahn_public/blob/m... Having LLMs play nonsense toy simulations like this tells us very, very little about whether they would use nukes in real life war. |
|
| ▲ | doctorpangloss 3 days ago | parent | prev | next [-] |
| If only there was some way you could tell the chatbots what you want them to do... |
|
| ▲ | aaron695 3 days ago | parent | prev [-] |
| [dead] |
