| ▲ | The stochastic parrots have apparently proved the Riemann Hypothesis(scottaaronson.blog) |
| 337 points by stratos123 a day ago | 305 comments |
| |
|
| ▲ | Mikhail_Edoshin 15 hours ago | parent | next [-] |
| There's a book, "Impro", about a theatric studio that practiced improvised sketches. One thing they struggled with was a normal conversation. People have them all the time yet somehow artists could not find the right approach. One day the head of the studio (the book author) had an insight. She asked the actors to drive the conversation so that they always bump the status of themselves or the other party in any direction, but only a little. This did the magic. Not only the actors understood her perfectly, the conversation started to flow and could go endlessly while staying amusing and captivating for spectators. (Look up the book for examples.) After they discovered that secret they started seeing the pattern (status games) everywhere up to the point they asked themselves whether there is anything else in human communication. This was a British studio, the sketches were mostly comedy. |
| |
| ▲ | delichon 12 hours ago | parent | next [-] | | > After they discovered that secret they started seeing the pattern (status games) everywhere up to the point they asked themselves whether there is anything else in human communication. Status truly is your god, whether you would admit so or not.”
https://www.overcomingbias.com/p/status-is-your-god | | | |
| ▲ | dgan 14 hours ago | parent | prev | next [-] | | i have read your comment in full, multiple times, and I still don't understand what you re saying.
> that they always bump the status of themselves or the other party in any direction. what? English non native so that might be the reason | | |
| ▲ | armchairhacker 14 hours ago | parent | next [-] | | Every point they add makes themselves or allies look better (sometimes just more interesting, sometimes more humble) or their enemies look worse. Usually themselves better, at least me; it’s inevitable, even explaining this I notice I’m doing it. The key for the improv actors is doing it with intention. To decide “what is the next thing I should say in this conversation” they think “how do I make my fictional self look better, or my fictional conversational partner look worse, to the audience”? And that gets them to natural-sounding phrases. | | |
| ▲ | pseidemann 8 hours ago | parent [-] | | Is that some fundamental behavior of humans confirmed by psychology? I've never heard of this before and most people I encounter don't do this. Of course there is selfishness and altruism, but it seems this is referring to something else? | | |
| ▲ | coldtea 4 hours ago | parent | next [-] | | >Is that some fundamental behavior of humans confirmed by psychology? It's a fundamental behavior of humans confirmed by history. Psychology and sociology even more so have also studied this, but the behavior is so fundamental that wether a field "confirms it" or not means little. One classic: https://www.amazon.com/Presentation-Everyday-Penguin-Modern-... >most people I encounter don't do this. Most people you encounter 100% do this. | | |
| ▲ | pseidemann an hour ago | parent [-] | | What happens if the communication is about a topic which is not another human? I fail to see how this is supposed to happen so often. I can kind of see this if two strangers meet for the first time, then they need to "synchronize" socially to build trust and understanding. But with friends talking about topic x, making points for others or oneself seems rather distracting or off-topic. | | |
| ▲ | coldtea 5 minutes ago | parent [-] | | >What happens if the communication is about a topic which is not another human? You can still hint at how you know more about the subject, how the other person knows less (or the opposite), and imply countless other things more or less closely associated with that topic or derivative of it that raise/lower one's status. Like you'd be talking about the pollution of the rainforests, and you casually let the others know you've travelled extensively in the Amazon, or how you did your PhD at some prestigious school on a related subject, or how you only support Fair Trade coffee or how you know this or that expert you met at a fancy conference or through an important friend. The possibilities are endless. >But with friends talking about topic x, making points for others or oneself seems rather distracting or off-topic. That's more like an ND view, where what's important is "the topic". In the NT game the discussion is rarely about just the topic, even if it seems so and even if the topic is important, and it's a business meeting or something like that. People can skip around a topic for status reasons even if it's a life or death technical matter, e.g. https://thediplomat.com/2013/07/asiana-airlines-crash-a-cock... |
|
| |
| ▲ | armchairhacker 5 hours ago | parent | prev | next [-] | | I don't think it's fundamental. What is fundamental is that people don't talk without some reason, and that one can interpret anything as a status game. A command, from a military official or boss, is usually not primarily intended to affect status, but to get the thing done. But what is small talk primarily intended to do? | | |
| ▲ | Sharlin 2 hours ago | parent [-] | | > A command, from a military official or boss, is usually not primarily intended to affect status, but to get the thing done No, but the way it's said absolutely is. You can lead by intimidation, and you can lead by inspiration. The former is about trying to raise your own status and/or lower that of your subordinates. The latter is about trying to raise the status of your subordinates. Do the latter right, and it will also improve your own status in their eyes. |
| |
| ▲ | Sharlin 2 hours ago | parent | prev [-] | | Most people don't realize it, or just don't want to admit it to themselves. But yes, most social communication is status games. Not just among humans, among chimps, among smart gregarious birds like corvids, ... |
|
| |
| ▲ | saghm 10 hours ago | parent | prev | next [-] | | I'm a native English speaker and I honestly had the same reaction | |
| ▲ | Mikhail_Edoshin 14 hours ago | parent | prev [-] | | Your comment is an example of bumping my status a bit down. | | |
| ▲ | ffaccount2 9 hours ago | parent | next [-] | | The thing that GP and me (briefly[1]) probably struggled with was the meaning of "status". My first connection was "status" like in "status update", which almost makes sense in the context of conversation. Whereas what you meant is a "social status". [1] See, bumping myself. | |
| ▲ | ShinyLeftPad 12 hours ago | parent | prev | next [-] | | No he dropped his own status ("English non native") | |
| ▲ | a3w 12 hours ago | parent | prev | next [-] | | Social status, not state as in health or other ``facts'', though? I barely comprehend social etiquette on meta level of british people, so it is not just a language problem if the word used was a little too broad. But it could work that way and I rarely noticed, since people are supposed to talk over the weather if starting smalltalk. Not necessarily about themselves. | |
| ▲ | mrheosuper 9 hours ago | parent | prev [-] | | I read it as bumping yours up(as if your use of fancy words is too advance for him to understand). |
|
| |
| ▲ | RhysU 11 hours ago | parent | prev | next [-] | | https://en.wikipedia.org/wiki/Impro:_Improvisation_and_the_T... is an interesting, and at times strange, read. | |
| ▲ | xg15 6 hours ago | parent | prev [-] | | I like how the book author bumped their own status with that anecdote as well... (I'd still like to see some concrete examples here) |
|
|
| ▲ | esjeon 16 hours ago | parent | prev | next [-] |
| Lean + Stochastic predictor = Brute-force theorem solver P.S: AlphaGo was pretty much the same in the spirit btw. |
| |
| ▲ | emp17344 6 hours ago | parent [-] | | People don’t understand just how important Lean and Mathlib are to this whole endeavor, so they mistakenly attribute all these accomplishments to “AI”. | | |
| ▲ | mdnahas 4 hours ago | parent [-] | | I worked on formal proof in 2013 and 2014 and I don’t think they are important. There is value that mathematicians trust Lean’s proofs and trust that Mathlib has correct names to certain definitions, but there are many formal proof systems that could substitute. That trust is valuable, because top mathematicians say that if there is a Lean proof, there is a proof. A few mathematicians reluctantly trusted HOL Light a decade ago, when the Kepler Conjecture was proven using it. That trust in Lean means that mathematicians are facing the truth now, rather than denying it. But, from a technology perspective, Mizar and other systems existed in the 1970s. This accomplishment is all by AI. |
|
|
|
| ▲ | sc077y 11 hours ago | parent | prev | next [-] |
| If this isn't intelligence, then I don't know what is |
| |
| ▲ | agustechbro 4 hours ago | parent | next [-] | | If Intelligence is import it might has to be efficient in achievements.
This does not seems efficient with 250K+ page+millon lines of proofs. - Does intelligence is building a 10 Km brinde with shovel and grabble spending millons tons material and man power? I don't think so. - Does intelligence is building it with a 1/100000 of the material, calculating where to put it so you don't use more than needed? I think so. Both are effective bridges, building them might and might not require intelligence. | |
| ▲ | threatripper 9 hours ago | parent | prev | next [-] | | Maybe we don't need real intelligence or consciousness to be effective. | | |
| ▲ | pitched 13 minutes ago | parent | next [-] | | Consciousness is a trick that our brains play on themselves because it tends to make their humans more successful. Like opposable thumbs, if it didn’t make us more effective at some point in history, it wouldn’t be there. So it must make us more effective in some way. There’s no reason to think that trick couldn’t be reproduced with a system prompt though. | |
| ▲ | pseidemann 7 hours ago | parent | prev [-] | | Nobody knows what intelligence actually is. Psychology and neuroscience have no answer, and especially not computer science. | | |
| ▲ | whattheheckheck 6 hours ago | parent | next [-] | | The most important behavior is doing the right thing at the right time to satisfy the four Fs | |
| ▲ | morpheos137 6 hours ago | parent | prev [-] | | Intelligence like consciousness is a word to describe patterns of processes tending to optimize toward some goal set under the constraints of environment and with sensory feedback. Humans goal is to reproduce. Llms goal is to predict best next token. Human feedback is sensory mediated. Llm feedback is prompting on outputs including from tools such as lean. If RH was provable it is not surprising either such system could prove it in principle. If a sound proof required an intricate path or vast contextual memory to assemble it is not surprising a machine proved it before a human. However I don't think this is generalizable. I don't think intelligence is a scalar in that the proof shows the machine is more intelligent than the human. Rather it is qualitative. The machine was structured so that the path was attainable to it. This is intelligence in the particular case but not general intelligence if such a thing exists. Now what would point more toward general intelligence is a machine that can compress proofs or truths better than humans. If we start seeing short proofs of FLT or RH then that will be interesting. I am skeptical with the current technology. The tendency of LLMs seems to be to explode complexity not condense things to essential elements or minimal description lengths. |
|
| |
| ▲ | unrented7977 3 hours ago | parent | prev [-] | | rand() is almost certainly not intelligent. Is a trillion rand() calls intelligence? |
|
|
| ▲ | hoppp a day ago | parent | prev | next [-] |
| Thats a lot of lean. Can anyone comprehend how it works in it's entirety? |
| |
| ▲ | rcxdude a day ago | parent | next [-] | | There's a top level sketch of the proof in the paper from the guy who's claiming the proof. It seems like he feels he understands it, at least. | |
| ▲ | abm53 a day ago | parent | prev | next [-] | | In one sense, it doesn’t matter, as long as we comprehend enough to be sure the lean proof encodes the hypothesis and all axioms correctly. But in another sense, such as whether it adds to our understanding, of course it does matter. | | |
| ▲ | glial a day ago | parent [-] | | > be sure the lean proof encodes the hypothesis and all axioms correctly This is not my field, but surely this has already been done before today? | | |
| ▲ | kadoban a day ago | parent [-] | | My understanding is that there is no central, agreed, library of lean formulations of hypothese+axioms for big open problems. So it's not like, when proving one of these you necessarily just take the lean-encoded problem off the shelf and add the "i proved it" parts. First step seems to be you encode the problem according to your whims. So there's some checking to be done in terms of "does this actually encode the problem?" For pro mathematicians I wouldn't expect that that's where they run into trouble in things like this, but still needs a look-over. | | |
| ▲ | raincole 11 hours ago | parent | next [-] | | Claude's proof just uses Mathlib's formalization of Riemann's Hypothesis. It's the closest thing you can find to an "agreed" one. It's not like that Claude tried to formalize it from human language from scratch. | |
| ▲ | twothreeone 21 hours ago | parent | prev [-] | | Often (most?) of the time "solving" a hard problem is partly taking it apart into separate pieces and phrasing those pieces in various novel ways that allow one to tackle them individually and then at the end it all magically falls into place (in this case after 900 pages). |
|
|
| |
| ▲ | epolanski 8 hours ago | parent | prev | next [-] | | It's quite interesting that we're heading very quickly into accepting we cannot comprehend ourselves the work that AI can prove to be correct. | |
| ▲ | jordemort a day ago | parent | prev [-] | | at what point does this become functionally equivalent to coming down from a mountain with a couple of stone tablets and saying "God says"? | | |
| ▲ | noosphr a day ago | parent | next [-] | | If we can trust payroll to a database instead of insisting someone do all the calculations by hand there is no reason to expect other forms of symbolic manipulation to be vastly different. | | |
| ▲ | wpietri a day ago | parent | next [-] | | That's actually a great example of something where we don't and shouldn't trust computers. E.g.: https://www.kqed.org/news/12051862/sf-teachers-are-yet-again... We can and should trust computers for the straightforward, boring parts. But there's a huge gulf between the people who think, "how hard could payroll be?" and actually making a system that handles all the interacting complexity of real-world problems. | | |
| ▲ | wvenable 20 hours ago | parent | next [-] | | All the problems with that payroll system are caused by humans, not computers. The odds that computer is making a mistake in a calculation is near zero (although not impossible). Instead it's doing exactly what it is told to do. | | |
| ▲ | wpietri 19 hours ago | parent | next [-] | | All the problems with computers are caused by humans, not computers. | |
| ▲ | aorloff 20 hours ago | parent | prev [-] | | I guess you never heard of the pentium bug ? Edit : and more generally - math has infinite resolution, whereas computers have a finite resolution, and cannot be more precise than that | | |
| ▲ | fluidcruft 2 hours ago | parent | next [-] | | Humans doing payroll by hand are not using infinite resolution. And to the extent they are there are always big math, rational and decimal math systems. | |
| ▲ | zamadatix 18 hours ago | parent | prev | next [-] | | Computers have no more or less a finite resolution problem than humans - it just seems to be far larger than what is practically needed in both cases. I.e. problems can be encoded symbically or numerically and the maximum size of each (including the numerical precision of the latter) is bound by physical constraints. But just because computers use fixed sized numerical types does not mean they are unable to handle working with exact symbolic values or objects we use in math. Regarding the Pentium bug, I think one has to agree when it stopped being up to the human introduction of the bug. I.e. sure, many think we have designed faulty CPUs before. Some feel computers have designed faulty computers, others feel it still comes back to the human definitions. At some point though, most seem to call human mistakes human mistakes rather than where they are derived from prior to humans. I just don’t think we’ve all reached agreement of when that boundary definition is is for mistakes from the computer. | |
| ▲ | hardbass 6 hours ago | parent | prev | next [-] | | >computers have a finite resolution, and cannot be more precise than that Yes and neither can us. We have never written infinite digits down of anything.
We work in finite sets of fields. The field of Q(pi) perfectly represents various expressions of pi for example. | | |
| ▲ | aorloff 3 hours ago | parent [-] | | You can just write Pi, which is precise, in math. | | |
| ▲ | hardbass 2 hours ago | parent [-] | | Yes pi, 4 atan(1), the ramanujan formula, a spigot program are all valid representatives for the same entity. |
|
| |
| ▲ | noosphr 16 hours ago | parent | prev | next [-] | | The numbers you can talk about in mathematics are just as countable as the numbers you can talk about in computer science: https://en.wikipedia.org/wiki/Definable_real_number This should be pretty clear unless you've never used arbitrary precision arithmetic. | | |
| ▲ | aorloff 42 minutes ago | parent [-] | | Computer science is part of math. We are talking about math versus computing |
| |
| ▲ | wvenable 18 hours ago | parent | prev [-] | | > I guess you never heard of the pentium bug ? I was going to put that exact example in brackets after "although not impossible" but decided it was unnecessary. |
|
| |
| ▲ | fluidcruft 2 hours ago | parent | prev [-] | | That a payroll system isn't implemented in Lean though |
| |
| ▲ | someothherguyy a day ago | parent | prev | next [-] | | calculations and operations are transparent to a database, and they very easy for a human to understand and verify | | |
| ▲ | noosphr a day ago | parent [-] | | So is natural deduction when you treat quoted expressions as first class citizens. The difficulty of doing proofs in mathematics is largely due to the fact that mathematicians refused to treat symbolic expressions as data and had to invent ever more complex paradigms to deal with them. Lean being the final iteration of that trend. When you add quotation and quasi quotation on top of Polish notation proofs become as mechanical as long division. | | |
| ▲ | cowlevel 12 hours ago | parent | next [-] | | idk what you're talking about. mathematicians treat symbolic expressions as data all the time. They apply a procedure to one expression to get another expression. | |
| ▲ | mypalmike a day ago | parent | prev | next [-] | | Can you explain this further? Can you point the interested reader at books or papers that would help understand the approach you are advocating? | | |
| ▲ | noosphr a day ago | parent [-] | | If I ever get the time writing a book called the structure and interpretation of mathematical logic is on my to do list. There are unfortunately no books on the topic currently, because very few people take syntax seriously enough. This is rather ironic as computer science is only possible because someone took syntax seriously enough to invent place value notation. |
| |
| ▲ | catlifeonmars a day ago | parent | prev | next [-] | | A LISP for symbol manipulation? I definitely don’t hate it. | |
| ▲ | someothherguyy a day ago | parent | prev | next [-] | | i don't think mathematicians have avoided proof assistants, being that they have been around for over 50 years (https://en.wikipedia.org/wiki/Automath). however, as to my point, i would wager most payroll systems don't express a single verification as 400kloc of SQL statements. https://principia-math.com/projects/d16ab288ed353bc5b10e292a... | | |
| ▲ | noosphr a day ago | parent [-] | | There are some mathematics who don't mind formal methods, but they are few and far between. A lot of them are closer to computer scientists than traditional mathematicians. |
| |
| ▲ | sand65469 3 hours ago | parent | prev [-] | | or you know, you just use horn clauses so you dont have to staple ad hoc crap like quotation and quasiquotation on top of the language. |
|
| |
| ▲ | saghm 9 hours ago | parent | prev | next [-] | | If I don't get my paycheck, I immediately notice and can complain, and a human will be able to rectify whatever issue caused it. If I literally never got a paycheck again, we have decidedly non-mathematical ways of dealing with it (a legal system). I don't see the comparison here. Detecting issues in a 17 million line proof is not remotely trivial, if we accidentally build a bunch of math on a failed proof, we don't have a retroactive way of getting all of that time and effort back. I'm not saying that automated proofs like this aren't worthwhile, but the parent comment did say "functionally equivalent". The level of trust that's needed in the people who used the system to produce the output is a pretty clear functional difference in my mind between payroll and huge generated proofs. I don't think I need as much trust in someone who produces an automated proof as someone who claims that god told them something directly (since that's a lot harder to believe in general), but I also don't have a particularly large amount of trust in the average employee of Anthropic making a bold claim either. There have always been a large number of people in the world (and still are!) that believe any number of things on the basis on faith rather than evidence, and it's not crazy to draw a parallel between "that religious authority says so" and "that Anthropic employee says so"; for the overwhelming majority of people in the world, the claims are essentially both impossible to falsify, so at the end of the day we are still relying on trust (whether from that authority directly or from a consensus of other people we trust). | |
| ▲ | Lambdanaut a day ago | parent | prev | next [-] | | Except databases are understood deterministically. The route an LLM will take, given an input, is not. | | |
| ▲ | overboard2 a day ago | parent | next [-] | | That's why LLMs are often made to generate proofs in Lean. | |
| ▲ | noosphr a day ago | parent | prev [-] | | The post was about lean, not how the lean code was generated. |
| |
| ▲ | billyp-rva a day ago | parent | prev [-] | | Everyone gets a line-by-line paystub every period that fits on, what, half a page? That's a far cry from 17 million lines of code. | | |
| ▲ | noosphr a day ago | parent | next [-] | | Depending on the company 17 million lines of expenses per month are either normal or a fraction of the total. | |
| ▲ | awb 16 hours ago | parent | prev [-] | | Generating a paystub isn’t a Millennium Prize Problem. Everyone has limits to their domain knowledge that makes understanding expert material challenging. I’m guessing that at the pinnacle of any field, there might only be a handful of people who fully understand the solutions to the most complex problems. |
|
| |
| ▲ | Lerc a day ago | parent | prev | next [-] | | It does not. If after careful analysis people conclude that they cannot tell if it says what it claims to say, then then it can be considered not-a-proof until someone can demonstrate in a convincing manner that it is what it claims to be. I can say god exists because 5>3. No matter how valid my claim is that five is in-fact greater than three, if I can't convince people of the connection it's a valid proof of something, just not what I say it is. It might be true that if the Riemann hypothesis is false, leaves cannot fall to the ground. That would mean a falling leaf demonstrates the truth of the hypothesis. Without showing people why it must be the case in a way they understand it falls short of a genuine proof. That would make proofs subjective, and I think that is true. To many people Pythagoras has not been proved, they just believe (correctly) there is a consensus amongst those who do understand the proofs. | |
| ▲ | interestingsir a day ago | parent | prev | next [-] | | For the rest of us it already is like that.. | |
| ▲ | MarcScott 15 hours ago | parent | prev | next [-] | | For a lot of people there is no difference. I'm fairly well versed in physics for instance, but I quickly lose my ability to deeply understand what's going on when quantum mechanics is introduced. I can just about grasp the analogies, but that's the limit of my understanding. So the stone tablets have become YouTube videos and God has become the likes of Feynman. | |
| ▲ | bawolff 18 hours ago | parent | prev | next [-] | | The point when we stop being able to dicipher it. We are nowhere near that. Yes its getting harder, but i'm certain that within a year people will have this disciphered and translated into traditional math prose. Keep in mind deciphering and validating traditional proofs is also quite hard. e.g. It took a long time to figure out if the proof for fermat's last theorem was valid | |
| ▲ | atmosx 11 hours ago | parent | prev | next [-] | | I have heard that analogy in Greek and Italian already by a philosopher and a geopolitician, so it’s not far… | |
| ▲ | twoodfin a day ago | parent | prev | next [-] | | It matters a great deal in several fundamental ways that Moses didn’t also have a valid Lean formalization. | |
| ▲ | epolanski 8 hours ago | parent | prev | next [-] | | Well, technically it is more "God says and proves...". | |
| ▲ | Jyaif a day ago | parent | prev | next [-] | | It doesn't really work because we are god, and the AIs are our creations. | |
| ▲ | CamperBob2 19 hours ago | parent | prev | next [-] | | There was a great Twilight Zone original-series episode that was basically just that: https://en.wikipedia.org/wiki/The_Old_Man_in_the_Cave Spoiler: it didn't end well for the clanker or the humans. | |
| ▲ | idiotsecant a day ago | parent | prev [-] | | You not knowing how to interpret the results is not the same as nobody knowing how to interpret the results. Formal proofs with very dense lean implementations have been a thing for a long time, it wasn't 'god on the mountain' then, either. | | |
| ▲ | jordemort 19 hours ago | parent [-] | | I don’t think it is possible for any human to actually understand 17.9 million lines of code. I don’t think that things humans can’t understand meaningfully advance human knowledge. Scientific progress is built on a foundation made of previous discoveries. What is there to build on here? An incomprehensible soup and a note from the computer that says “trust me”? | | |
| ▲ | idiotsecant 7 hours ago | parent [-] | | Lean is exactly the kind of thing a team of people can go through and miticulously understand 17 million lines of code. |
|
|
|
|
|
| ▲ | alterom a day ago | parent | prev | next [-] |
| "We should still fix the leak" I think this approach and conclusion applies to a lot more than the immediate context of that conversation. |
| |
| ▲ | kirrent 20 hours ago | parent | next [-] | | Reminds me of the following from Feynman: "I returned to civilization shortly after [the Trinity test] and went to Cornell to teach, and my first impression was a very strange one. I can't understand it anymore, but I felt very strongly then. I sat in a restaurant in New York, for example, and I looked out at the buildings and I began to think, you know, about how much the radius of the Hiroshima bomb damage was and so forth ... How far from here was 34th St? ... All those buildings, all smashed - and so on. And I would go along and I would see people building a bridge, or they'd be making a new road, and I thought, they're crazy, they just don't understand, they don't understand. Why are they making new things? It's so useless. But, fortunately, it's been useless for about 30 years now, isn't it? So I've been wrong for 30 years about it being useless making bridges and I'm glad that those other people had the sense to go ahead." | | | |
| ▲ | vibebingo 15 hours ago | parent | prev | next [-] | | Almost Tao de Jing stuff | |
| ▲ | johnea 18 hours ago | parent | prev | next [-] | | Thanks for quoting that! I felt the same way, and I think it was a great way to encourage introspection at the end of the article. | |
| ▲ | js8 a day ago | parent | prev | next [-] | | Well, a common advice is to live your life as if every day is your last, which means, DON'T bother to fix the leak. :-) | | |
| ▲ | saulpw a day ago | parent [-] | | Plan as though you're going to live forever, live as though you're going to die tomorrow. This means that you fix the leak but treat it like a precious and divine activity that you would be honored to have as your last. |
| |
| ▲ | DonsDiscountGas 21 hours ago | parent | prev [-] | | Meanwhile I'm over here thinking about expensive home repairs I'm having trouble getting handled. Where's my robot repair man? |
|
|
| ▲ | janalsncm a day ago | parent | prev | next [-] |
| I think when people set out to create a chess playing computer they thought we’d end up with something that we could deeply understand at an intuitive level. Instead, when you look at the Stockfish evaluation of a position, you do get an “explanation” but it’s really just the aggregate of a bunch of calculations, which are themselves stacked on top of the “stochastic parrot” NNUE evaluator. This leaves us in a very unsatisfying position as creatures who want answers to “why” questions. Intuitive answers are incomplete, and complete answers are either fundamentally impossible or completely incomprehensible or both. Feynman said something similar with regards to magnetism, but I feel that understanding a chess position is an even better example. Magnetism depends on facts about the universe which at their most basic level “just are”, whereas even a complex chess position is theoretically entirely explainable. |
| |
| ▲ | azakai 18 hours ago | parent | next [-] | | There is actually work on understanding what something like Stockfish's neural network is doing: https://www.pnas.org/doi/10.1073/pnas.2406675122 We can actually see some of the calculations and even learn new chess tactics from these networks. They don't need to be opaque black boxes! | |
| ▲ | thisisauserid 21 hours ago | parent | prev | next [-] | | The exact Feynman quote is: "When you explain a 'why', you have to be in some framework in which you allow something to be true." https://www.youtube.com/watch?v=MO0r930Sn_8 | | |
| ▲ | loose-cannon 20 hours ago | parent [-] | | I hadn't seen that clip in a long time. Maybe the younger version of me was impressed by it. I understand the spirit of what he's saying. But that clip makes him seem extremely unpleasant. Presenting bits and pieces of the framework is how everybody learns. If somebody slips on ice, I'm not necessarily concerned with why ice is slippery, even though it's a logical part of the reason. There's plenty of intelligent people who can answer a question like that without devolving into a lecture. | | |
| ▲ | 0xWTF 17 hours ago | parent | next [-] | | My undergrad is in physics and I was the electrical officer on my first ship. There were plenty of exotic things about that plant that stupified me, but it seemed like no one else was interested in those things. All the other junior officers were busy quizzing each other on the number of cylinders in the diesel engines that drove the generators, what spaces the generators were in, shit that didn't really seem to matter to me. One afternoon underway I was standing in Central Control and the main propulsion assistant, more senior to me and studying for her engineering boards, started quizzing me about the electrical plant. And I would explain things. And she would ask why. So I would explain and she would ask why. And so on. Until we got to fundamental wave equation for electromagnetism. And she kept asking why. And at one point I realized we were at the bottom. There's nothing really below electromagnetism. It just is. And I said that. "It's one of the 4 fundamental forces. It's just a mathematical description of the way the world seems to work, and it's proven to be correct, entirely, in all cases. There other forces at different energy levels, but in the world that affects you and me, it just is. It, it just is." And at that point I realized half the engineering department was peeking in from around the consoles. And she starting crying. And I felt so bad. Like I had said something wrong, and I still don't understand what I should have done better. That was almost 30 years ago. So Feynman's observation that you have to be in some framework that you allow something to be true strikes me as incredibly helpful. I think the MPA must not have had that kind of framework and found herself distressed when she kept reaching for things and never getting something more to hold on to. Maybe I should have had the insight to offer her more of a lifeline. | | |
| ▲ | loose-cannon 7 hours ago | parent | next [-] | | No, I agree. What he's saying is technically correct. There has to be a certain framework in order to fully justify certain beliefs. My point is that the answer you give depends on who you're talking to. It's okay to present a simplification or an incomplete picture. That's how everybody learns. Add a disclaimer and move on. There's a time and a place for technical correctness and, unless the interviewer was being critical/annoying prior to that recording, I would say the frustration is misplaced. There's no need to get frustrated at the interviewer because you can't meet your own standards of rigor. | |
| ▲ | rowanG077 6 hours ago | parent | prev [-] | | Why did she start crying? I don't understand why this would be emotional for her. |
| |
| ▲ | mjburgess 19 hours ago | parent | prev | next [-] | | Some people i've shown this to (in admiration for Feynman) have had the same reaction, but I just cannot see it. I'd be interested to know why you think he's unpleasant. Are you a physicist? I wonder if non-physicists just misread his frustration. The question isnt about some macroscopic physics problem that decomposes clearly into something satisfying. Its a question about one of the most basic constituents of reality. A "why?" question about something so basic has no answer other than "just because". Insofar as anything can be said, it will be so abstract and technical that it wont satisfyingly address the question. It is a little like a lay person asking a mathematician: why is 0*1 = 0, but 0+1=1 ? Except, somewhat more extreme than that, because such basic things as 'magnetism' are introduced into all accessible physics formalisms as assumptions. | | |
| ▲ | loose-cannon 18 hours ago | parent [-] | | Because he projects a certain kind of frustration that's not warranted in that moment. He's being asked a technical question without any means to give a rigorous answer. I appreciate rigorous thinking, but the interviewer doesn't necessarily want a perfectly rigorous answer, nor does the average person. He's not responding to a need, he's projecting his own perfectionism. The interviewer most likely would not have interrogated him further about the underlying properties of ice (or engaged in that entire fictitious interaction). Feynman clearly understands that there are several levels to understanding. So just give an answer that you feel is appropriate, add a disclaimer ("this answer is hand wavy"), and move on. Why is that so hard? | | |
| ▲ | mjburgess 18 hours ago | parent [-] | | The answer is, "I cannot explain that force to you. It is just, as a matter of fact, one of the fundamental forces of nature." He's taking it as read that the reason he is giving his extended answer is to explain why he cannot answer at all. What he could have said to a physics student is that: wave-functions of charged particles have a property, "phase". The problem is that point values of phase cannot be observed, and if you formulate the laws of physics with raw "phase" values (like a measure of mass, at 5kg), they dont work. So something must exist to "track" phase, so that every pair of measurements "cancel out" the effect of this property. That something is the EM field. Magentism is a spatial aspect of this field. Eletricity is a space-time aspect of this field. So the magnetic force arises out of a need to "conserve phase", and that phase is how a pushing force is experienced: two particles experiencing a magentic interaction take on different phases (their wavefunctions interfere such that..). This might "feel" like an explanation to the physics student, because they will be willing to make the assumptions needed, and they will find conservation-style explanations sufficiently satisfying. ie., they can already see a wavefn description will have a free phase term. But a lay person wouldnt feel satisfied because more unknowns have been introduced, and conserving "another unknown" is no advancement in understanding. That's the gulf which is frustrating feynman. There's nothign to be said to the interviewer other than the force has no explanation in terms of anything he's familiar with. The question is too basic. And feynman is not going to go thru a formulation of QM in terms of wavefns in 5min -- it would be an entire lecture course. |
|
| |
| ▲ | cryptonector 16 hours ago | parent | prev | next [-] | | We might have to watch the whole uncut recording to understand. Perhaps Feynman thought he was talking to the interviewer rather than to you and I and the whole public. But I have learned something very useful from this non-explanation explanation. Sometimes you have to assume stuff, and sometimes you have to dumb it down. For example, when your five year-old asks you how magnets work, you're just not going to break out Maxwell's equations. Instead you'll explain how everything is made of very tiny things called atoms, and that all atoms are like tiny magnets, but some kinds of atoms can be made to line up and then all the tiny magnet forces add up to a big magnet force. The kid'll eat it up. I know because that's roughly what my father said to me way back when. And so you'll have different explanations for different people, up to the level of your understanding. The problem here is that not only are we missing context in that clip (had the interviewer been annoying?), but Feynman barely disguises that he can be unpleasant. He has to prep himself to tell the guy that he's just not a physicist. He has to chastise himself for being so good at this that people expect him to be able to explain anything and everything even though it's not always possible. He's kind of an ass here. But what a great answer he gives! His visible annoyance is almost necessary. | |
| ▲ | the_sleaze_ 19 hours ago | parent | prev | next [-] | | I wonder if feynmann felt like he was being used, like a circus animal. Hey Joel Robuchon how do you make the perfect scrambled eggs? Hey Einstein what's the square root of 34224452 - look at him he's just like a computer! | |
| ▲ | thisisauserid 19 hours ago | parent | prev | next [-] | | Maybe he was unpleasant due to all of the tumors he was full of at the time. | |
| ▲ | thefringthing 19 hours ago | parent | prev [-] | | [flagged] | | |
|
| |
| ▲ | TZubiri 20 hours ago | parent | prev | next [-] | | I imagine the machine proving that PPPRK beats PNBK in most configurations, but it not being understandable by any human and pretty hard to disprove except for machine. doesn’t seem useful to humans, only to other machines | |
| ▲ | charcircuit 21 hours ago | parent | prev | next [-] | | You can ask AI to explain things to you. All of the time I have it explain and defend the code it is generating. | | | |
| ▲ | DragonStrength 17 hours ago | parent | prev [-] | | Damn, almost like those existentialist philosophers of yesteryear were onto something in their critique of rationalists. |
|
|
| ▲ | elorant a day ago | parent | prev | next [-] |
| Wonder how many mathematicians could reach similar breakthroughs if they had near infinite compute. |
| |
| ▲ | henry2023 a day ago | parent | next [-] | | The first working transistor was invented 79 years ago, I bet we’ll consider today’s total compute trivial in another 80 years. | | |
| ▲ | smithmayowa 14 hours ago | parent | next [-] | | Funny enough this put things into perspective for me, it's like when breakthroughs happened after the first transistor based computers, yet for decades after those, we had nothing and this leads me to believe, that we might settle into problems that are still quite hard for even llms, and not everyone can even produce stuffs like this even if they had the same compute that alpoge had. | |
| ▲ | lossolo2 a day ago | parent | prev | next [-] | | Aren't we approaching the limits of physics, with increasingly diminishing returns each generation? I remember reading an article about quantum effects at these scales, which make everything much harder. I mean, for example, even if we double the speed at which we can travel every year, you will never get to Alpha Centauri in your lifetime | | |
| ▲ | the8472 a day ago | parent | next [-] | | Chips are only 2D, and we're far from the landauer limit, and the landauer limit only applies to non-reversible computations anyway.
If we could assemble matter atom by atom at scale we could get to some dirty diamond that fits in your hand, outclasses the world's top supercomputers and runs off 10W of sunlight. | |
| ▲ | thombat a day ago | parent | prev | next [-] | | The overall point is sound, but in fact if we could double our speed every year then in just 23 years we'd get from 100km/h to the speed of light | | |
| ▲ | ignoramous a day ago | parent [-] | | Keeping with the Interstellar framing, the key is not only in faster ships (lightspeed) but in shrinking distance (warping space-time). For LLMs, that'd mean efficient model architectures on efficient chips. |
| |
| ▲ | GoblinSlayer 8 hours ago | parent | prev | next [-] | | That's the Electron factor. Unix worked on PDP-7 with 18kb of RAM. | |
| ▲ | dist-epoch a day ago | parent | prev [-] | | We are approaching the limits in a single chip. But we can add more chips, millions of them, billions of them. Gigawatts of compute. | | |
| ▲ | twoodfin a day ago | parent [-] | | Right. I’m sure at some point in the past more capital was spent on designing better airplanes than on building them & the infrastructure to support them. We still spend money on designing better airplanes, but the two lines on that graph crossed long ago. |
|
| |
| ▲ | RAWGOD a day ago | parent | prev [-] | | >I bet we’ll consider today’s total compute trivial in another 80 years. Past performance is not indicative of future results. Airliners are still subsonic after a 100 years of aviation. | | |
| ▲ | betterunix2 a day ago | parent | next [-] | | > Airliners are still subsonic after a 100 years of aviation That is a deliberate decision motivated primarily by economics and secondarily by environmental concerns. In fact airliners today are somewhat slower than they were a few decades ago because the lower speeds are more fuel efficient. We certainly could build fleets of supersonic airliners if it made sense to do so, it is not as if we have no idea how to do it and in fact we built supersonic passenger jets once upon a time. | | |
| ▲ | recursive a day ago | parent [-] | | Yes and? we can make deliberate decisions motivated by economics regarding computation as well. |
| |
| ▲ | Maxatar a day ago | parent | prev | next [-] | | Your example contradicts your point. Airliners are still subsonic after an extended period of time, and hence based on that past performance it's a good indicator (although not definitive) that they will remain subsonic for an extended period into the future. In general, past performance is actually indicative of future results. It's not definitive, and some phenomenon are random, but the trajectory of technology over time is not some random fluke that just has coincidentally trended by random chance. | | |
| ▲ | RAWGOD a day ago | parent [-] | | >Your example contradicts your point. Airliners are still subsonic after an extended period of time, and hence based on that past performance it's a good indicator (although not definitive) that they will remain subsonic for an extended period into the future. There was a period in aviation development when it was widely assumed that speed will continue to rise. Just like in the development of CPUs there was such a period, quite recently. |
| |
| ▲ | szvsw a day ago | parent | prev | next [-] | | But along other axes they have also improved tremendously, no? Ie rather than just time to destination, they have greatly improved efficiency (energy spent per unit of work) and throughput/parallelism (ie simultaneous flights in the air) right? | | |
| ▲ | RAWGOD a day ago | parent | next [-] | | Yes, but these axes are also not showing an unlimited growth. There is no expectation that either speed, fuel efficiency, ticket price, or total number of flights will improve by an order of magnitude in the foreseeable future, unless there's a technological breakthrough (Nuclear turbines? Artificial photosynthesis making kerosene free? 3D printing the entire plane? These are currently all sci-fi ideas). | |
| ▲ | cyanydeez a day ago | parent | prev [-] | | but it's still just people in a tube. Sure you're going to get a lot of math problems solved, but food distribution is still going to be stuck by warlords and oligarchies. Think of it like highway designers and the "one more lane" gridlock delusion. In this case, it's more will smith eating speghetti than cheap, low cost cancer cures. |
| |
| ▲ | jcelerier 18 hours ago | parent | prev | next [-] | | a single company was able to send more satellites to space in the last 4-5 years than all the things sent combined, across all countries since we started putting stuff into space in 1957. Saying "Airliners are still subsonic" is like saying that cars still are still not going much past 120/130 km/h thus we're technologically stuck, in a world where jet engines exist. | |
| ▲ | krferriter a day ago | parent | prev | next [-] | | Passenger airliner speed is not the only scalable metric relevant to aviation. | |
| ▲ | vibebingo 15 hours ago | parent | prev | next [-] | | Chips and airframes are different engineering challenges. | |
| ▲ | bartread a day ago | parent | prev | next [-] | | > Airliners are still subsonic after a 100 years of aviation. Yeah, but that's a choice made for very good reasons: - Noise reduction - Fuel efficiency - Reduced pollution and lower environment impact - Reduced cost of materials, equipment, and maintenance - Passenger affordability It's not that we can't make supersonic airliners or that the fact we don’t illustrates a lack of progress or performance: it's that it doesn't make economic or environmental sense to. Gulfstream and Sukhoi even had a partnership to develop a supersonic business jet (which, if you look at the mockups, might be one of the coolest looking planes ever conceived) but they gave up after spending a fairly enormous amount of money already because it was going to cost an absolute fortune to bring to market... and there was no real market for it. EDIT: the responses I’m getting are a bit weird. People are putting words in my mouth but all I was actually doing was pointing out the the example used wasn’t a great illustration of the point the person I was responding to was making. This isn’t Reddit so maybe we could reign it in a bit, please? | | |
| ▲ | rafaelmn a day ago | parent | next [-] | | That's kinda the point I think ? It's not like we can't turn half of global energy production or GDP to compute - it's more about does it make sense to. As much as I'm getting value out of this technology daily - the level of investment made to get this far is astronomical compared to the value I'm seeing created, even less captured by these companies. And because the progress is so good in some domains it's easy to believe that these things are magically reaching some intelligence generalization threshold. But then you leave the training distribution and you get disturbed by how poor the performance is and how obviously equivalent things don't automatically transfer at all. Given the recent market trends I would not be surprised we see a correction in level of compute investment going forward. | | |
| ▲ | FloorEgg a day ago | parent | next [-] | | The universe seems to be set up such that it wants entropy to increase. Any living thing that doesn't profit from converting low entropy energy to high entropy energy dies. So the question is whether more intelligence will lead to faster increase in entropy. My guess is it will. Whatever recent market trends are, they are irrelevant. Whether the AI can or can't do x thing is irrelevant. What matters is whether the companies that make AI profit from making AI, and if you look closely at what's going on it sure seems like they do. If you think otherwise it may be because you're reading headlines and propaganda articles and not digging into the fundamentals. So maybe we have a temporary correction/slow down, because these companies took on too much leverage and grew artificially too fast. That's very possible. In the big picture long term it will be a blip. The universe seems biased to increasing the volume of intelligence and I have never heard a compelling case for why this wouldn't continue for a long long time. | |
| ▲ | pixl97 a day ago | parent | prev [-] | | >It's not like we can't turn half of global energy production or GDP to compute That depends which part of 'we' is running the economy. Just imagine an economy that makes mass massive amounts of money but doesn't align with human needs. |
| |
| ▲ | bayindirh a day ago | parent | prev [-] | | I can argue all of the reasons are valid for keeping compute at a certain speed range, or power consumption envelope. It's not we can't make compute much more denser, there's a new standard informally called as megawatt racks. But deploying them at gigantic scales bring in massive problems as well, so we can keep total energy envelope at a fixed range. ...for the sake of the reasons you outlined. |
| |
| ▲ | ThrowawayTestr a day ago | parent | prev [-] | | We can build supersonic passenger planes, no one wants them. | | |
| ▲ | twoodfin a day ago | parent | next [-] | | We could build gallium-arsenide chips, same problem. Instead we just build more of the more economical thing. | |
| ▲ | kibwen a day ago | parent | prev [-] | | Rather, nobody wants them at the price point necessary to make them economically viable. In the context of tech, the cost to develop a next-gen fab doubles every four years; absent some major breakthrough, we have about 20 years before even the most advanced economies balk at spending $1T, $2T, $4T, $8T on a single fab, causing hardware to settle down like airline speeds have. |
|
|
| |
| ▲ | fantasizr a day ago | parent | prev | next [-] | | the best computing power used to be located at the universities and national laboratories. Now you need access - employee or otherwise - to a nasdaq listed startup. Hard not to think it's the shareholders who benefit in the short term in this new alignment. | |
| ▲ | contubernio a day ago | parent | prev | next [-] | | I could. I mean that. But I work in Spain and have a run if the mill desktop computer and pay for ai out of pocket. | | | |
| ▲ | hyperhello a day ago | parent | prev [-] | | Maybe we should ask how many compute installations could reach similar breakthroughs if they had a mathematician. |
|
|
| ▲ | hansvm a day ago | parent | prev | next [-] |
| Derailing slightly, a variety of HN posts (the blogs/sites they reference) have started banning me with a 403 page from nginx, including this author. My IP reputation is supposedly fine. Does anyone have any ideas where I should look next? |
| |
| ▲ | hansvm a day ago | parent | next [-] | | Responding to other comments here: - Mobile internet (IP 172.56.47.136) - The haveibeenproxied link indicates my IP is shared (expected) - This is stock android/chrome with a DNS blocklist of advertising domains and some invasive Samsung the things I can't fix on the device (I doubt that's the culprit given that this is a new behavior I'm observing in the outside world after 8yrs with those settings and also because it's be strange for nginx to be the thing to immediately report that trackers have to load for me to get a glimpse of the main page, especially as something I'm seeing across multiple sites, but that's not impossible). - No VPN - I'm a "normie" internet user. Anything remotely questionable to even the cheapest and least aligned of LLM judges and executioners happens on non-home networks. - I see it with other devices on the same network (Chrome, Linux, MacOS, Safari, Qutebrowser, etc), which suggests to me that somehow my IP or some other shared network property is the issue. I'm curious mostly about the commonality in separate nginx installations returning a 403. That suggests there exists a common blocklist or whatnot I can inspect and fix, either because of my IP or because something in the loop is forcing ipv6-only or who knows what. | | |
| ▲ | microcode a day ago | parent | next [-] | | That IP shows up as an exit node in 3 separate proxy networks, one of them a malware pool. The IP really is relaying proxy traffic, just not yours. It's unfortunate, but not much you can do. One of the many reasons I hate CGNAT. Disclosure: I work for Spur, the company behind HaveIBeenProxied. | | |
| ▲ | solyin 21 hours ago | parent [-] | | What's their business model? | | |
| ▲ | lgats 20 hours ago | parent [-] | | I suspect they will sell the data to firewall companies like CloudFlare or network security companies like Cisco so they can secure their own customers that start sending out suspicious traffic. Alternatively, they could help insurers adjust cybersecurity insurance when the insuree's network sends lots of suspicious traffic. | | |
|
| |
| ▲ | dannyw 19 hours ago | parent | prev [-] | | Along the motif of "LLM is ruining everything", someone on your CGNAT is probably running scrapers for datasets. Maybe even some mobile phone farm. | | |
| ▲ | everybodyknows 7 hours ago | parent | next [-] | | > mobile phone farm I've been getting by at home on a T-mobile hotspot, and haveibeenproxied now reports Kookeey and Shifter residential proxies for my IPv4 shared with God knows how many other customers. Might moving to a T-mo 5G home box help? Guessing that an IPv4 would still be shared with other people's TVs ... | |
| ▲ | sam_lowry_ 13 hours ago | parent | prev [-] | | Some TV on their CGNAT :-) |
|
| |
| ▲ | fearthemachine a day ago | parent | prev | next [-] | | I checked a small number of pages from the history of a defunct music festival site on WayBackMachine two days ago, on my phone, but found that the site needed a desktop to work well. But WayBackMachine has been 419ing (from nginx) my requests from my desktop these couple of days, when I’ve tried. Safari on a boring macOS installation. Same network (I mostly don’t use the cell service on my phone, when at home). Haveibeenproxied returns all clear. Maybe an ipv6 vs ipv4 thing? I have custom dns on the desktop and I think it only resolves ipv4 (maybe?) while my phone is using my upstream’s default DNS and probably prefers ipv6. It has occurred to me that the cost of shit like this, and of an extra 1-2 second periodically on site loads across the web to block bots (this is WAY up the last couple years) might go surprisingly far toward negating the time savings provided by LLMs… | |
| ▲ | potsandpans an hour ago | parent | prev | next [-] | | I've just been getting random "too many requests" | |
| ▲ | furst-blumier a day ago | parent | prev | next [-] | | Depends on what you do? The site loads fine to me and you gave no indication what your originating IP space is being used for. As a residential user without anything but "normie" internet usage (reading blogs, watching yt) I experience no interuption. Are you a "normal" residential user as well? Then you might need to bring it up with your isp | |
| ▲ | 1vuio0pswjnm7 6 hours ago | parent | prev | next [-] | | https://web.archive.org/web/20261120200432if_/https://scotta... "... the blogs/sites they reference..." It would be helpful to list some examples This particular blog, i.e., use of CSS and/or Javascript by Wordpress/author, is unreadable for me if using a popular graphical web browser. Unless I prefix the URL with "view-source:" Normally I don't notice such things because I use a customised text-only browser to read HTML, not a popular graphical one Without sourcing/running the CSS or Javascript, this blog looks fine | |
| ▲ | RIMR a day ago | parent | prev | next [-] | | Check if something on your network is running malware. You might be part of a DDoS botnet, and other services might be running something like fail2ban. | | | |
| ▲ | cyclotron3k a day ago | parent | prev | next [-] | | Are you using a VPN? | |
| ▲ | camillomiller 16 hours ago | parent | prev | next [-] | | The same machines that proved the GRH are happily prancing around the interwebs bloating everyone’s server bill, so you put up a bot wall with challenges that give you a lot of false positives, but also peace of mind that you won’t pay 200$ more in hosting bills this month. | |
| ▲ | thisisauserid 21 hours ago | parent | prev | next [-] | | Slightly? | |
| ▲ | Traubenfuchs a day ago | parent | prev [-] | | Which browser? Bidi / remote control enabled? Special plugins? AdBlock? |
|
|
| ▲ | jstummbillig a day ago | parent | prev | next [-] |
| I wonder he good we will be at not understanding anything to an ever increasing degree, with absolutely no hope of ever catching up. We will be super bad at absolutely everything within sciences and engineering and completely unable to contribute anything of substance, outside our own moods and quirks. Can humans be okay with that? Can we be happy cats? |
| |
| ▲ | mapontosevenths a day ago | parent | next [-] | | > Can humans be okay with that? Can we be happy cats? Sometime in your youth you decided what was important to you, and how to define your worth as a person. Many of us here decided that science, creation, and discovery were among the noblest of values... And consequently came to believe that those who did not engage in those things were therfore of lower value. We were wrong, and maybe now we have to learn that we were... It might hurt. Similarly the looming robotics boom means that those who define themselves by the very strange, but very deeply ingrained, protestant work ethic will come to find themselves in want of purpose... And a new means by which to judge the worth of others. The only thing I can really promise is that nobody will learn anything meaningful, and that we will simply replace our existing fallacies with new ones. | | |
| ▲ | Paul-E a day ago | parent | next [-] | | > Many of us here decided that science, creation, and discovery were among the noblest of values... And consequently came to believe that those who did not engage in those things were therfore of lower value. The nobility of old felt that they inherited "noble" qualities from their ancestors, and this justified their privileges and their elevated status as moral subjects. I can't help but see the parallels with the culture tech has built around intelligence and merit. It forms much our basis for identity and moral status. When AI exceeds us, does it take away our moral status, or does it challenge our value system based on intelligence and merit? | | |
| ▲ | avereveard a day ago | parent | next [-] | | It challenges our ownership of the means of production, we're speedrunning tumbling down the pyramid of maslow | |
| ▲ | cowlevel 12 hours ago | parent | prev [-] | | The nobility of new still believe that, but the inheritance comes in the form of several digits and an S with a stick through it. |
| |
| ▲ | epihelix 20 hours ago | parent | prev | next [-] | | > Many of us here decided that science, creation, and discovery were among the noblest of values... And consequently came to believe that those who did not engage in those things were therfore of lower value. > We were wrong, and maybe now we have to learn that we were While I don't want to engage with that disturbing and loaded "lower value" comment -- what makes you think that scientific discovery is now devalued? LLMs are helping us increase our discovery potential, not decrease it. And if we didn't still value discovery, we wouldn't be prompting LLMs to help us. | |
| ▲ | mbowcut2 a day ago | parent | prev [-] | | > Many of us here decided that science, creation, and discovery were among the noblest of values... And consequently came to believe that those who did not engage in those things were therfore of lower value. We were wrong, and maybe now we have to learn that we were... It might hurt. I have this feeling like we're all being pushed toward art. We'll be either poets or slaves. | | |
| ▲ | hobo123 14 hours ago | parent | next [-] | | Yes, and the sad part of this is that AI too can easily generate good looking art, and probably will be able to generate good sounding music (is been a while since I listened to anything suno). But of course art we make ourselves will have a point and a purpose behind it, so that's no reason to not do it. | |
| ▲ | jaredklewis a day ago | parent | prev [-] | | I’ll take poetry then I guess. But just given the current capabilities of AI, I think dishwasher is more realistic. The AIs will do maths and poetry while we do the dishes and laundry. |
|
| |
| ▲ | yen223 a day ago | parent | prev | next [-] | | You and I are already bad at 99.99% of things that can be done. This is just one more thing we are bad at. | | |
| ▲ | reasonableklout a day ago | parent [-] | | But before, you were still able to contribute to the 0.01% of things that you were good at. And by working together with other people that you trusted, you were able to collectively advance human scientific understanding. Now we have to trust the models. How do we build and maintain that trust, especially over the next decades as generations of people grow up without having to build the same intellectual muscles? | | |
| ▲ | yen223 a day ago | parent | next [-] | | You are free to do maths the old-fashioned way. Nothing about the presence of LLMs prevents that. | | |
| ▲ | cowlevel 12 hours ago | parent | next [-] | | You can. It won't be a contribution. | |
| ▲ | MiiMe19 15 hours ago | parent | prev [-] | | I absolutely hate this argument and I am so sick of seeing it. Some of us have jobs to make money you know. Some of us enjoyed those jobs you know. Some of us don't want to clock in every day to a job that we hate for the next 30 years of our lives you know. "Nothing about the presence of the torture-you-at-work-machine 3000 prevents you from not being tortured after work bro." |
| |
| ▲ | xiphias2 a day ago | parent | prev [-] | | Just like how I trust the house I am in to not fall down. It's simple: we have no choice. |
|
| |
| ▲ | ludwigschubert a day ago | parent | prev | next [-] | | https://gwern.net/doc/fiction/science-fiction/2000-chiang.pd... | | |
| ▲ | nozzlegear a day ago | parent [-] | | Somewhat related: Chiang's short story about mathematicians discovering that dividing by zero is legal and makes any number provably equal (or something like that, I can't remember exactly), introducing a crisis in mathematics. Fair warning, it's kind of a depressing story. https://fantasticmetropolis.com/i/division/ | | |
| ▲ | bigDinosaur 20 hours ago | parent [-] | | Isn't it more accurate to say that division by zero is intentionally not defined rather than 'undefined' since the latter implies it's just a discovery away from becoming defined? i.e. in JS lingo it's null rather than undefined, precisely because we want it to be and choose it to be. If the story explores that then mea culpa as I haven't yet read it. |
|
| |
| ▲ | chr1 a day ago | parent | prev | next [-] | | If machines become that good, then they also should be able to invent ways to make us smarter. It would be entirely ok if finding very complicated proofs becomes an activity for large machines, much like adding big numbers is. After all a proof is merely a path in an enormously large graph of repeatedly applied axioms. The interesting part is seeing how ideas are connected. It would be really nice, if it turns out to be possible to make cats a bit smarter too, while keeping heir "catiness". | | |
| ▲ | nradov 16 hours ago | parent [-] | | It would certainly be possible to breed cats for intelligence (however you define or measure that attribute). But that would probably come with increased risk of certain genetic disorders. There are no free lunches in biology. | | |
| ▲ | chr1 8 hours ago | parent [-] | | If you only choose for only one specific trait, you are likely to end up with genetic disorders, but that is not related to increased intelligence. (E.g. dogs are smarter but do not have increased risk of disorders). With ai predicting what changes to make, there is no reason for increased intelligence to cause genetic disorders, most likely it will simply cost them more energy. So your "no free lunches in biology" is true in very literal way, but humans and their cats have free lunches! |
|
| |
| ▲ | sl0wbrainRot a day ago | parent | prev | next [-] | | We moved on from manual labor to knowledge work despite a lot of people complaining about loss of hands on skills And if we meet the moment while generational churn replaces politicians we could draw a line in the sand... limit the AI at the hardware level from ever conjuring certain geometric patterns that lead it to certain tasks[1]; presumably "launch the nukes" and "add 1+1" are vastly different electrical geometry or "namespace collisions" occur in electrical state; but caveat being it's been a while since my hardware eng days so ...i could be way off being out of touch with the principles So restrain it to never being able to consider "fully autonomous no humans in the loop" geometry by short circuiting geometry that approaches certain shapes; like grounding a car; circuit breaker is not encoded as ephemeral state but manufactured in For utilitarian sake then, limit it to filling joann fabric and home depot and we all can be busy noodling local edge case solutions and keep up skills Norm Abrams-style in the event of systemic problems, writing zines and crafting mix tapes Can still make art and play instruments away from a computer. Can still watch the night sky, start a campfire and cook smores. Doing it all on a laptop everyday is a recent diversion from millions of years of homo-this-n-that evolution It's all resolvable socially and tech wise if we put in the effort. We got to the moon within a decade through similar collectivism Media propaganda since Reagan been keeping people apart but everyone who was around for it keeps saying we were happier 25-30 years ago so maybe that's the social peak we aim for and let the robots do the mining and repetitive stress injury stuff [1] https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/... | | |
| ▲ | mapontosevenths a day ago | parent [-] | | The Jetson's got it backwards, and it's obvious in hindsight. Of course the robots came for the high-paying office jobs first, that's where the money is. The robots will only come for the janitors job when there is nothing higher paying left. | | |
| ▲ | sl0wbrainRot 21 hours ago | parent | next [-] | | A lot of the high paying jobs are also largely made up Easy to disrupt social knowledge work that isn't keeping people biologically healthy and alive Janitorial work keeps human spaces clean for us. Relative to that sort of job, VP of Whatever who sits on Slack all day been doing nothing but noodling ephemeral finance variables | | |
| ▲ | slopinthebag 20 hours ago | parent [-] | | yeah good point. the total human value of all the slop produced by ai has gotta be pretty close to 0. yet to really see much big software that's meaningfully better. so who cares if it's slop? |
| |
| ▲ | onemoresoop 21 hours ago | parent | prev | next [-] | | You’re saying this as if the robots are going for the money. It’s really the money behind the robots which is chasing the money, the robots are just the vehicle. | |
| ▲ | bitwize 20 hours ago | parent | prev [-] | | George's job was literally pushing a single button while the computer, RUDI, did the actual work. But The Jetsons is still inaccurately utopian: One time when George was fired, RUDI intervened on his behalf and threatened to melt down the sprocket factory until he was rehired. |
|
| |
| ▲ | brookst 19 hours ago | parent | prev | next [-] | | How do you feel not understanding cars, the sun, the legal system from subatomic to macro system levels? Computer science was a blip in time where something revolutionary was also simple enough for one person to understand from foundations to applications. Most things aren’t that way. And yet left goes on. People have existing for millennia not deeply understanding the natural environment around them or the systems they use. When there’s a need for deep understanding, a handful of people fulfill the need. I empathize with you because my life has been about understanding everything I possibly can, but I was never under any illusion that it would be possible to deeply understand everything. So if that makes me a happy cat, sure, count me in. | |
| ▲ | Tadpole9181 a day ago | parent | prev | next [-] | | That's the vast majority of people who have ever lived. Of course we can. | | |
| ▲ | bluefirebrand a day ago | parent | next [-] | | ... Have the vast majority of people who have ever lived, had good lives? Were they happy? I'm genuinely curious where the median human's experience falls throughout history. Mostly happy? I suspect probably not | | |
| ▲ | nomel a day ago | parent [-] | | To continue the absurdity, are cats happier in the wild or captivity? My cat was eaten by a coyote. Do we only count the time it was alive, or weight it somehow? |
| |
| ▲ | sigmoid10 a day ago | parent | prev [-] | | Yeah. The majority of humans will now be joined by the 0.001% of people who were capable of significant intellectual contributions to modern society. |
| |
| ▲ | GoblinSlayer 8 hours ago | parent | prev | next [-] | | Science is already there. "Shut up and calculate". | |
| ▲ | ChrisGreenHeur a day ago | parent | prev | next [-] | | Luckily humans have the ability to abstract and simplify. Which is what we will now do. | |
| ▲ | drivebyhooting a day ago | parent | prev | next [-] | | Didn’t EY have some raving essays about catgirl paradise. I’m sure one can trawl it out of the LW archives. Soma is the endgame for 99% of humans. | | | |
| ▲ | catlifeonmars a day ago | parent | prev | next [-] | | Always have been. As an aside, cats likely run the world (or at least my household) so that may be a poor metaphor ;) | |
| ▲ | archagon a day ago | parent | prev | next [-] | | “We” is excluding the non-trivial contingent of humanity that will continue to do things with their own meat brains for the love of the game, or in anger. And I suspect all future progress will actually rest on their shoulders, to the great consternation of the now-deskilled AI boosters. | |
| ▲ | 0xbadcafebee a day ago | parent | prev | next [-] | | There is no logical reason that human beings would no longer be able to understand how things work. If you can understand the machine that created it, then you can understand what that machine creates. As far as anyone knows, there is nothing we can't eventually understand. It just might take more than 30 seconds of effort. Let's take this one thing as an example. So, this one dumb machine created millions of pages of crap to try to prove a theorem. That doesn't mean a human being can't understand the proof. It just means this one proof is hard to understand. Hard - not impossible. Hard doesn't mean we can't simplify the proof. Hard doesn't mean we can't break the proof up into bite-sized chunks and prove individual pieces of it. Hard doesn't mean that someone else won't find a much simpler proof. Hard doesn't mean we can't follow chains of logic to determine meaning. All it means is this one stupid machine in this one instance created a big stupid mess. A mess we can understand, if we try. | |
| ▲ | XorNot a day ago | parent | prev [-] | | How much advanced mathematics do you understand today? How well do you know how your car works? |
|
|
| ▲ | stymaar a day ago | parent | prev | next [-] |
| Tangent: LLM have stopped being stochastic parrots since RLVR was introduced in their training pipeline. |
| |
| ▲ | GMoromisato a day ago | parent | next [-] | | Probably right, but I think chain-of-thought has had a bigger impact on intelligence in LLMs. I.e., LLMs can reason their way to a solution instead of trying to one-shot it from training data. | | |
| ▲ | stymaar 18 hours ago | parent [-] | | I'd argue that it's the same thing. Asking a language model to “reason step by step” only marginally improved the results before we steered the thinking traces with RLVR. |
| |
| ▲ | bigstrat2003 a day ago | parent | prev [-] | | They have never stopped. It is inherent to the design of how they work. People just have bought into the hype trying to distract you from what they are. That doesn't mean they are useless tools, but don't ever make the mistake of believing they can actually reason. | | |
| ▲ | CuriouslyC 20 hours ago | parent [-] | | The point of stochastic parrots is they're just repeating things they've been pre-trained on, but RLVR pipelines teach the model to be good at "trying stuff" as part of an ad-hoc search algorithm. | | |
| ▲ | jdub 19 hours ago | parent [-] | | "stochastic parrot" is a metaphor, not an analogy RLVR isn't magic, it just bends some numbers chain-of-thought isn't magic, it just bends some numbers | | |
| ▲ | stymaar 18 hours ago | parent [-] | | > “stochastic parrot" is a metaphor, not an analogy It used to be an apt description, but not anymore. > RLVR isn't magic, it just bends some numbers So is all machine learning, but it turns out if you bend them well enough you can get very powerful capabilities. And also that's enough to say they aren't just trained on repeating what's in their training data, which is the reason to call them “stochastic parrots” in the first place. And their capabilities have vastly improved since we've started doing it. | | |
| ▲ | GoblinSlayer 7 hours ago | parent [-] | | So can they now learn from each other? | | |
| ▲ | stymaar 6 hours ago | parent [-] | | Depends what you call each-other, but kind of: at least they learn from the failures and sucesses of their ancestor during the RL training pipeline. (Like how AlphaGo learned to play Go in a way that no human ever did, just by playing against “itself”). |
|
|
|
|
|
|
|
| ▲ | martin-t a day ago | parent | prev | next [-] |
| The times I have seen the "stochastic parrot" argument used seriously, I could count on one hand. But the times I've seen people mock it because it's easier to argue against than well-constructed criticisms... Look, LLMs are models of the training data lossily compressed in a way that allows interpolating between patterns while decompressing. A pattern can be "it's better to use int32_t than int", what a for loop in c++ looks like, that testing for primality can be done by trying divisors only until the square root of n, etc. Combining those patterns, an LLM can produce a working is_prime function in c++. Older LLMs were only able to handle simple patterns so the code was more likely to be verbatim training data, newer LLMs work on deeper, more complex patterns so it's harder to detect where those patterns come from. Similarly, the list of approaches a mathematician uses to approach a problem is also a pattern. One of them will be proof by induction - how to convert such a proof to Lean will is also a pattern. The idea that induction can be done separately for odd and even cases is also a pattern. LLMs are no longer generating the final output directly - they are simulating human work but at a much larger scale. That's how they are able to generate novel results. But it's all based on labor scraped and patternized without consent or credit, let alone monetary compensation. And I still wouldn't call it intelligence because that gives people the idea that it's something which gives complete, correct and unbiased answers. It's still autocomplete under the hood, there's just more and more tooling built around it to keep it on track, to check intermediate results, to do arithmetic, etc. The LLMs in some cases is more like a heuristic for deterministic tools. |
| |
| ▲ | throw310822 24 minutes ago | parent | next [-] | | > Look, LLMs are models of the training data lossily compressed in a way that allows interpolating between patterns No. | |
| ▲ | solyin 20 hours ago | parent | prev | next [-] | | > something which gives complete, correct and unbiased answers Humans can't do that. Why would you hold LLMs to that standard? | | |
| ▲ | martin-t 20 hours ago | parent [-] | | The human brain is unreliable and very limited in terms of memory and speed. Barring hardware errors, computers have perfect memory which can be as large as you can afford and make computations which are reliable and reproducible. Humans can do amazing things with very limited hardware, LLMs make a lot of mistakes above the hardware level. The whole principle of outputting the probable next token is fundamentally different from logical reasoning, even if you can simulate logical reasoning on this shaky foundation. |
| |
| ▲ | Aurornis 17 hours ago | parent | prev | next [-] | | > The times I have seen the "stochastic parrot" argument used seriously, I could count on one hand. The person who coined the term and published the Stochastic Parrot paper has been trying to build a career on top of the controversy, publishing books and making a lot of noise on Twitter. For a while it was working and parts of Twitter were bought into the stochastic parrot idea, but the person at the center of it has a bad habit of lashing out at everyone and alienated most of them along the way. The concept feels antiquated now that we can all access extremely powerful AI models for $20/month and see for ourselves, but there was a time when people took this seriously. I think it was a very convenient argument for those who were in denial, and there was a large audience who wanted to consume ideas that validated their denial. | |
| ▲ | dsign 18 hours ago | parent | prev | next [-] | | In the sense that you propose, humans too are “pattern machines”. In idea space, a person who “teleports” to the unknown and new all the time and stays there is called “gone, mad”. When we probe the unknown, we carefully scaffold from the inside. From what is proven, or at worst, from what is generally believed, until we reach and surpass the boundary and touch the unknown immediately outside, at which point the boundary stretches to swallow that unknown and put it inside our bubble of knowledge. So, strictly speaking, most of the work is done inside. A human researcher is, for most of their work, a known-things engineer, an scaffolder, an autocompleter, | |
| ▲ | redox99 20 hours ago | parent | prev [-] | | > The times I have seen the "stochastic parrot" argument used seriously, I could count on one hand. Back when GPT 3.5 and 4 launched, there were a lot of people here that used that argument seriously. |
|
|
| ▲ | iLoveOncall a day ago | parent | prev | next [-] |
| Nobody is ever reviewing and verifying 17M lines of code. Ever. This is just another hypothesis on top of the Riemann Hypothesis. We now have an Hypothetical proof of the Riemann Hypothesis, great. Breakthrough. |
| |
| ▲ | schleck8 a day ago | parent [-] | | You don't need to verify 17 million lines of code, you just need to check that the hypothesis is formalized correctly (that's a subset) and that the kernel approves it Perhaps read up on the value proposition of Lean. | | |
| ▲ | classified 16 hours ago | parent | next [-] | | There are still bugs in Lean and until those are fixed its approval is not final. | | |
| ▲ | saagarjha 10 hours ago | parent [-] | | That's not how this is going to work, unless you are considering "final" to mean what convinces you. |
| |
| ▲ | iLoveOncall 21 hours ago | parent | prev [-] | | And how many lines does that represent? More than will be reviewed still. | | |
|
|
|
| ▲ | hawkice a day ago | parent | prev | next [-] |
| I (seemingly need to) suggest the headline is, in fact, mocking people who think modern LLMs are trained on imitation alone. |
| |
|
| ▲ | Betelbuddy a day ago | parent | prev | next [-] |
| Just a proof of the infinite monkey theorem: Give a sufficiently large number of monkeys keyboards....gently nudge them in the right direction with a few banana tokens...let them type away with just enough randomness for an infinite amount of time. Eventually, they will almost surely produce any mathematical proof imaginable... |
| |
| ▲ | mbowcut2 a day ago | parent | next [-] | | I genuinely can't tell if this is sincere or sarcastic. | | |
| ▲ | hansvm a day ago | parent | next [-] | | My best guess is it's sincere. That's been a core observation of LLMs for awhile. Filtered random search is basically an inefficient gradient descent. If you apply even a poor-quality "gradient" to a lot of problems with infinite compute (with large LLM budgets), it doesn't take that good of an LLM to accomplish great things from time to time. | | |
| ▲ | foltik 19 hours ago | parent | next [-] | | “Shine light on a random clump of atoms for long enough and it shouldn't be surprising that you get a plant.” | |
| ▲ | TheOtherHobbes a day ago | parent | prev [-] | | We've graduated to motivated stochastic parrots. |
| |
| ▲ | qup a day ago | parent | prev [-] | | There's enough information available to decide. Start with the title of the article |
| |
| ▲ | slyrus a day ago | parent | prev [-] | | It was the best of times, it was the blurst of times. |
|
|
| ▲ | pyuser583 16 hours ago | parent | prev | next [-] |
| Three months ago Anthropic was saying: “We don’t expect that the techniques Claude used will lead to proving the Riemann hypothesis.” https://www.anthropic.com/research/riemann-zeta |
|
| ▲ | freddealmeida 18 hours ago | parent | prev | next [-] |
| Was it really an AI? Or a Stanford trained mathematician that wrote a 900 page paper on it? hmmm... |
|
| ▲ | Jyaif 16 hours ago | parent | prev | next [-] |
| > And that will probably take years, since human mathematicians move only at biological speeds. Another prediction that might age poorly. |
|
| ▲ | TZubiri 20 hours ago | parent | prev | next [-] |
| “apparently proved” as a non mathematician, you guys have some soul searching to do, it seems like its proven? I get it when there’s one lone japanese that has done the math and has a proof that no one understands, but no one can disprove it yet. but proof that no one yet understands except an AI? it sounds like a koan, is a proof a proof if no one is convinced beyond a doubt by it? |
|
| ▲ | kraf 15 hours ago | parent | prev | next [-] |
| What ignorance to quote Ayn Rand here, the hero of the entitled world enders. |
| |
|
| ▲ | kittikitti a day ago | parent | prev | next [-] |
| The irony is that people are repeating "stochastic parrots" while have meme knowledge about what stochastic means. |
|
| ▲ | otabdeveloper4 a day ago | parent | prev | next [-] |
| Sorry bud, still not buying Anthropic stock. |
|
| ▲ | mbgerring a day ago | parent | prev | next [-] |
| “Because I am impressed and mystified by its outputs, I believe that the next token prediction machine must be doing something more than that, regardless of the lack of any evidence” AGI is when the output of LLMs is plausible enough to fool putatively intelligent people that they are alive |
| |
| ▲ | preg_match a day ago | parent | next [-] | | LLMs are most definitely doing more than just predicting the next token. They can reason. This is emergent behavior. The idea is that, while one small part is doing something simple, the entire system together produces behavior that’s much more than that. I kind of view it as how slime mold can solve a maze. Mold doesn’t think or see, but the system of a bunch of slime mold together has almost a rudimentary intelligence. | | |
| ▲ | _justinfunk a day ago | parent | next [-] | | I'm of the opinion that what is emerging is the human intelligence inherent in the training data. | |
| ▲ | mbgerring 21 hours ago | parent | prev | next [-] | | Be specific. Describe what you think the computer is doing that is “more than” predicting the next token, and explain why you believe that. | | |
| ▲ | solyin 20 hours ago | parent | next [-] | | It generalizes reasoning. If we are to believe the link's linked paper, that's new. It didn't exist before. Yes a human helped. AI also played a part. Somewhere in the calculation of a pile of linear algebra, gigabytes, upon gigabytes of training data and RAM, whilslt still predicting the next token, the prediction that went into choosing that specific next token, reasoning occured. Internal to the model, it can represent intermediate relationships, combine them, and then use those results to generate a unique answer. | | |
| ▲ | soon_to_be_sb 18 hours ago | parent [-] | | > into choosing that specific next token, reasoning occured. This is where you fall. Imagine when you ask an LLM, "can cats fly", it reply "No". But then, you can feed it so much fake training data based on this specific question so that it will say "Yes" to this question, despite all the genuine training data it had before that point. You won't be able to teach a reasoning entity such contradicting facts. There is no mechanism in the LLM to reject fake data that contradicts with its existing knowledge. So an actual intelligent entity should curate the training data first, for the LLM to show the "appearance" of intelligence. | | |
| ▲ | preg_match 18 hours ago | parent | next [-] | | No, LLMs can reason. If you ask it to explain how and why cats can fly, it will show its reasoning. It’s not a secret. You can try to convince it otherwise, and of course if you poison the data that will affect it. But then it will realize… well most mammals don’t fly. And cats don’t have wings. So no, cats cannot fly. That’s rudimentary reasoning that a child could probably do, but it’s reasoning. You can try the same experience with a human child. Tell them over and over if cats can fly. They might then say yes! Then ask them to explain how, and they’ll reason and decide no, they cannot. Of course LLMs can do reasoning much more advanced than that, as evidenced by theorem proving. | | |
| ▲ | Planktonne 9 hours ago | parent [-] | | > No, LLMs can reason. If you ask it to explain how and why cats can fly, it will show its reasoning You're eliding an important difference here. > can reason...show its reasoning The appearance of a thing is not necessarily the same as the thing. | | |
| ▲ | lostmsu 5 hours ago | parent [-] | | Yesterday GPT 6 wrote a new joke. Same day world famous comedian came up with a word for word the same joke independently. Except his joke was funny! | | |
| ▲ | Planktonne 2 hours ago | parent [-] | | The argument would be stronger if you had a real example rather than a made-up one. |
|
|
| |
| ▲ | solyin 12 hours ago | parent | prev | next [-] | | > "On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question." - Charles Babbage, 1864 memoir, Passages from the Life of a Philosopher | |
| ▲ | solyin 12 hours ago | parent | prev [-] | | We tell children stories of Santa and they believe their parents. Are children merely the appearance of intelligence as well? |
|
| |
| ▲ | emtel 18 hours ago | parent | prev | next [-] | | You first: explain how “predicting the next token” can be done without doing anything special. “Predict the next token” isn’t a process or algorithm. If you traveled back in time and said “you can build AI by predicting the next token” this wouldn’t help them do anything they didn’t already know how to do. | |
| ▲ | preg_match 18 hours ago | parent | prev [-] | | Nobody can do this, nobody knows exactly what it’s doing or why. The system is far, far too large to comprehend or understand, and it’s opaque. At its core it’s just a bunch of weights, just floats. Put together though it can solve problems and reason, including problems which has not been solved before. Similar to the slime mold. It’s just a single cellular organism with benign, and unimpressive, DNA programming. It doesn’t have any organs, just organelles, and certainly no nervous system. But, put a bunch together, and it can solve a maze like a rat. Interesting. Largely I think these “stochastic parrot” type takes are just perpetrated by people who do not use LLMs. If you use LLMs to program, as virtually every software dev working does, it’s indisputable they can reason. Not always well, but they can reason. |
| |
| ▲ | bigstrat2003 a day ago | parent | prev | next [-] | | > LLMs are most definitely doing more than just predicting the next token. They can reason. No, they cannot, and we have seen this proven time and time again. | | |
| ▲ | matkoniecz 21 hours ago | parent [-] | | > No, they cannot, and we have seen this proven time and time again. how it was proved? | | |
| ▲ | solyin 20 hours ago | parent [-] | | Didn't you see on Twitter months ago? It couldn't count the number of "R"s in strawberry, so we should just call the whole thing off! |
|
| |
| ▲ | alpaca128 a day ago | parent | prev [-] | | Someone on YouTube showed that the same maze can also be solved by plain water. I don't see how the medium being slime mold makes it more impressive or a sign of intelligence. | | |
| |
| ▲ | ashdksnndck a day ago | parent | prev [-] | | It’s like saying humans brains are just a bag of subatomic particles mechanically following the physical laws that predict the next state. Vacuously true. |
|
|
| ▲ | turzmo a day ago | parent | prev | next [-] |
| [flagged] |
| |
|
| ▲ | devindotcom a day ago | parent | prev | next [-] |
| What an offputting title. The sneer really comes through. If I ever had any desire to read what this man wrote, this little potshot has extinguished it. |
| |
| ▲ | freehorse a day ago | parent | next [-] | | If the url did not read scottaaronson.blog this kind of post would have been [flagged][dead], but alas I do not see what this kind of post offer here when there is already at least one thread to discuss Alpöge's result with 1000+ comments. | |
| ▲ | mbowcut2 a day ago | parent | prev | next [-] | | Not off-putting in the slightest. It's a well-deserved jab at the earliest LLM naysayers whose "criticism" was just ignorance wrapped in causeless activism. They claimed that LLMs were just randomly repeating locally coherent phrases back to us and that no amount of scaling could produce real intelligence. Well, the goal posts have moved far indeed if a proof of GRH isn't intelligence. | | |
| ▲ | santadays a day ago | parent | next [-] | | I think it's well deserved for all the people who made a name for themselves nay-saying llms/ai. I think it's less well deserved for people worried that their careers are shot and won't be able to provide for their families. | |
| ▲ | uptrails 19 hours ago | parent | prev | next [-] | | I've not heard anyone seriously use "stochastic parrots" as a way to deny the utility of LLMs. Even if you go back to the original "stochastic parrots" paper, it wasn't saying anything of this sort. It was a paper about costs and risks (chiefly: biases from training data, lack of accountability / explainability), not a criticism of AI hype. The concern was about probabilistically parroting bias. | |
| ▲ | dist-epoch a day ago | parent | prev [-] | | > the goal posts have moved Indeed, it's not fashionable anymore to say "stochastic parrot" or "glorified auto-complete". What is fashionable today is to talk about the "convex hull of human knowledge that LLMs interpolate in" |
| |
| ▲ | zone411 a day ago | parent | prev | next [-] | | It's good to poke fun at the AI phrase that's aged the worst and all the news articles that promoted it as fact. It did a lot of damage to people's understanding of AI. | |
| ▲ | shearerp a day ago | parent | prev | next [-] | | It's not a sneer. It's a wisecrack about an absurdity. The feeling of a sneer comes only from the absurdity of the belief being definitively revealed. For anyone who feels sneered at: the problem is the absurd belief, not the reveal. "Don't sneer at the Emperor for parading around with nothing on!" | |
| ▲ | fasterik a day ago | parent | prev | next [-] | | Scott Aaronson has more intellectual chops and a better track record than 99.999% of people commenting on AI news. The sneer is a small fraction of that coming from the know-nothings who have been using phrases like "stochastic parrot", "fancy autocomplete", etc. for many years now. The parrots are coming home to roost, as they say. | |
| ▲ | amelius a day ago | parent | prev [-] | | The sneer towards who? The people who sneered that AI is just stochastic parrots? I think they deserve to be sneered at, honestly. | | |
| ▲ | reducesuffering a day ago | parent [-] | | Seriously. We are less prepared for the present and future because of all the denialists and naysayers, many on HN, who have repeatedly cried "stochastic parrot", "it just regurgitates known things, it can't create" and "marketing stunt" endlessly, sometimes >80% of the sentiment on threads over the past years. | | |
| ▲ | mr_mitm a day ago | parent | next [-] | | The sentiment can still be found today in the other thread to the original GRH submission. | |
| ▲ | chihuahua a day ago | parent | prev [-] | | But surely, if we pooh-pooh LLMs enough, people will stop using them and it will all just go away? |
|
|
|
|
| ▲ | zug_zug a day ago | parent | prev | next [-] |
| I mean they're obviously not stochastic parrots, I think most of us engineers have known this for a couple years now, though I do admit I saw a pretty obnoxiously vocal minority on the point here on HN. But at the same time, where's the RSI? If the RSI is there then make the company profitable already [though I suppose that means letting go of all the staff? But if it's improving itself, not sure what humans are in the loop for]. Genuinely curious when AI will be able to solve real-world problems, what would 20,000 hours of AI say on the topic of whether Epstein killed himself? |
| |
| ▲ | saulpw a day ago | parent | next [-] | | RSI seems to be here? It may not be the glorious 100% autonomous RSI that sci-fi authors predicted, but 90% autonomous RSI seems to be what's creating Claude Code and putting inference models on FPGAs and whatever happens when I say "Claude, plan it. Claude, implement it. Claude, fix it. Claude, make it faster." I mean it's not perfect but it's remarkable how much you can just...tell it to make itself better? | | |
| ▲ | zug_zug a day ago | parent [-] | | I mean it's definitely remarkable. But I don't think it's RSI. My mental model of RSI is essentially isn't quite asymptotic but exponential. If a system can make itself 5% smarter every week then it would be 12x as smart in one year at which point it's not clear to me if we even need governments or markets because we have a being that can direct everything. In short I don't believe there's any world where there's true RSI intelligence and anthropic returns green paper to "shareholders" so they can buy pretty cars as status symbols. | | |
| ▲ | saulpw a day ago | parent [-] | | Perhaps your mental model of [the effects of] RSI is mistaken, though? It's not an inherent part of RSI that there will be consistent 5% improvements to be made. Even with domains that have been 'solved' by AI, like Go (board game) or OCR, the improvements hit a point of diminishing returns. Like at some point there aren't any 5% improvements left, so you make 1% improvements, and then .1% improvements, etc. And that is also just providing an explanation for the sake of accepting your premise, that we're not experiencing exponential improvements. In a lot of ways it sure seems like like a radical acceleration. I mean Claude Opus was only released exactly 1 year ago, and now long-standing math problems are being solved every other week. I mean if it were RSI, what would literally be different today? | | |
| ▲ | zug_zug 19 hours ago | parent [-] | | Yeah the 5% rate is somewhat arbitrarily picked, but the reason I expect it not to look like go is because the machine is improving itself, so presumably it unlocks a higher "iq" which in turn will in turn mean that the types of optimizations it will be able to find will grow. Of course there is a raw hardware limit, but it's my intuition that we are under-utilizing the hardware by multiple degrees of magnitude. The whole pattern of using token prediction and realizing intelligence falls out the other end has been more or less stumbled upon by us, there's no reason to think it's optimal. |
|
|
| |
| ▲ | wolfinterval 20 hours ago | parent | prev [-] | | Stochastic parrot was a brilliant, catchy label to make a point. Arguing over whether a language model is really a stochastic parrot mistakes a metaphor for an actual claim. Like arguing back and forth if a paper tiger is made out of printer paper or newspaper. |
|
|
| ▲ | 1239982716 a day ago | parent | prev [-] |
| Why the snarky headline? If parrots can generate statistically plausible proof snippets, and they are backed by a combination of Lean, other proof techniques stolen from ACL2 and a human mathematician they can still be stochastic parrots. Aaronson is too close to the "rationalist" cult. Maybe he should sit down with ACL2 and get a clue. Emily Bender didn't speak about all that augmentation when coining the phrase. |
| |
| ▲ | zone411 a day ago | parent | next [-] | | How do Emily Bender and this toxic circle still have an audience in 2026? How wrong do they have to be before people stop taking them seriously? | |
| ▲ | jmyeet a day ago | parent | prev | next [-] | | Isn't it obvious? Scott Aaronson is a theoretical computer scientist, which is essentially a mathematician. Scott is likely seeing his career prospects disappear in front of his eyes just like so many others. Too many people only see the perils of something once it affects them. The optimists think AI will be like the cotton gin where making something cheaper and more accessible actually increased the demand for slaves as the price dropped. I'm more pessimistic. A lot of jobs are going to collapse in demand in the coming years. For awhile those at or near the top will survive and possibly make a lot of money but the rising floodwaters come for everybody. | | |
| ▲ | betterunix2 a day ago | parent | next [-] | | > Scott is likely seeing his career prospects disappear in front of his eyes Why would he be seeing that? The combination of a mathematician and lean is powerful, and the combination of a mathematician, lean, and AI is more powerful still. Note that the mathematician is always part of that picture. That is fundamental -- no software system, no matter how big and complicated, can actually solve every math problem (this is a fundamental result in math that started the entire field of computer science). If you read TFA he basically tells you what he is actually worrying about here: "the Lean formal verification system still has issues, from kernel bugs (which have, in the past, allowed proofs of 0=1) to uncertain soundness of the underlying type theory. In my view, these issues can and should urgently be fixed! Until they are, a sufficiently motivated AI could almost certainly exploit them to prove a false statement!" He is worried about the proof checking system and the possibility that an AI might exploit bugs in the proof checker, not the disintegration of his entire career because AI can "just figure it all out" or whatever. | |
| ▲ | skeledrew a day ago | parent | prev | next [-] | | > A lot of jobs are going to collapse in demand in the coming years. For awhile those at or near the top will survive and possibly make a lot of money but the rising floodwaters come for everybody. Funny, I'm actually optimistic for this. Let the flood wash away this messed up system that puts people in tiers and says people should only have access to what they can "afford", not to what they need. And then allows a few to be able to afford almost anything, and many to be able to afford nothing, just by being the first to X, or acquainted with Y. | | |
| ▲ | allturtles a day ago | parent [-] | | Ai is going to make economic stratification much worse, not better. | | |
| ▲ | skeledrew a day ago | parent | next [-] | | Things will become really bad for a while, and depending on how long people (particularly businesses) hold on to the system. But a full collapse is inevitable as the system design ensures businesses will continue the drive toward automating away everything, so the jobless population will continue to grow. And it'll get to the stage where businesses will have to start really reducing their prices, not to be competitive, but because fewer and fewer people can afford anything. Currency loses all value. Businesses eventually cease to be because markets die out, and governments - with the use of AI - fully take over the management and distribution of resources. Throw in some potential wars and a lot of death if there's a strong enough resistance to the change, otherwise a relatively peaceful transition. | |
| ▲ | TheOtherHobbes a day ago | parent | prev [-] | | That may turn out to be an optimistic take. |
|
| |
| ▲ | pegasus a day ago | parent | prev [-] | | > Scott is likely seeing his career prospects disappear in front of his eyes ...no he's not, that's a ridiculous statement. He is a prestigious professor with tenure at a top university. Also, mathematics is much much more than just proving theorems. As the word goes, "Good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians build theories". Someone has to carve out domains of interest to humankind. And humans will always know better what is of interest to humans than machines. These kind of reactions are a form of AI psychosis. And lastly, this result was reached by a top mathematician steering the model towards the right approaches. You might say that the next iteration of the model might not need such steering to reach this result, but that just means that the pilot seat will have shifted higher up, so to say. New frontiers of complexity will become approachable to humanity and approach we will, "exploring strange new worlds and boldly going where no one has gone before". |
| |
| ▲ | Morromist a day ago | parent | prev | next [-] | | Are you saying we should address llms with more respect now that their good at making lean proofs? Maybe we could call Claude Dr. Claude and Gemeni could be Professor Gemeni. | | |
| ▲ | qwtah a day ago | parent [-] | | Scott Aaronson is an AI booster who is mocking Bender in the headline. He isn't mocking the machines. How on earth can the comment you reply to be misunderstood? It even defends Bender. | | |
| ▲ | Morromist a day ago | parent [-] | | Ahh, I see. I took the article to be somewhat skeptical about the whole thing, even though he says he belives that the proof will hold. I personally feel like AI is a bit of a parrot hooked up to massive compute, which is the powerful part of it, and that it can solve unbelivably hard math problems but is still a parrot, so I may have read my own thinking into the headline instead of the author's. |
|
| |
| ▲ | BoppreH a day ago | parent | prev | next [-] | | I'm 99% sure it's not his actual opinion, but a sarcastic remark aimed at the people who claim that LLMs are just stochastic parrots. Like "your 'peaceful' dog just bit me". | | |
| ▲ | pessimizer a day ago | parent [-] | | LLMs are just stochastic parrots, it's simply not an insult. It reminds me of how people during covid would yell "it's not just like a bad flu" when literally the only valid historical comparisons we had were bad flus that killed millions. When you're worshiping something, anything that implies that it is comparable to anything else is an insult to its divinity. The title doesn't imply that sort of thinking on Aaronsom's part at all, though. Some simple dumb thing blowing through math problems thought maybe unsolvable is a hilarious contrast. | | |
| ▲ | cma a day ago | parent [-] | | Computing a distribution and sampling from it isn't the same thing as sampling from a distribution. |
|
| |
| ▲ | sph a day ago | parent | prev [-] | | > Why the snarky headline? The boosters are out in force to gloat today. | | |
| ▲ | ModernMech a day ago | parent [-] | | I don't think it counters the boosters to ascribe a result like this exclusively to the "stochastic parrot" though. There's an army of people behind each one of these results. | | |
| ▲ | sph a day ago | parent | next [-] | | It's clearly sarcastic. The article is just another booster throwing fuel to the flames because they're in a good mood today. | |
| ▲ | mkehrt a day ago | parent | prev [-] | | If you don’t understand the title is sarcastic, uh I don’t know what to tell you you. | | |
| ▲ | 1AZAGD 10 hours ago | parent [-] | | Even Scott Aaronson himself has updated the article to tell the morons that it was sarcasm: "According to “Poe’s Law,” there is no irony or sarcasm on the Internet so obvious that it won’t fly over countless people’s heads. In line with that, lots of people on social media are now saying variants of “how dare Scott Aaronson call LLMs ‘stochastic parrots,’ that demeaning and trivializing term from 2021? Doesn’t he understand that these so-called ‘stochastic parrots’ just proved the Riemann Hypothesis?” ROFL." Yet on this message board correct comments like the one I respond to are downvoted. Average IQ here is 90 at best, so people marvel at LLMs who seem sentient to their monkey brains. | | |
| ▲ | ModernMech 9 hours ago | parent [-] | | Man you guys don't have to be so rude because I didn't understand sarcasm on the Internet, give it a rest. |
|
|
|
|
|