| |
| ▲ | irthomasthomas 4 hours ago | parent | next [-] | | Choice quotes from my notes on the affair: To: Buckmaster
From: A friend at OpenAI
Subject: Openai is about to do something really stupid.
"I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, 'Why would you ruin your career?' I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, 'If you don't want me to be nice, then I don't have to be nice.'" (Sébastien Bubeck later apologized on X for the remark.)
"Sebastien twice asserted that he wanted Levent removed from authorship..."
"The second was that, after posting Euler, I alone write a paper presenting the Navier-Stokes result, acknowledging that an internal OpenAI model had resolved it.""I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer."
(OpenAI subsequently stated publicly that while unlikely, it could not rule out that de-identified data from their product usage improved the model.) | | |
| ▲ | zelphirkalt an hour ago | parent | next [-] | | "While unlikely ... it could not be ruled out ..." So basically he hit the nail on the head and they want to lie about it without lying. Either that, or they are supremely incompetent, not knowing even what they ingest into their models. Both are very much not inspiring any faith in their trustworthiness. | |
| ▲ | keeda 3 hours ago | parent | prev [-] | | In broader context the “being nice” part seems like both of them talking past each other. What happened was the two researchers, Buckmaster and Alpoge (from TFA) solved a significant subset of Navier Stokes (referenced as “Euler”) which set off rumors. Through whatever game of Chinese Whispers ensued, OpenAI thought they had solved the full Navier Stokes problem, which they hadn’t. So OpenAI pointed their latest unreleased model at it, and it actually solved the full problem!! Pure George Danzig vibes on that one. So when this meeting happened, the OpenAI folks were surprised that these researchers had NOT solved the full problem, only the subset. Yet from their perspective, they were offering them credit for the full problem, and so “we’re being nice.” The Anthropic employee (Alpoge) was clearly an optics issue and so they felt they didn’t have to be that nice. But the other researcher, Buckmaster, was suspicious that their work had been stolen, and so was understandably in a combative mood. You can see that from his perspective they were not being “nice”, they were trying to scoop them, possibly on stolen work, and then bribe them with shared credit. But they had not solved the full problem, and academic plagiarism is a very serious allegation that can ruin careers of both the accused and the accuser. So from OpenAI’s perspective it was like, why would you accuse us of plagiarism when you yourself have not solved the full problem and when we’re willing to share undue credit for it?? This was made worse by the fact that OpenAI would not categorically deny if the others’ work had been used for training until much much later. This was terrible optics, but anyone who’s dealt with large scale data pipelines compounded by privacy issues knows that it gets very difficult to trace these things, so it takes time to make a definitive statement. Alpoge somehow seemed less troubled by this whole episode, but I guess it’s because not only was he gonna be a multi-millionaire soon, he had Riemann stewing on his backburner. I feel like when they finally make a movie out of all this, this scene would end up being a bit of comic relief. | | |
| ▲ | irthomasthomas 2 hours ago | parent | next [-] | | That version of events skips some central facts from Buckmaster’s statement. "they came prepared and they wanted to show us the prompt... they showed me on the screen and they just literally copy pasted the Millennium Prize problem, so they said that they took the prize and they pressed enter and they got a solution, and at that point I just had to laugh because it's just so ridiculous. It came out within a few minutes that this was a lie" "Levent had been told by Sebastien 'very little human input' had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem..." "I was offered the opportunity to write as sole author of their paper and throw my collaborator under the bus... they were asking me to throw him under the bus, someone who I've been working with for a year, to throw him under the bus just because he's an Anthropic employee" "the right paper to compare is the unforced Euler equation. Now the unforced Euler equation, all the mechanisms in this unforced Euler equation are identical to a different version of the Euler paper that we haven't released yet... basically all the main architectural ingredients in that unforced Euler were in our work for which they would have had on their servers, and it hadn't been published"
"they didn't use 10,000 agents to get to the Euler problem, they used 100 agents. So they worked from absolutely nothing with only 100 agents... and then they ended up with an identical architecture to what we had" | | |
| ▲ | randysalami an hour ago | parent | next [-] | | Not to put anyone on an idol but the only one who comes out of this not acting like a greedy, anti-human is Buckmaster and his colleague (I think). Straight up lying by OpenAI, very predatory "meritocratic" behavior. What gets me the most is "they didn't use 10,000 agents to get to the Euler problem, they used 100 agents. So they worked from absolutely nothing with only 100 agents... and then they ended up with an identical architecture to what we had". What the hell! So they lie about their technical claims too on a factor of 100x?? And companies are taking their words seriously, no wonder there is a house of cards being built in the industry. | |
| ▲ | keeda an hour ago | parent | prev [-] | | I don’t see how those details change the overall outline of what I presented above through? Maybe you could nitpick that it was a team with a model using 100 agents, and not a solo model with 1000 agents. But the fact remains OpenAI ended up solving the bigger problem themselves just because they thought the researchers had solved it already when they hadn’t. That is low-key hilarious. The Euler plagiarism insinuations are much more serious but given all the 100s of other results we’ve seen come out, highly unlikely. |
| |
| ▲ | ameliaquining 2 hours ago | parent | prev | next [-] | | I think it's clear enough that something broadly like this is right, even if people don't want to admit it because the vibes of extending any charity at all to frontier labs are bad. | |
| ▲ | bdelmas 2 hours ago | parent | prev [-] | | Terrible optic is such a bs executive wording when they lied, coerced, bullied, and threatened him… I like the summary but knowing what irthomasthomas posted that’s missing from it, and having a better picture of what happened; terrible optic doesn’t work. |
|
| |
| ▲ | simonw 5 hours ago | parent | prev | next [-] | | Yes, it was a joint effort with Tristan. Part of the scandal around it came when OpenAI offered to let Tristan help write up their result but wouldn't allow Levent to be involved because he worked for their competitor. | |
| ▲ | Betelbuddy an hour ago | parent | prev | next [-] | | >> Almost unprovable of course. Very much provable, if he would sue, and a lawyer would ask for discovery. | |
| ▲ | an hour ago | parent | prev [-] | | [deleted] |
|
| |
| ▲ | finghin 5 hours ago | parent | next [-] | | It's almost certain that they have had people working on Riemann since way before the Navier-Stokes theorem proof was nearing completion. It's a long shot, but the value of a proof is next-level compared to NS | | |
| ▲ | thaumasiotes 4 hours ago | parent [-] | | > It's a long shot, but the value of a proof is next-level compared to NS The value of a proof? People have assumed it's true for so long that there are any number of results already proved "if the Riemann hypothesis is true". What value does the proof add? | | |
| ▲ | DennisP 4 hours ago | parent | next [-] | | There are also lots of results proven "if the exponential time hypothesis is true," and now it looks like an AI has proven it false. | |
| ▲ | SubiculumCode 4 hours ago | parent | prev | next [-] | | "... but what I saw during the course of progress so amazed me that I believe it is one dawn across science of something
far greater." | |
| ▲ | QuesnayJr 2 hours ago | parent | prev | next [-] | | Doesn't that increase the value? We now know all of those results are true, where before we didn't. | | |
| ▲ | kypro an hour ago | parent [-] | | If value here is a measure of utility, its not super useful on its own that something that was almost universally assumed to be true is now known to be true. But with a problem as difficult and fundamental as the Riemann Hypothesis the thought has always been that in the process of solving it we'd be likely to develop or discover something that did have real world utility. At least this is often what happened in the past, the obvious example here being Newton developing calculus, a field of mathematics which can be applied to other problems beyond the narrow set of problems he was trying to solve. What I find interesting about this, if the claim holds up, is whether this will assumption will remain true, or if the value of these AI proofs is going to significantly lower than those of human discoveries in the past. Maybe a good analogy here is to think about how all of tools we humans have created has been ultimately in the pursuit of solving some problem. For the developers here, consider this – if ChatGPT had existed before React, would React ever have been developed? I mean, if an LLM can brute force the same result with just vanilla JS the need to develop frameworks and libraries to solve what was once difficult for a human to solve drops. I'll note that I'm playing devils advocate here as I think AIs would probably build something better than React had it not existed, but it's something to think about at least. I suspect there's a higher probability than some here are assigning that this proof ultimately ends up have relatively little utility. | | |
| ▲ | QuesnayJr 33 minutes ago | parent [-] | | It wasn't assumed it was true. People thought a world in which it was true was more beautiful than one where it wasn't, but proving it true wasn't a pesky little detail. In fact, there was a whole literature of papers proving that if the GRH was false in one particular way (the existence of Landau-Siegel zeroes) many theorems followed from it. For example, if there's a Landau-Siegal zero, the twin primes conjecture is true. |
|
| |
| ▲ | ncallaway 4 hours ago | parent | prev | next [-] | | Well, for one all those other results that assumed Riemann was true are now also known to be true. That seems valuable | |
| ▲ | applicative 4 hours ago | parent | prev | next [-] | | That there are so many results beginning 'if RH' has exactly the opposite significance you are trying to suggest. | |
| ▲ | Jweb_Guru 4 hours ago | parent | prev | next [-] | | The value it adds is that those theorems are now actually true, which means all those derived results are "real" mathematics... as every mathematician knows, it is quite easy to prove very surprising theorems when False is in your context. | | |
| ▲ | thaumasiotes 3 hours ago | parent [-] | | But that's 100% wrong. If it was easy to prove surprising theorems from an assumption of the Riemann hypothesis, it would also be easy to disprove the Riemann hypothesis. | | |
| ▲ | Jweb_Guru 3 hours ago | parent | next [-] | | To borrow the usual quote about ingenious fools, the trouble with "easily" disproving that False is in your context is that there are such ingenious proofs of False. One can appear to be doing very substantial and sophisticated mathematics, encountering many obstacles along the way that look identical to what you'd face if you were relying on something true, only to realize much later that one of your hypotheses is uninhabited. The concern was that were the Riemann Hypothesis false, a very, very large subset of modern mathematics might fall into this category. | |
| ▲ | xrisk 3 hours ago | parent | prev [-] | | not at all. we can prove a lot of things using P != NP, but that doesn’t make a proof or disproof of that statement easy |
|
| |
| ▲ | baq 3 hours ago | parent | prev | next [-] | | 150GB of lean must be worth something | |
| ▲ | lern_too_spel 4 hours ago | parent | prev | next [-] | | The post mentions some generally useful machinery derived on the way to the proof. | |
| ▲ | 3 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | thevalueisai 4 hours ago | parent | prev [-] | | The value is proving AI can be useful after all (and we're fucked). Remember, not too long ago we had hordes of folks here proclaiming machines can't think, reason or be creative in any productive sense. Many still do, but it's starting to look bleaker by the day. | | |
| ▲ | agentultra 4 hours ago | parent | next [-] | | This is a competent person using a tool in their work. The tool isn’t doing the thinking or the proof on its own. It also hasn’t been vetted/verified yet. There’s still work to do. | | |
| ▲ | 3 hours ago | parent | next [-] | | [deleted] | |
| ▲ | 0c3ca83 2 hours ago | parent | prev [-] | | What were the prompts? For many of the proofs coming out of the AI companies where the prompts were posted, they ended up being variants of "solve the problem", "keep going", "I believe in you". Many of the results could have come from a 15 year old that heard of the problem in passing. The result here is a quarter of a million pages. Human involvement is going to be minimal. | | |
| ▲ | what an hour ago | parent [-] | | If you believe these were the prompts, I’ve got a bridge to sell you. |
|
| |
| ▲ | ThrowawayR2 3 hours ago | parent | prev | next [-] | | These AI bros don't get it. These LLM systems solving math problems have the benefit of being constrained by formal rules of math providing constant feedback when they break the rules that prevents them from wandering into la-la land. As Dijkstra said in his prescient "On the Foolishness of 'Natural Language Programming'": > "The virtue of formal texts is that their manipulations, in order to be legitimate, need to satisfy only a few simple rules; they are, when you come to think of it, an amazingly effective tool for ruling out all sorts of nonsense that, when we use our native tongues, are almost impossible to avoid." To whatever extent that LLMs can "think, reason, or be creative" in generating these proofs (or writing code), they're aided by very high guardrails so it doesn't say very much about what they can do in broader, fuzzier domains. | | |
| ▲ | randomNumber7 an hour ago | parent | next [-] | | Goedel proofed that you can formulate nonsensical statements in math (and any formal system that is complex enough to be useful) | | |
| ▲ | ThrowawayR2 31 minutes ago | parent [-] | | Gödel proved that statements can be created that are definitely true or false but for which no proof can be constructed. It is not relevant here since proofs are successfully being constructed, according to checking using Lean, by an LLM. |
| |
| ▲ | brookst 2 hours ago | parent | prev [-] | | Would you say that the early results of AI beating humans at backgammon and checkers had any predictive power towards chess and go? | | |
| ▲ | ThrowawayR2 2 hours ago | parent [-] | | The kind of AI used for backgammon/checkers and the kind of AI used for chess/go, and LLMs differ greatly so I'm not sure what the point of your question even is. A casual check says that the state of the art LLMs have much lower [EDIT]Elo ratings for checkers than the purpose built "AI" that beat humans. | | |
|
| |
| ▲ | ToucanLoucan 4 hours ago | parent | prev [-] | | > Remember, not too long ago we had hordes of folks here proclaiming machines can't think Machines can absolutely think. It's really all they do. > reason Murky, but LLMs prove with an avalanche of statistical modeling, they can do something kind of analogous to reasoning, with the caveat that whatever they output will necessarily have to exist in some fashion in their training data. > or be creative Absolutely not. By definition an LLM cannot create original work, it can only remix and recombine what it already knows. That can be interesting, and bloody useful especially for code tasks, but it is not making anything original. | | |
| ▲ | retsibsi 4 hours ago | parent | next [-] | | > Absolutely not. By definition an LLM cannot create original work What (if anything) unambiguously counts as original work, according to this definition? | | |
| ▲ | ToucanLoucan 4 hours ago | parent [-] | | Arguing the definition of original is pointless for the exact reason you point out: that it's ambiguous. Due to the nature of how an LLM works, it cannot make anything truly new, it, again, can only recombine and remix things from it's training data. This is why AI images/writing are so easily spotted, and even AI code to a degree: they have a distinct "style" as a consequence of how they work. I leave what is truly original to the philosophers. What I can say for certain, is that anything you get out of any of these models is a remix of what is present in it's training data, because gravity exists and energy is conserved. They are not intelligent, they are not conscious, and they cannot make New things. | | |
| ▲ | retsibsi 3 hours ago | parent | next [-] | | > What I can say for certain, is that anything you get out of any of these models is a remix of what is present in it's training data, because gravity exists and energy is conserved. But doesn't that same argument apply to humans, where 'training data' = all our sensory inputs (plus what's encoded in our DNA, depending on whether we want to analogise that to the training data or the underlying architecture)? > Arguing the definition of original is pointless for the exact reason you point out: that it's ambiguous. I'm not trying to be a smartarse here but you are the one who is claiming absolute certainty that something is false 'by definition', so I think the onus really is on you to have a precise definition in mind. > They are not intelligent, they are not conscious, and they cannot make New things. I'm not a consciousness denier or anything, in fact I think it is the most important thing in the world, but I don't understand the connections you're making here. 'Intelligent', like 'original', can be defined and needs to be defined for this statement to be meaningful. 'Conscious' is probably inherently slippery, but if we mean the same thing by it (having subjective experience/qualia) then I have no idea how you could be certain that any other entity lacks it. And it seems like you may be implying that consciousness is a necessary condition for originality, but I don't think you've explained how. | |
| ▲ | jonahx 3 hours ago | parent | prev | next [-] | | > Due to the nature of how an LLM works, it cannot make anything truly new, it, again, can only recombine and remix things from it's training data Fwiw, David Hume made this exact argument about human imagination: > But though our thought seems to possess this unbounded liberty, we shall find, upon a nearer examination, that it is really confined within very narrow limits, and that all this creative power of the mind amounts to no more than the faculty of compounding, transposing, augmenting, or diminishing the materials afforded us by the senses and experience. [David Hume — An Enquiry Concerning Human Understanding (1748), Section II, Paragraph 5] | |
| ▲ | WarmWash 3 hours ago | parent | prev | next [-] | | Everything is either deterministic, random, or a mix of that. New is just a mix of deterministic and random. LLMs are not deterministic. You don't even need an LLM to create something new and improved. Evolutionary algorithms have been doing it since the 80's. | |
| ▲ | hardbass 3 hours ago | parent | prev | next [-] | | Non answers are best ignored. You have failed to define what creativity is and therefore failed to prove whether llms are "creative" or not. What exactly are you trying to say about gravity and thermodynamics? You may be confused but these are universal physical laws that apply to everything, humans, computers, anything else. At its most basic level if I had to start with defining creativity, it should be some kind of intelligent filtering process. The brain can be in a huge number of state configurations, the interesting and novel among them are "creative" ideas. We can build upon it further. I am happy to say I am no philosopher and I will defer to those with expertise on it. But if you don't have any definition for it, its quite strange to make up some beliefs about it without proof. | |
| ▲ | drdeca 2 hours ago | parent | prev | next [-] | | You’re the one who said “ By definition an LLM cannot create original work”, so I think responding asking for the definitions you are talking about is quite justified? Often when someone says that something is true by definition, their claim is incorrect. | |
| ▲ | 3 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | brookst 2 hours ago | parent | prev | next [-] | | Does your same argument hold for humans’ inability to do more than remix? Gravity exists and energy is conserved for us, too. | |
| ▲ | CamperBob2 2 hours ago | parent | prev [-] | | Arguing the definition of original is pointless for the exact reason you point out: that it's ambiguous. Well, it certainly wasn't "ambiguous" two posts ago when you said "Absolutely not. By definition an LLM cannot create original work." Make up your mind. You've been beating this drum since roughly GPT 3.5, as I recall. Your goalposts aren't even on the field anymore, they're gathering dust in a far corner of the parking garage down the street from the stadium. Time to update your priors. |
|
| |
| ▲ | haickernews 3 hours ago | parent | prev | next [-] | | If you can be persuaded, looking into how modern LLMs are trained with reinforcement learning may change your views. | | | |
| ▲ | squidmaster 3 hours ago | parent | prev [-] | | [dead] |
|
|
|
| |
| ▲ | gauze 4 hours ago | parent | prev [-] | | Even after all the MPs are beat, there's still work to do to actually understand them, communicate them, and transform them in a way that can be leveraged. I predict that's going to be much harder for AI to solve. |
|