| |
| ▲ | CapmCrackaWaka 3 hours ago | parent | next [-] | | If his proof is 250,000 pages, then I am extremely dubious. Just today, I was working with about 1000 lines of code and asked opus 5.4 to make a change. It introduced an obvious bug, that actually ended up doing the exact opposite of what I asked to do( print out less lines of logging, instead of more).. even if he has access to some kind of super LLM, I am extremely skeptical that it can output a 250,000 page document that is coherent and bug free, no matter what this guy says. | | |
| ▲ | chunky1994 2 hours ago | parent | next [-] | | Its a formally verified proof. Its the same thing as saying your code compiled successfully (for a compiled language). If opus made a change and your code still compiled, you would certainly still say, my code compiled. The statement being asserted in lean is analogous to my code compiled. Lean works based on this: https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon... | | |
| ▲ | YeGoblynQueenne an hour ago | parent | next [-] | | >> The statement being asserted in lean is analogous to my code compiled. No, it's not. https://xenaproject.wordpress.com/2026/11/12/and-the-proof-i... | | |
| ▲ | chunky1994 an hour ago | parent [-] | | This article is saying the same thing I said with more words. To someone who does not understand what formal verification is, this detail is unnecessary. Lean certificates of this magnitude come with comparator. They are checked independently on different kernels to avoid kernel bugs. See the zulip thread: https://leanprover.zulipchat.com/#narrow/channel/113488-gene... This is exactly analogous to my code compiles (albeit with some extra flags on the compiler). |
| |
| ▲ | CapmCrackaWaka 25 minutes ago | parent | prev | next [-] | | Yes, my code compiled too, but there was still a bug in it. | | |
| ▲ | chunky1994 18 minutes ago | parent [-] | | I'll try to make it simpler, in formal verification (at the most base level): code compiles = the proof of the statement you have posed is correct, by definition. The statement you are trying to prove is: This is compilable code.
Since your code compiled, you know your code is compilable code.
Therefore your initial statement is correct. That's at the very basic level what a formal proof does. The expression you want to prove (in this case GRH) is stated in such a way that if your code compiles, you have proved the statement. |
| |
| ▲ | smallnix an hour ago | parent | prev [-] | | Until opus exploits a bug in my compiler to smuggle in invalid code | | |
| ▲ | mofeien 25 minutes ago | parent [-] | | I wish, but the proof is verified by another kernel ("con-leche") that is itself formally verified, Lean's own kernel, and two other kernels ("nanoda", "pestka") so far.
Joachim Breitner who's a Lean developer: "The high level summary is that this proof is now very very thoroughly checked." |
|
| |
| ▲ | ThePhysicist 3 hours ago | parent | prev | next [-] | | I would assume they do not produce the proof at once, and Lean is different from programming language (as far as I understand) in the sense that you can check if a program is correct or not within the Lean model, so it seems ideally suited for AI as they can experiment and extend the proof without ever producing a bug. I assume the real difficulty is checking that what you have proven is actually what you wanted to express but I would assume this is easier. Makes me wonder if general programming languages are too expressive for AI making it too easy to express conflicting or incorrect ideas, we might need something like Lean but for general purpose programming. I'm aware there are some formal systems for that but I think more along the lines of general purpose programming. | | |
| ▲ | lheck an hour ago | parent [-] | | I‘m doing a PhD in software verification right now and I agree. I’m not sure whether it’s about the power of the programming languages but more about their interfaces. Proof checkers like Lean are already what you describe, as Lean is also a programming language, but yes, there will be more general purpose tools for writing code that is correct by construction. The remaining hurdle is that you cannot actually verify correctness of many programs as you don’t know their full spec. But that is a very different problem to what we had before. |
| |
| ▲ | 3 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | cakeface 3 hours ago | parent | prev | next [-] | | Did you deliberately pick Opus 5.4 over Opus 5.5? That was the biggest thing that stood out to me in your comment. Opus 5.5 has been an all time banger for me. | | |
| ▲ | plorkyeran 2 hours ago | parent [-] | | Opus 5.4 doesn't exist, so they presumably meant either GPT 5.4 or Opus 5.5. |
| |
| ▲ | redbluered3 3 hours ago | parent | prev [-] | | You don't have access to the secret model the labs keep for themselves. Neither can you spin up thousands of subagents and burn billions of tokens a day. Proof can be total bunk but underestimating the labs is silly. |
| |
| ▲ | lossolo 3 hours ago | parent | prev [-] | | So how do we know if it does what it does? Anyone that is using frontier models for a few years, will tell you that they try to reward hack whenever they can. | | |
| ▲ | Frost1x 3 hours ago | parent | next [-] | | That’s the purpose of Lean. It’s kind of like asking: how do you know your CPython is running all the correct binary? Well, you kind of don’t, as it’s beyond human comprehension (in theory you could trace everything down to a certain level) but there are layers and layers of build abstraction that you assume are correct, compilers, runtime environments, and mature libraries that are checked and validated. Not a perfect analogy but that’s kind of the gist of it. | | |
| ▲ | lossolo 3 hours ago | parent | next [-] | | OK, but if I write code, I can verify it in a few simple steps. What about 250k pages of proof? How can I verify that page 133,432 has the correct assumptions and that it isn't reward hacked just to make lean code pass? How it's done? | | |
| ▲ | utkuumur 2 hours ago | parent | next [-] | | You don't need to! That is the beauty of Lean. You only need to check the problem definition (which is no where near 250k pages) and then Lean checks the proofs it self by checking whether each next line can be derived from previous lines. And for this problem, there are many validated problem definitions in Lean. So it is probably true. | | |
| ▲ | lossolo2 2 hours ago | parent [-] | | > and then Lean checks the proofs it self by checking whether each next line can be derived from previous lines Ah, so it's similar to something like Merkle trees, where each leaf must be cryptographically valid up to the root otherwise the whole branch is invalid? So you start with a simple definition that then evolves? | | |
| ▲ | solomonb 12 minutes ago | parent [-] | | The core idea is that you can map the types and terms of a type theory to the propositions and proofs of a formal logic Different type theories correspond to different systems of formal logic with different levels of power and expression. Simply Typed Lambda Calculus corresponds to intuitionistic propositional logic. If you lay out the typing judgements of STLC in natural deduction style and then physically erase the terms (leaving the types) then you literally have the rules of intuitionistic propositional logic. So if you can write a term that typechecks at a type, that term is a
proof of the corresponding proposition. The easiest example is modus ponens, which is exactly the typing rule
for function application: G |- M : A -> B G |- N : A
-------------------------------- (App)
G |- M N : B
"Given a function from A to B and a value of A, you get a value of B."So then in a proof checker you prove the proposition P => (P => Q) => Q by writing a
program of that type: apply :: a -> (a -> b) -> b
apply x f = f x
`apply` is demonstrated in haskell which does not have a rich enough type theory to give you more then an intuition (no pun intended) for how this all works. Things like `undefined` and general recursion let you "prove" any type, so it isn't a consistent logic. Languages like Lean and Agda are built on richer dependent type theories that give you far more expressive power for writing proofs (quantifiers, induction, etc.), and they won't let you cheat the type theory. |
|
| |
| ▲ | cman1444 3 hours ago | parent | prev [-] | | You can personally verify every binary digit of your code in a few simple steps? |
| |
| ▲ | prmph an hour ago | parent | prev [-] | | So Lean itself is proved? With what... Lean? A 250k pages proof is massive, even if correct, there is almost certainly something extremely inefficient or weird in there. I cannot imagine that a fundamental proof like this will require hundred of thousand of logic steps. Unless some part of the proof is about brute-force checking that something holds for a huge number of literal cases. |
| |
| ▲ | m11a 3 hours ago | parent | prev [-] | | In theory, if you formulated your proposition correctly, and Lean says it is proved, then the proposition is true. This works because of the Curry-Howard correspondence -- proofs can be represented as computation. | | |
| ▲ | lossolo 3 hours ago | parent [-] | | OK, so if I understand correctly, I formulate a proposition in natural language, then an LLM creates 250k pages of proof based on it, then it takes those 250k pages/lines of proof and creates 15 million lines of code to prove those 250k pages. Is that correct? If so, how can I be sure that my proposition is correctly translated into those 250,000 pages? Isn't it possible that the LLM can reward hack here in the same way I've seen it do in some cases where it struggles to pass the test and get its reward? For example, it tries to change assumptions without telling me because something doesn't work or it removes something or changes the test etc. | | |
| ▲ | rdlw 2 hours ago | parent | next [-] | | What assumptions could it change? The model should not be able to alter the definition of the RH, and if you want to prove to yourself that that definition is accurate, you just have to read through it and build a mental model of the Lean code: https://github.com/bhgomes/lean-riemann-hypothesis Then if it constructs a valid proof in Lean, how could it be reward hacking? I suppose it could have found a Lean compiler bug, or a mistake in the formalization of the hypothesis. I would expect that those problems would be found and fixed reasonably quickly, and we would need to see if the proof could be fixed. Other than that, I suppose it could find a contradiction in ZFC, but that would be an even more incredible result than proving RH. I don't really see the possibility for reward hacking with the stipulation that the proof is ultimately verified by compiling the Lean program. | |
| ▲ | hazbot 2 hours ago | parent | prev | next [-] | | > For example, it tries to change assumptions without telling me because something doesn't work or it removes something or changes the test etc. You have in Lean a headline statement of "given X, then Y is true". Where X and Y and precise statements in code. So the mathematical assumptions are upfront. If you trust the lean compiler and are very careful to vet that the headline statement is what you expect, then you know the theorem is true. How well the human language mechanics of the proof corresponds to what the actual lean mechanics of the proof is very much up on the air. | |
| ▲ | YeGoblynQueenne an hour ago | parent | prev | next [-] | | >> If so, how can I be sure that my proposition is correctly translated into those 250,000 pages? It's not the proposition (i.e. the theorem) you need to worry about but the proof. I'm sorry to say but the comments on here (and everywhere on the internets) that assure each other that "if it's veified by Lean, then it's true" misunderstand what they're even trying to claim. Here's a good article on the subject that was recently posted on HN: https://xenaproject.wordpress.com/2026/11/12/and-the-proof-i... And here's the tl;dr: >> This post is an attempt to explain in simple terms how to interpret claims of the form “this theorem is correct because it was checked in Lean”. The summary is: there is an important signal that can be extracted from Lean code, but if this code is AI-generated then you need to do some tests first. We explain the tests, and note that in many cases (such as the OpenAI theorem dump) these tests have not yet been systematically done. | |
| ▲ | chunky1994 3 hours ago | parent | prev | next [-] | | The 250k pages is the 15M LOC of lean proof. The code is the proof. | | |
| ▲ | 2 hours ago | parent | next [-] | | [deleted] | |
| ▲ | lossolo 2 hours ago | parent | prev [-] | | I see, so there is one step less, then how I can know if that 250k pages describe my proposition correctly and not something else? Lean code was written by LLM, correct? | | |
| ▲ | PhilipRoman 2 minutes ago | parent | next [-] | | Assuming Lean kernel is correct (and there are multiple independent implementations of it, some proved in other ways themselves), the only part up for debate is whether the Lean-translated theorem (roughly half a page of code, maybe a bit more if you fully expand all the libraries it uses) accurately models the original GRH The GRH is a bit too long to paste, but here is the ABC conjecture: def radical (n : ℕ) : ℕ := n.primeFactors.prod id
@[category research open, AMS 11]
theorem abc (ε : ℝ) (hε : 0 < ε) :
{(a, b, c) : ℕ × ℕ × ℕ | 0 < a ∧ 0 < b ∧ 0 < c ∧ ({a, b, c} : Set ℕ).Pairwise Nat.Coprime ∧
a + b = c ∧ (radical <| a * b * c : ℝ)^(1 + ε) < c}.Finite := by
sorry
not exactly beyond the real of human review | |
| ▲ | rdlw 2 hours ago | parent | prev | next [-] | | 250k pages describe the proof, not the proposition. You have to prove to yourself that the proposition is correctly described: https://github.com/bhgomes/lean-riemann-hypothesis | |
| ▲ | chunky1994 2 hours ago | parent | prev [-] | | In formal proof verification the only thing you need to verify is the problem statement expressed in lean, you don't need to check the lean code that is the proof. If the lean code that is the proof compiles, its a verification of the proof. |
|
| |
| ▲ | aseipp 2 hours ago | parent | prev [-] | | In theorem proving environments like Lean, you are equipped with a standard set of axioms and statements (eg set theory). Most people, as far as I am aware, agree with Set Theory. From these basic fundamentals, you can build up all of mathematics, just like a normal mathematician can. The trick is that in systems like Lean, proofs are properly type checked, like a programming language; and just like a programming language, those types help stop bugs in the proofs, but the types here are far richer, able to encode all of mathematics, and far more strict. To prove a mathematical statement, you must write a proof that is accepted by the type checker. Beyond that, the core "language" itself is generally specified to be "sound" -- that is, there is a very small subset of Lean, a tiny "kernel language", that is shown to be incapable of proving False, and thus any proof which is well typed in said language (ie it typechecks) is also sound. In other words the system is designed, by construction, to only rely on a very small "trusted" set of axioms, code, and mathematical ideas. This is a very engineering-oriented design principle! Actually, a pretty significant amount of the theoretical research in theorem proving over the past 2 decades has been in "automating proofs", ie tools and methods of making those proofs as easy as possible to write. Entire book chapters have been written about how to replace a 20 line inductive proof in Rocq with a single line "function call" that can automatically discharge the proof for you, because the idea is that this "trusted kernel", the tiny and sound mathematical language, will reject invalid proofs by design. So "who cares" what the proof is, in a sense? In many ways simply writing the theorems and whatnot down requires you to understand the problem well enough already. So in a way LLMs are just taking this idea to the absolute limit. So all this means is that the difficulty really is in specifying the theorems. Which was kind of the whole problem all along! Except now you have a computer that can check your actual proof. You can write a theorem in Lean that isn't the theorem you want, and then write a type checked proof of it, but you still need the theorem to be accurate. Unless Claude is either 1) finding bugs in the Lean kernel to exploit them, 2) using 'sorry' (a very convenient tool in Lean used as a placeholder for proofs, but it can literally allow you to prove False), then it mostly comes down to: are all the theorems stated correctly? This is trickier than it sounds, of course, so some skepticism is certainly warranted in my opinion! There are also other various engineering principles applied to help try to reduce bugs and ensure correctness like any other engineering project, and there are various other "things" regarding dependently typed programming language design and the impact of those design decisions on proofs (and why they are notoriously hard to read and write), but they aren't germane to my point here. |
|
|
|
|