| ▲ | lossolo 36 minutes ago |
| So how do we know if it does what it does? Anyone that is using frontier models for a few years, will tell you that they try to reward hack whenever they can. |
|
| ▲ | Frost1x 28 minutes ago | parent | next [-] |
| That’s the purpose of Lean. It’s kind of like asking: how do you know your CPython is running all the correct binary? Well, you kind of don’t, as it’s beyond human comprehension (in theory you could trace everything down to a certain level) but there are layers and layers of build abstraction that you assume are correct, compilers, runtime environments, and mature libraries that are checked and validated. Not a perfect analogy but that’s kind of the gist of it. |
| |
| ▲ | lossolo 21 minutes ago | parent [-] | | OK, but if I write code, I can verify it in a few simple steps. What about 250k pages of proof? How can I verify that page 133,432 has the correct assumptions and that it isn't reward hacked just to make lean code pass? How it's done? | | |
| ▲ | cman1444 7 minutes ago | parent [-] | | You can personally verify every binary digit of your code in a few simple steps? |
|
|
|
| ▲ | m11a 34 minutes ago | parent | prev [-] |
| In theory, if you formulated your proposition correctly, and Lean says it is proved, then the proposition is true. This works because of the Curry-Howard correspondence -- proofs can be represented as computation. |
| |
| ▲ | lossolo 16 minutes ago | parent [-] | | OK, so if I understand correctly, I formulate a proposition in natural language, then an LLM creates 250k pages of proof based on it, then it takes those 250k pages/lines of proof and creates 15 million lines of code to prove those 250k pages. Is that correct? If so, how can I be sure that my proposition is correctly translated into those 250,000 pages? Isn't it possible that the LLM can reward hack here in the same way I've seen it do in some cases where it struggles to pass the test and get its reward? For example, it tries to change assumptions without telling me because something doesn't work or it removes something or changes the test etc. | | |
|