Remix.run Logo
tristanj 3 hours ago

Full mastodon post: https://mathstodon.xyz/@tristanbuckmaster@mastodon.social/11...

Mathematical explanation by Terrance Tao: https://mathstodon.xyz/@tao/117233527638291447

It seems there is much background drama behind this, and this is what I've pieced together of what happened:

Over the past year, Buckmaster and Alpöge have been using AI to work on fluid dynamics maths problems. Alpöge works at Anthropic, which will cause future issues.

In mid-August, they found a counterexample for a simpler version of the Navier-Stokes problem. They spend the next few weeks preparing their paper.

In early September, rumors start spreading on X that Anthropic has solved a Millennium prize problem (and that it's Navier-Stokes). Buckmaster reaches out to OpenAI to explain this is their own personal research, not an Anthropic project.

A few days later, OpenAI gets back to him, and tells him an internal model found has a counterexample for Navier–Stokes, potentially worth the $1 million Millennium prize. The proof uses the same method that Buckmaster and Alpöge chose to work on. They don't show him the proof.

Buckmaster pressed them for more details. OpenAI reveals they had an entire team had been working on the problem, and that they started work in the past few days, after the rumors that Anthropic had solved a Millennium prize problem.

Buckmaster says OpenAI talked about a shared publication timeline. They want to Buckmaster to publish first, then give Buckmaster shared credit for the Millennium Prize when they publish the full result. But they want to exclude Alpöge as an author because he works at Anthropic. An agreement is not reached. Buckmaster had been using OpenAI Codex to draft/check his work, and asks if his private AI chats were used to accelerate OpenAI's result.

Buckmaster and Alpöge think they have found a counterexample for Navier-Stokes, but the paper is not yet presentable. It's unclear what date they found this result.

Because of the situation with OpenAI, they published their existing papers earlier than planned (today), alongside this statement announcing they have a tentative result on Navier-Stokes and revealing the OpenAI drama.

The post is missing context from both sides, and this isn't my field, so hopefully someone else can unpack what's happening here.

akersten 3 hours ago | parent | next [-]

> They were coordinating with OpenAI regarding a publishing timeline, but could not come to an agreement,

Skimming the PDFs it seems much more dramatic than that? It sounds like at least one of them is concerned OpenAI "solved" the problem by having their internal model use the chats of the independent researchers and want to claim the credit instead? I don't know. The tone is pretty accusational though:

> the one Levent and I had quietly chosen to attack. Almost nobody else I know of was working on it. It is not the direction one arrives at in a few days by giving a model the problem statement. When I heard “forced,” it was a bright red flag.

> I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used.

> I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.

> I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer. [0]

[0]: https://cims.nyu.edu/~tristanb/statement.pdf

tristanj 3 hours ago | parent | next [-]

Yeah these are major accusations. But the story is incomplete, the conversation is missing a lot of details. It's not clear who was working on what, and when. The entire thing feels rushed, like they wanted to get this result published and out the door quickly.

3 hours ago | parent [-]
[deleted]
cma 2 hours ago | parent | prev [-]

Does he claim to have opted out of training too?

kzrdude an hour ago | parent [-]

There are various forces at play here, academic honesty requires them to disclose any inputs regardless of license or ToS circumstances.

While common sense reminds us here that if you send your data to an external entity’s computer, you are no longer in control of said data. The lines have blurred here clearly over the last decade, but that should have made the theory yet more clear to everyone involved: your data will be vacuumed up unless you keep it sealed. Use your own computer if you want to be in control.

dumberquestions 3 hours ago | parent | prev | next [-]

>...has solved a millennium problem and is sitting on the result

Someone correct me if I'm wrong, but the work involved here is not the actual millennium problem, but it concerns versions with an added external force that the author thinks is a path that may help toward solving the harder unforced problem.

modeless 2 hours ago | parent [-]

Apparently forcing is allowed in the Millenium Prize problem statement. So OpenAI's claimed proof could win the prize. OTOH the results Tristan and Levent are publishing here do not go far enough to win the prize, though apparently they are suggestive of a general approach that could produce a solution, which seems likely to be the general approach OpenAI's proof uses.

The question is whether OpenAI's pursuit of this direction happened spontaneously, or as a result of them learning about Tristan's work somehow. To be clear, while the tone of this post seems quite accusatory, Tristan does not claim to know for sure whether OpenAI unfairly benefited from his work. Sholto Douglas from Anthropic is also on record saying the suggestion that OpenAI used Tristan's codex transcripts somehow is extremely unlikely to be true[1], which I agree with, though it doesn't rule out them learning of Tristan's work some other way. I am sure OpenAI will have a statement out tomorrow clarifying their position.

[1] https://x.com/_sholtodouglas/status/2097218240397410733

dumberquestions an hour ago | parent [-]

Why do you find it to be extremely unlikely?

MathmoKiwi 7 minutes ago | parent | next [-]

As modeless said, Sholto Douglas works for Anthropic!

So to be fair, if Anthropic is *also* doing this (quite likely!) then Sholto would have a very strong incentive to try and spin it as highly unlikely that any of the big AI labs are possibly doing this.

tristanj 34 minutes ago | parent | prev [-]

Because very few people actually have access to these logs, all access is monitored and recorded, and improper access will get you fired. It's not worth risking your job over something like this.

dumberquestions 22 minutes ago | parent | next [-]

Not that I have strong reasons to think this is not true, but what reasons do we have to think it is? Has this been audited before?

MathmoKiwi 8 minutes ago | parent | prev [-]

There is almost zero risk to your job (quite the opposite, you might be richly rewarded!) if you're simply doing something here which the company wants done. (remember, billions and billions of dollars are at stake here! Do you really think there is no chance at all they would do it??)

instagraham an hour ago | parent | prev [-]

> A few days later, OpenAI gets back to him, and tells him an internal model found a counterexample for Navier–Stokes

Why is OpenAI chatting with him at all at this stage? Is the discussion along the lines of "hey we used the work you are famous for to do a bigger piece of work, just thought you should know" or "heyyy....so we kinda liked what you were typing in your private chat, and thought we'd develop those ideas a bit. and yeah we solved Navier-Stokes in the process. But it's our finding, so do you want like an honorary acknowledgement or do you want to go to court?"

traes an hour ago | parent [-]

Perhaps out of a sense of academic good will, knowing that he got there first?

It seems like the timeline according to OpenAI is that:

1. Buckmaster developed a counterexample to a reduced version of Navier-Stokes with Anthropic employee Levent

2. Rumors start spreading that Anthropic has solved Navier-Stokes

3. OpenAI learns this and starts throwing a ridiculous amount of compute at it, now knowing it's within reach of LLMs

4. Their LLMs (with human assistance) get FARTHER than Buckmaster, using the exact same method.

5. OpenAI reaches out to Buckmaster to negotiate a fair way to publish both results and properly assign credit

Perhaps they simply and honestly feel he is owed credit. I can't help but imagine it at least played some small part. I expect that's what Bubeck is going to claim: https://xcancel.com/SebastienBubeck/status/20972141224714323...