Remix.run Logo
nsagent an hour ago

Can you speak to why in both cases, the problems OpenAI's models solved used the same techniques the mathematicians were exploring, which also happened to be niche approaches to the problem. As an NLP researcher myself, I find that coincidence highly suspect unless the models focused most of their attempts on the predominant approaches (they are trained for MLE after all).

tedsanders 40 minutes ago | parent | next [-]

I'm not a mathematician and I don't want to speculate about anything I can't back up. All I know about Navier-Stokes is from my graduate fluid dynamics class at Stanford a decade ago (where I received a poor grade). However, I don't want to leave you hanging, so what I will say is:

- I've heard some people say the model's solution is quite different from theirs (but I have no clue how to personally assess the spiritual truth of this, so please give it zero weight)

- Thousands of agents costing millions of dollars searched for ideas, and they were encouraged to explore a diversity of approaches, so it wouldn't be too surprising to me if the approaches they tried overlapped with other mathematicians', especially considering the models have knowledge of so much published math research

- This model has been beastly at solving all sorts of math problems (if it was Euler in particular, I'd agree that would look suspicious/lucky)

- The Euler regularity disproof itself took ~100 agents working for ~50 hours (if it was very quick, and then the subsequent NS work took a long time, I'd agree that would look suspicious/lucky)

I understand the skepticism, but from what I know internally at OpenAI, we have zero reason to believe our models did anything fishy. It's hard for us to prove a negative, especially when you have to take us at our word, so I understand why people still feel suspicious.

Edit: Reminds me a bit of the Scarlet Johansson voice cloning accusations and FrontierMath eval training accusations, where the rumors of misbehavior seemed to travel faster than the truth. In both of those cases, we hadn't done what was accused, but suspicions persisted nonetheless.

fhub 35 minutes ago | parent | prev [-]

I think for OpenAI to win back some hearts and minds here we should have the option to retrospectively turn off "Help improve our AI models". i.e. Any new model trained would exclude all those user's sessions. This could be technically hard but I'm sure an intelligent AI model could work out how to do it :-)

ChatGPT agrees with this too.

https://chatgpt.com/share/6aa31959-b0e8-83ec-bee6-851ed18d45...