Remix.run Logo
aizk 8 hours ago

People had joked a couple years ago "Well if they solve a Millenium problem it's AGI"... Well here we are.

20k 7 hours ago | parent | next [-]

Yeah well, its easy to do if you steal someone elses work and then try to threaten them into staying quiet about it

Edit:

OpenAI have now admitted they were training on prompts at the time they made their breakthrough:

https://mastodon.social/@tristanbuckmaster/11723647135247030...

logancbrown 6 hours ago | parent | next [-]

Steal someone elses work, whose work was also AI generated . . .

Lapra 4 hours ago | parent [-]

Whether that's relevant to the conversation depends on what their prompt was.

boshalfoshal 5 hours ago | parent | prev | next [-]

lol, the "other work" was also probably 95-99% AI generated. By a similar breed of OpenAI (and some Anthropic) models, as well.

I dont know why this monumental achievement is being drowned out by some arbitrary drama. No matter which way you slice it, AI solved this problem. Doesn't matter if it was some internal OpenAI model, or whether it was Astra + Fable.

demibabs 4 hours ago | parent [-]

Yeah but the mathematicians are claiming that the key insight that made the problem tractable for AI in the first place, came from them.

orangecat 4 hours ago | parent | prev | next [-]

What is that supposed to prove? OpenAI is almost always going to be training new models.

HDThoreaun 4 hours ago | parent | prev | next [-]

All the ai labs are open about training on prompts. The question is if buckmaster had disabled that with the toggle openAI provides.

20k 4 hours ago | parent | next [-]

That does not make it ethical

HDThoreaun 4 hours ago | parent [-]

Isnt the entire history of academic progress iterating on work that other academics shared with you? Obviously this situation is spicy but openAI cited their work no?

20k 3 hours ago | parent [-]

OpenAI trained on their private unpublished research notes effectively, while also trying to get one of the paper authors fired

daveguy 4 hours ago | parent | prev [-]

The question is more whether OpenAI can be trusted to honor that toggle switch. Given that they hold all chats for 30 days "for safety and security".

simianwords 6 hours ago | parent | prev [-]

what's there to admit? they always said they do it and there's a way to opt out. you are making it sound more dramatic than it is.

20k 5 hours ago | parent [-]

This is textbook plagiarism, scooping their result knowing that the research was part of the training data

matteoraso 5 hours ago | parent | prev | next [-]

Jokes aside, that's a horrible test for AGI. I like to think that I'm sentient, and I could never solve a millenium problem.

simianwords 7 hours ago | parent | prev [-]

> I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon

https://news.ycombinator.com/item?id=38433655

> Let's talk when we've got LLMs proving the Riemann Hypothesis (or any mathematical hypothesis) without any proofs in the training data. I'm confident in my belief that an LLM can't do that, and will never be able to. LLMs can barely solve elementary school math problems reliably.

https://news.ycombinator.com/item?id=42331654

> An LLM is like a well read college student with a nearly photographic memory that sometimes mixes things up. It's great for bouncing ideas off of and getting feedback on them. And yeah, it might product "novel ideas" by mixing and matching existing ideas, but LLMs will never create truly novel ideas. Not in their current form.

The paper didn't really answer the question sadly: their conclusion was just that humans rate LLM answers as more novel than human ones, but less feasible.

https://news.ycombinator.com/item?id=41522605

> Solving Millennium problems is a whole different ballgame. It's not known if these problems are solvable within ZFC axioms. (In one case, the Yang-Mills prize, stating the problem mathematically is part of the challenge.) All of the obvious applications of known tricks have been tried and failed. To solve such problems, one probably has to invent new and surprising mathematical definitions, building a framework in which the problem becomes solvable. This is something that LLMs will be crap at; the process of invention is not represented in any training data we have access to.

https://news.ycombinator.com/item?id=38435909

> LLMs cannot reason or use mathematics - in a way, they don't know what they are talking about. Why would such technology lead to superhuman smarts?

https://news.ycombinator.com/item?id=35752293

> But still, the questions in that test are "solved" in the sense of "I can take a dictionary and answers these questions with full certainty". Beyond established knowledge LLMs are monkeys with typewriters, at best.

> I agree but I have tried many times to intersect two ideas with a LLM that would be novel and the LLM can not do this at all. We shouldn't expect the stochastic parrot to be able to do this though and it is unfair to the stochastic parrot.

> It is like expecting a real parrot to say words it has never heard before.

> No one asks that of a real parrot because we don't anthropomorphize a real parrot like we do the LLM

https://news.ycombinator.com/item?id=41525962

WarmWash 7 hours ago | parent | next [-]

Will history look back at comments like these as people being dumb, or people trying to cope?

siva7 5 hours ago | parent | next [-]

It's denial and coping. Most people i see show this tendency around AI which is also why it 's easy to be far ahead of most population nowadays

cyclopeanutopia 4 hours ago | parent [-]

I'd say that believing to be "far ahead" is much deeper kind of coping.

siva7 4 hours ago | parent [-]

how i wish so..

keeda 4 hours ago | parent | prev | next [-]

A 3rd possibility is that they simply have not been exposed to the best models available (which is extremely likely if you only use the free tier chatbots), and/or did not invest the effort needed to truly harness this new very weird new technology, and so had a very skewed perspective of their actual capabilities.

stevenhuang 5 hours ago | parent | prev [-]

Both

rvz 7 hours ago | parent | prev | next [-]

You can see that your math friends completely wrote off LLMs entirely and were showing signs of coping.

4 years ago it was a "not yet" [0], since ChatGPT at this time was not ready nor it was "AGI". Now with this 'unreleased' AI model, it has reached a point where it has solved an unsolved problem which only one human solved a millennium prize problem (Poincare conjecture).

Now finally "AGI" means something again.

[0] https://news.ycombinator.com/item?id=33905609

quantumwoke 7 hours ago | parent | prev | next [-]

Some observations:

1. It seems at least possible that some of the proof of NS was contained in the training data, making it less novel.

2. The formalisation of mathematics into lean has been an underappreciated force multiplier on discovery.

kypro 7 hours ago | parent | prev [-]

As someone with a background in AI and who has been playing around with neural nets for decades at this point, it's been genuinely amazing watching extremely intelligent people make confident predictions about AI capabilities and progress, then be so completely wrong.

There's a kind of theory of mind for AI (specifically neural nets) which I now realise I seem to have which is very hard to explain to people who haven't felt the magic of these algorithms. In fact, the algorithmic details almost doesn't matter at all. When you have a generalised learning algorithm really the only essential components are – compute, data and time. So long as you can scale these you can be certain you will also scale capabilities. There is never any exception.

That said, the capabilities neural networks tend to progress in step-functions rather than scale in correlation with compute, data and time, because algorithmic improvements tend to come every ~5 years and bring a significant step change in capability (or efficiency depending on what you measure).

I think people like Dario and others working at frontier labs see and understand this very clearly. And I suspect it's also why they worry about AI risk because even if you ignore the significant increases in compute and data these models are being trained with, it's concerning that it only took two real algorithmic improvements to take us from mostly useless predictive language models to AGI-level intelligence – and we're due another step change.

6 hours ago | parent | next [-]
[deleted]
reducesuffering 6 hours ago | parent | prev [-]

> extremely intelligent people make confident predictions about AI capabilities and progress, then be so completely wrong.

The ability for the human mind to rationalize conclusions to maintain denial in the face of a very scary future is immense. Genuinely grappling with the implication of where we're headed is usually very crushing. It's not easy to engage with the possibility, and very intelligent people will use those smarts to feel safe.

kypro 3 hours ago | parent [-]

> Genuinely grappling with the implication of where we're headed is usually very crushing.

As someone currently prepping for various AI doom scenarios and who has been dealing with AI-related nightmares for years this is very relateable.

Although, I don't personally think it's this. In my experience the opposite is more true – the majority of high probability doomers seem rather laid back about considering what they believe will happen to the people they love in a few years. Equally I don't get the sense those who don't have such extreme predictions are worried at all. If anything there's not enough emotion.

In my opinion people just don't reason well when it comes to exponentials and are ignorant about things they don't have good mental models of. At least I know I struggle with this.