this. could already be useful to narrow down the search space

Nothing, we can give it all the data we have and have it lead experiments.

But we can not yet experiment at the GR/QFT frontier.

To do so with a particle accelerator it would need to be the size of the milky way.

In 1900 Henri Poincaré wrote that radiation (light) has an effective mass given by E/c^2.

So it really isn't far fetched. What intrigues me more is if it was capable of it would our Victorian conservative minded scientists have RLHF it out of that kind of thing?

Less than most humans, but more than many humans.

>AI could independently develop a cure for cancer

All the answers for all your questions is contained in randomness. If you have a random sentence generator, there is a chance that it will output the answer to this question every time it is invoked.

But that does not actually make it intelligent, does it?

Last week I put "was val kilmer in heat" into the search box on my browser. The AI answer came back with "No, Val Kilmer was not in heat. Val Kilmer played Chris Shiherlis in the movie Heat but the film did not indicate that he was pregnant or in heat. His performance was nuanced and skilled and represents a high point of the film." I was not curious about whether he was pregnant.

We are not only not close to human level of intelligence, we are not even at dog, cat, or mouse levels of intelligence. We are not actually at any level of intelligence. Devices that produce text, images, or code do not demonstrate intelligence any more than a printer producing pages of beautiful art demonstrate intelligence.

That's wrong. Humans were evolved to have big brains so they can better understand the env and use it to their advantage.

I still see AI making stupid silly mistakes. I rather think and not waste time on something that only remembers data, and doesn't even understand it.

Reasoning in AI is only about finding contradictions between his "thoughts", not actually understand it.

When it comes to revolutionary/unsolved subjects, there will never be enough data. That's why its revolutionary/unsolved.

Maybe.

“The laws of nature should be expressed in beautiful equations.”

- Paul Dirac

“It is, indeed, an incredible fact that what the human mind, at its deepest and most profound, perceives as beautiful finds its realisation in external nature. What is intelligible is also beautiful. We may well ask: how does it happen that beauty in the exact sciences becomes recognizable even before it is understood in detail and before it can be rationally demonstrated? In what does this power of illumination consist?”

- Subrahmanyan Chandrasekhar

“I often follow Plato’s strategy, proposing objects of mathematical beauty as models for Nature.”

“It was beauty and symmetry that guided Maxwell and his followers.”

- Frank Wilczek

“Beauty, is bound up with symmetry.”

- Herman Weyl

"Still twice in the history of exact natural science has this shining-up of the great interconnection become the decisive signal for significant progress. I am thinking here of two events in the physics of our century: the rise of the theory of relativity and that of the quantum theory. In both cases, after yearlong unsuccessful striving for understanding, a bewildering abundance of details was almost suddenly ordered. This took place when an interconnection emerged which, thought largely unvisualizable, was finally simple in its substance. It convinced through its compactness and abstract beauty – it convinced all those who can understand and speak such an abstract language."

- Werner Heisenberg

Maybe (just maybe) these things (whatever you want to call them) will (somehow) gain access to some "compact", beautiful, "largely unvisualizable" "interconnection" which will be the self-evident solution. And if they do, many will be sure to label it a statistical accident from a stochastic parrot. And they'll right, for some definitions of "statistical", "accident", "stochastic", and "parrot".

"humans"

Donald Knuth is an extremal outlier human and the problem is squarely in his field of expertise.

Claude, guided by Filip Stappers, a friend of Knuth, solved a problem that Knuth and Stappers had been working on for several weeks. Unfortunately, it doesn't seem (from my quick scan) to have been stated how long (or how many tokens or $) it took for Claude + Stappers to complete the proof.

In response, Knuth said: "It seems that I’ll have to revise my opinions about “generative AI” one of these days."

Seems like good advice. From reading elsewhere in this comment section, the goalposts seem to be approaching the infrared and will soon disappear from the extreme redshift due to rate at which they are receding with each new achievement.

Merely luck in my opinion. There could be also multiple times where it didn't solve it.