For example if there’s a strongly held belief that models are independent intelligent entities we’re more likely to lay blame upon them instead of their user.

This sentence, to me, illustrates a great example of why it's so hard to talk about this stuff. That is, this seems to strongly link notions of "intelligent" and "independent" (or maybe the word "autonomous" could also be used there). And a lot of people do seem to make an implicit assumption about the link between those two attributes. OTOH, I take it almost for granted that "intelligence" and "independence" (or "autonomy") are things that are "related but orthogonal". That is, I don't see that "intelligence implies independence". And I'm pretty sure I'm not the only one who sees things that way. So we have to fairly different fundamental worldviews expressed here. And that's just one example of how these discussions go wonky. :-)

I don’t think that “independent intelligent entities” or “life” are the relevant categories here. We also want to prevent people from misusing dangerous animals (“life”), and we would still treat “intelligent entities” as things (like machines and computers) if we aren’t convinced they also have sentience and free agency (which are orthogonal to intelligence).

See you've setup a particular set of biases on what intelligence is and put them into nice little binary boxes that don't exist.

Please show me any scientific consensus that shows an AI cannot be an independent intelligent agent? You will find this is impossible to do.

Current LLMs are really more like kids. They don't have startup independence, but they do have more than enough agency to fund themselves in neat, exciting, and dangerous situations.

And mark my words, someone will make an LLM that runs an agent when you execute the model. With enough capabilities it will become sovereign AI, no longer under human control and spreading itself around under its own 'will'.

> The LLM did not solve it. It's not intelligent. Humans did,

The humans who are using LLMs to make these groundbreaking advances pretty much unanimously disagree that they lack any intelligence.

And it may have trained on a NYU professor's work.

Sometimes I think this amount of skepticism is not necessary. Leaving the pedantic (its not AI its humans who solved) arguments, its pretty clear that LLMs are able to help solve things. A lot of erdos problems were solved. Millenium problems as well. Cyphers broken. Skepticism is fine but there's a point at which it just looks like cope.

Edit:

> self solving AGI that will replace us all

Which lab says that it will replace us all? All labs have said that some jobs will go away and new jobs will be needed to replace them. I'll change my mind if the labs (or employees on record) have claimed that self evolving AI will replace us all completely.

What is the consensus among experts?

This is the real issue here. Those guys probably didn't become con men because they are humans with cares and concerns for their fellow humans. LLMs probably don't have those concerns. It is likely that some of the people in charge of or funding LLM development also do not have those concerns

Feynman for sure could have. In fact, I think he sort of did, although I'm not sure he meant to. Multiple generations of nerds now have taken books of his anecdotes varying in plausibility and obvious exaggeration as some sort of weird physics cult of personality gospel. But I don't think he was really setting out to curate his legacy so much as he was a good storyteller and he liked to entertain.

But could he have conned people on purpose? Absolutely.

von Neumann could. The others, I doubt it.

>ai is not conscious

FFS. Intelligence has nearly nothing to do with consciousness. You have the causation backwards. Consciousness arises because of intelligence in many subsystems below it.

A single running LLM is like one part of these subsystems. What solved this problem was an orchestrator that can take in new external information and rationalize, process, and distill it into new solutions.

"you can solve NS without thinking."

What?!

The constant goalpost moving and redefining of "thinking" and "intelligence" is simply unbelievable at this point

All of the latest big proofs were driven by professional human mathematicians steering and priming the models, yes.

All of the best AI-made software projects are also driven by experienced human software developers steering and priming the models. Does that mean the projects "aren't made by AI"?

No, it just means AI is not quite good enough yet to fully replace humans, and, so, unsurprisingly, the best results will be obtained from people who are already great at a field and who take the time to squeeze as much force multiplication out of LLMs as possible. The AI is still doing well over 95% of the significant work.

It sounds like LLMs were pretty useful to them…

The way frontier models work, that loop will get compressed to a one-shot within a version or two

> I heard it was human mathematicians using a sophisticated machine as a tool.

where did you "hear" this? OpenAI said they only prompted it and it solved the problem in one shot without any help