> And then they come up with another thing that needs to be solved in order to prove it is important/hard/impressive. And once that happens, they do it again. And again. That's what "moving the goalposts" means.

The fundamental argument that I've personally made since the early days of this is that LLMs are not reasoning, in the way that word is commonly understood.

There are lots of reasons why that argument needs to evolve that could certainly appear to be "moving the goalposts", but let's take an example.

A lot of AIs were tripped up by the question "Should I walk or drive 50m to the carwash?" Several folks liked to use that as an example that illustrates that LLMs aren't reasoning, but as the models have been trained on that specific example, it's of course less useful. An AI can mostly nail it now.

So a different example is needed. A new demonstration of how these things fail at basic reasoning a child can do.

Did I move the goalposts? I don't think so. The fundamental argument stays the same. It's not hard to find lots of examples that trip up LLMs, because they are what they are: statistical inference machines. Nothing more and nothing less.

Useful, sure. But also commonly misapplied to areas for which they are inappropriate solutions.