I think we keep changing the goalposts on AGI. If you gave me CC in the 80's I would probably have called it 'alive' since it clearly passes the Turing test as I understood it then (I wouldn't have been able to distinguish it from a person for most conversations). Now every time it gets better we push that definition further and every crack we open to a chasm and declare that it isn't close. At the same time there are a lot of people I would suspect of being bots based on how they act and respond and a lot of bots I know are bots mainly because they answer too well.

Maybe we need to start thinking less about building tests for definitively calling an LLM AGI and instead deciding when we can't tell humans aren't LLMs for declaring AGI is here.

> I think we keep changing the goalposts on AGI

Isn't that exactly what you would expect to happen as we learn more about the nature and inner workings of intelligence and refine our expectations?

There's no reason to rest our case with the Turing test.

I hear the "shifting goalposts" riposte a lot, but then it would be very unexciting to freeze our ambitions.

At least in an academic sense, what LLMs aren't is just as interesting as what they are.

Turing himself argued that trying to measure if a computer is intelligent is a fool's errand because it is so difficult to pin down definitions. He proposed what we call the "Turing test" as a knowable, measurable alternative. The first paragraph of his paper reads:

> I propose to consider the question, "Can machines think?" This should begin > with definitions of the meaning of the terms "machine" and "think." The > definitions might be framed so as to reflect so far as possible the normal use > of the words, but this attitude is dangerous, If the meaning of the words > "machine" and "think" are to be found by examining how they are commonly used > it is difficult to escape the conclusion that the meaning and the answer to the > question, "Can machines think?" is to be sought in a statistical survey such as > a Gallup poll. But this is absurd. Instead of attempting such a definition I > shall replace the question by another, which is closely related to it and is > expressed in relatively unambiguous words.

Many people who want to argue about AGI and its relation to the Turing test would do well to read Turing's own arguments.

I don't think the goalpost has been shifted for AGI or the definition of AGI that is used by these corporations. It's just they broke it down to stages to claim AGI achieved. It was always a model or system that surpasses human capabilities at most tasks/being able to replace a human worker. The big companies broke it down to AGI stage 1, stage 2, etc to be able to say they achieved AGI.

The Turing Test/Imitation Game is not a good benchmark for AGI. It is a linguistics test only. Many chatbots even before LLMs can pass the Turing Test to a certain degree.

Regardless, the goalpost hasn't shifted. Replacing human workforce is the ultimate end goal. That's why there's investors. The investors are not pouring billions to pass the Turing Test.

I don't think so... I think most of the sci-fi I grew up reading presented AGI that could reason better than humans could, like make a plan and carry it out.

Like do people not know what word "general" means? It means not limited to any subset of capabilities -- so that means it can teach itself to do anything that can be learned. Like start a business. AI today can't really learn from its experiences at all.

Related: https://en.wikipedia.org/wiki/AI_effect

The truth is, we have had AGI for years now. We even have artificial super intelligence - we have software systems that are more intelligent than any human. Some humans might have an extremely narrow subject that they are more intelligent than any AI system, but the people on that list are vanishing small.

AI hasn't met sci-fi expectations, and that's a marketing opportunity. That's all it is.

The Turing test pits a human against a machine, each trying to convince a human questioner that the other is the machine. If the machine knows how humans generally behave, for a proper test, the human contestant should know how the machine behaves. I think that this YouTube channel clearly shows that none of today's models pass the Turing test: https://www.youtube.com/@FatherPhi

> Maybe we need to start thinking less about building tests for definitively calling an LLM AGI and instead deciding when we can't tell humans aren't LLMs for declaring AGI is here.

If you've never read the original paper [1] I recommend that you do so. We're long past the point of some human can't determine if X was done by man or machine.

[1]: https://courses.cs.umbc.edu/471/papers/turing.pdf

People thought Eliza was alive too in the 60s. AGI is not determined by how ignorant, uninformed humans view a technology they don't understand. That is the single dumbest criterion you could come up with for defining it.

Regarding shifting goalposts, you are suggesting the goalposts are being moved further away, but it's the exact opposite. The goalposts are being moved closer and closer. Someone from the 50s would have had the expectation that artificial intelligence ise something recognisable as essentially equivalent to human intelligence, just in a machine. Artificial intelligence in old sci-fi looked nothing like Claude Code. The definition has since been watered down again and again and again and again so that anything and everything a computer does is artificial intelligence. We might as well call a calculator AGI at this point.

The goal post keeps moving because LLM hypeists keep saying LLMs are "close" to AGI (or even are, already). Any reasonably intelligent individual that knows anything about LLMs obviously rejects those claims, but the rest of the world doesn't.

An AGI would not have problems reading an analog clock. Or rather, it would not have a problem realizing it had a problem reading it, and would try to learn how to do it.

An AGI is not whatever (sophisticated) statistical model is hot this week.

Just my take.

Sure, in the 80s after interacting with CC 1 time you would call it 'alive'. After having interacted with it for 5-10 minutes you would clearly see that it is as far from AGI as something more mundane as C compiler is.

Maybe moving the goalposts is how we find the definition?

By that measure Eliza might pass the turing test too. It just shows it's far from being a though-terminating argument by itself.