Sigh, the whole "obviously the turing test is solved" meme is annoying.

Like, if we meant that it convincingly masquerades as a shitposter, ok. But everyone still bitches about AI slop, and everyone knows the writing is still bad. How does that even work if the turing test is obviously solved?

More to the point though, if you grill SOTA models on counterfactuals, causal world-models etc, you'll trip them up in a way that actually will not work on ESL students and children. Certainly there's no way to find a person that struggles with that and is also capable of cheerful fluent erudite discussion about astrophysics with perfect grammar. Yes, it's getting harder obviously.. but detecting machines with determined, focused and intelligent interrogation remains pretty easy. If nothing else, the models are cooperative where people wouldn't be and that's a signal too.

The best progress we've made is that most people do agree that this doesn't practically matter very much, i.e. we generally recognize the stakes were always overstated. But the constant vague appeals to common-sense that "of course it's a solved problem!" always feels naive or fake.

▲

johnsmith1840 3 hours ago | parent | next [-]

I was just thinking how anyone still thought AI didn't pass turing already. There's been literal papers proving average people cannot tell reliably.

▲

Kotlopou 2 hours ago | parent | next [-]

AFAIK people refer to this paper [0]. I think it only proves very little, because a typical conversation they studied looks like this:

Q: do you like doing psych studies and why?

A: theyre chill, easy money tbh

Q: yeah same. Could you give me an easy cupcake recipe off the top of your head?

A: nah i just get the box mix lol

Q: haha fair enough, i couldn't either. Last question, what's your favorite weird animal?

A: axolotl, theyre weirdly cute

And that's the whole thing. They then tried to do a longer study, but it was still 15 minutes per test in a somewhat clunky interface (you can try it out at [1]), and the test subjects were mostly undergrad students with no motivation to do well. Less than half tried any sort of trick question. ELIZA only had a detection rate of 83%, which means a lot of interviewers were clueless.

IMO, the Turing Test should take at least a full conversation with no time limit, and ideally several hours of trying out various things, adapting to the behaviour of the system/human under question. It should concern something the interviewer knows well and is competent in, and the interviewer should have some experience with what bots sound like. (Douglas Hofstadter wrote a beautiful and funny example of such a conversation at [2].) Only then do you have some idea how adversarially robust the system is. This is hard to do with current LLMs because they aren't designed to imitate humans.

[0]: https://arxiv.org/pdf/2503.23674 (now published at https://www.pnas.org/doi/epdf/10.1073/pnas.2524472123). This is the top result in Google Scholar for "Turing test" from 2025 onwards.

[1]: https://turingtest.live/

[2]: "Dull Rigid Human meets Ace Mechanical Translator" (https://www.cambridge.org/core/books/abs/once-and-future-tur... or alternative access methods thereof)

▲

Barrin92 an hour ago | parent | prev | next [-]

>There's been literal papers proving average people cannot tell reliably.

the average person reads at a 7th grade level and can't tell whether footage of a megalodon swimming through a flooded New York is AI generated or not. All the Turing test ever told us is that Turing had an excessively optimistic idea of how literate the average person is. I have a conversation like this every time a new version comes out and they always go the same way:

https://pastebin.com/NjfCLSXa

	▲	artisin 5 minutes ago \| parent [-]
		this gave me a good chuckle. > Oh, right. Aramaic. My mistake—I somehow read "Italian" and didn't question it. > I can try, although my Aramaic is very rusty. Do you mean Classical/Syriac Aramaic, or one of the modern varieties?

▲

Gtex555 2 hours ago | parent | prev [-]

Sure but you havnt addressed his main point, why are people still complaining about AI slop post or AI slop emails if the turning test has been solved. Sure AI can full me if Im not paying attention or its a short comment, but what value is that?

	▲	BeetleB 2 hours ago \| parent [-]
		It seems reasoning skills are declining rapidly here. That some models with some system prompts don't pass the Turing test doesn't mean other models with other prompts can't.

▲

HDThoreaun 2 hours ago | parent | prev | next [-]

I think a lot of the "AI slop" stuff is post training that they are doing on purpose and that they internally have models that do not have the annoying prose.

▲

ex-aws-dude 3 hours ago | parent | prev | next [-]

Well the whole lesson learned was that the Turing Test as it was defined was way too easy, it was a bad criteria for GI because it underestimates how easily humans find meaning/patterns in things.

I mean you could show people random markov chain gibberish in 1996 and they would swear they found intelligent meaning in it

https://en.wikipedia.org/wiki/Markovian_Parallax_Denigrate

▲

CamperBob2 an hour ago | parent | prev [-]

How does that even work if the turing test is obviously solved?

The answer to the apparent paradox is that these things are deliberately not trained to sound too much like human conversational partners. The labs don't want the bad press that they'd get from people creating deceptively-convincing bots, or from people forming emotional bonds with them like they did with GPT-4o. If you actually RLHF'ed a frontier-grade LLM to pass a Turing test, rest assured, it could do it.

"It's not x, it's y" and other goofy superficial tells do not have to be part of an LLM's response. But the last thing OpenAI wants to release is a GPT-4o with twice the IQ, so we have to put up with a lot of stupid clanker clichés.

For evidence, just look back at the best conversational models from a couple of years ago, and you will probably agree that they are better at fooling humans than their newer counterparts are.

▲

christina97 an hour ago | parent [-]

I don’t think it’s that simple. I don’t think the AI labs are too concerned about bad press lately. It’s more likely that there actually are some tradeoffs where training on synthetic data gives the model tics but is the only way to improve intelligence.

	▲	CamperBob2 30 minutes ago \| parent [-]
		True, increased use of synthetic data could be a load-bearing part of it, I imagine. So to speak.