GPT-3 might not have said “load bearing” as much, but a savvy interrogator could still catch it out nearly every time just asking dumb gotcha questions like, “How many Rs are there in strawberry?”

▲

famouswaffles 16 hours ago | parent [-]

Those are questions that are sidestepped with simply a different input paradigm than BPE tokenization. See the Byte Latent Transformer - https://arxiv.org/pdf/2412.09871 - where a similar scale byte latent model trained on the same dataset >>> a vanilla transformer on word and character manipulation tasks.

For example, Llama 3 trained on 1T tokens scores 1.1% on a CUTE spelling benchamrk, while the equivalent byte latent equivalent trained on the same dataset scores 99.9%. Another example is 0.4% vs 48.7% on a Substitute Char benchmark.

It all falls down to the same thing. Researchers are not optimizing for passing as a human.

▲

bunderbunder 15 hours ago | parent [-]

So, sure, we can special plead the Turing test out of the picture. But if we don’t propose an alternative to take its place, we’re left right back at the same silly situation that the top level commenter was observing and that Turing was trying to move away from: enmired in a useless, meaningless argument about semantics.

▲

famouswaffles 15 hours ago | parent [-]

The Turing Test itself in the form exactly envisioned is not important. You don't need an alternative 'special test'. You just need to understand the point Turing was making. That the question itself is an irrelevant one by its very nature. People that want to be enmired in meaningless semantic debates will continue to do that, no matter what test you devise.

	▲	14 hours ago \| parent [-]
		[deleted]