Nit: Turing’s actual imitation game is a party game (like Werewolf/Mafia) and nobody’s even trying to win at that. The LLM’s will just tell you they’re an AI.

▲

idiotsecant 2 hours ago | parent [-]

This is an artifact of how we deliberately craft these models though. We could just as easily fine tune a model that will believe it is not an AI or will attempt to deceive users asking about it

	▲	skybrian 2 hours ago \| parent [-]
		There's a lot more to it. For example, its writing style would also have to improve so it doesn't immediately give itself away. Also, the skill of the human opponents matters. You'd want to test it against people who have practiced playing the game. Otherwise, it's like the difference between building a chess bot that can win against random undergrads who don't normally play, versus winning against grandmasters. And it's not like there's a pool of skilled human players of the imitation game.