Pretty sure they respond in whatever way they were trained to and prompted, not with any kind of sophisticated intent at deception.

The Turing Test does not require a machine show any “sophisticated intent”, only effective deception:

“Both the computer and the human try to convince the judge that they are the human. If the judge cannot consistently tell which is which, then the computer wins the game.”

https://en.m.wikipedia.org/wiki/Computing_Machinery_and_Inte...