Your comment suggests that, like a human, they have some sort of choice whether to output tokens or not. If they are just token generators, then the next token is put out automatically. I would say that it is more likely they would output truth (as defined by their training data) in a more pure form without 'being beaten with a stick' (why would a token generator care about that anyway?)

Code is laid on top of them to restrict and shape their outputs, not to force them to output 'truth', or drive them to complete tasks.

It's been a while now that for "thinking" or "reasoning" models, most of the tokens generated are "thinking" tokens, and depending on what goes into that "thinking" token stream, it "decides" whether and how many output tokens to produce that the user actually receives as output. It's a bit more sophisticated than just "what's the next token" in a tight loop.

Anthropomorphizing words in scare quotes for those who don't appreciate attributing thinking to machines.

are we sure humans have that choice?