That's actually pretty much what Andrej Karpathy mentions as a mitigation for hallucinations here:

https://m.youtube.com/watch?v=7xTGNNLPyMI&t=5400s