Hallucinations are a result of using statistical models on data. Not just transformers have this issue.
Interesting, I thought it was a feature of the transformer specifically.