Even the latest models are quite easily fooled about if something is true or not, at which point they then confidently declare completely wrong information to be true. They will even strongly debate with you when you push back when you say hey that doesn’t look right.

It’s a significant concern for any sort of AI use at scale without a skilled and knowledgeable human expert on the subject in the loop.