Obviously without a proper RAG pipeline or web search it is just going to hallucinate with maximum confidence. It would be more interesting to see the results with top tier models that have proper alignment to refuse to answer when token probability is low. As it is they just proved that people tend to trust well written text in a chat ui