> The sources it provides when it decides to use RAG are there more for instilling confidence in the answer instead of encouraging their users to check its work.

This is an insightful point. I had an exchange with someone on the internet a few days ago; he wanted to argue something and used ChatGPT to "find sources" ("I'm at work and it's quicker"). It did its search, found a page with a figure, dutifully gave him that figure and a link, and he posted it.

The figure was, if you thought about it for a second, verging on ludicrous, and when prompted about it he linked the original source - it was a blog post that was evidently slop and the figure was just something that the original AI tasked to write that post had made up! The "research" was just a non-human centipede but he was perfectly happy to trust it and post it because well ChatGPT has gone and found a source so it must be true. Thought terminating.