I love AI and use it daily, but I still run into hallucinations, even in COT/Thinking. I don't think hallucinations are as bad as people make it out to be. But I've been using AI since GPT3, so I'm hyper aware.

Yea. I think people underestimate this. Yesterday I was writing an obsidian plugin using the latest and most powerful Gemini model and I wanted it to make use of the new keychain in Obsidian to retrieve values for my plugin. Despite reading the docs first upon my request it still used a non existent method (retrieveSecret) to get the individual secret value. When it ran into an error, instead of checking its assumptions it assumed that the method wasnt defined in the interface so it wrote an obsidian.shim.ts file that defined a retrieveSecret interface. The plug-in compiled but obviously failed because no implementation of that method exists. When it understood it was supposed to used getSecret instead it ended up updating the shim instead of getting rid of it entirely. Add that up over 1000s of sessions/changes (like the one cursor has shared on letting the agent run until it generated 3M LOC for a browser) and it's likely that code based will be polluted with tiny papercuts stemming from LLM hallucinations