The "problem" of hallucination has not been solved, and never will be fully solved, because that's just how LLMs work. They're "hallucinating" 100% of the time but we only call it a hallucination when they're wrong.

But they're right much more often than they used to be. If you haven't tried a near-Frontier model lately (say, Claude Opus or ChatGPT Sol) you probably need to update your priors for just how often they can be right.

I only run models I can run locally. Perhaps I'm framing my question incorrectly.

AI is being used in contexts that involve life and death. Being right more often is not good enough in that context. If we are going to put AI in drones and have them autonomously acquire and fire on enemies, perhaps the fact that all AI hallucinates at all makes it morally and ethically unusable for purposes in which the stakes are very high, such as life and death.

This is what scares me about it. And also what I was able to accomplish air-gapped with a Ryzen 7 and RTX 3060 12GB scared me.

This is not the kind of thing we should all have to speculate about, someone should have solid answers. Perhaps the fact that some, and maybe most, AI models are black boxes even to those that trained them, should have been the first red flag and that was years ago.

One of the problems with this is companies and governments have invested 1.5 Trillion dollars on the bet that AI would revolution everything and reshape the world. AI companies lose money. All of the money invested is being recirculated among the major players in the game. It's not the people paying money for AI as a product, it's the companies wanting so badly for that to happen that they have created a spending bubble not too dissimlar to the sub-prime mortgage disaster that resulted in the housing bubble to burst. This is no different. AI companies have proven that they cannot make a profit at this point. There's 1.5 trillion at stake, and who do you think is going to be left holding the bag? My guess is the people.