> Large language models (LLMs) have astounded the world with their capabilities, yet they remain plagued by unpredictability and hallucinations – confidently outputting incorrect information. In high-stakes domains like finance, medicine or autonomous systems, such unreliability is unacceptable.

This misses a point that software engineers initmately know especially ones using ai tools:

* Proofs are one QA tool

* Unit tests, integration tests and browser automation are other tools.

* Your code can have bugs because it fails a test above BUT...

* You may have got the requirements wrong!

Working with claude code you can have productive loops getting it to assist you in writing tests, finding bugs you hadn't spotted and generally hardening your code.

It takes taste and dev experience definitely helps (as of Jan 26)

So I think hallucinations and proofs as the fix is a bit barking up the wrong tree

The solution to hallucinations is careful shaping of the agent environment around the project to ensure quality.

Proofs may be part of the qa toolkit for AI coded projects but probably rarely.