I can't remember who it was I spoke to recently, but it was some startup for doing code reviews. So I asked how they're handling larger diffs and especially when a code change is related to something requiring knowing a lot of context. Will it hallucinate things that make the review low-quality, etc. I was told "no it's been given guardrails to prevent hallucinations. it doesn't hallucinate." but I was thinking, that's not really how LLMs do or do not hallucinate, is it? (I'm not an AI person so I really don't know).

This is a good/fast/cheap tradeoff, but in the form of accurate/precise/creative, or something in that ballpark anyway.

You can constrain the AI to be more accurate by having it a) equivocate (lose precision) or b) turning the temperature to effectively zero, thus losing "intelligence" and creativity.

There's no such thing as a free lunch. The models perform best out of the box, and everything you add into context makes them perform worse generally, but more likely to do what you want. Every token is a burden, but without those constraints the generation won't do anything.

So by that standard, hallucination is something completely core - you can't get rid of it without losing what makes them useful. You can make different tradeoffs, or use a very smart model in a way that is completely locked down (or adversarially verified, same result) which results in far more tokens used than if you just let the model do it's thing and validated for hallucination independently, outside of the generation loop.

Guardrails can and do help but they don't prevent hallucinations entirely. I generate trivia questions for my app. Questions are true/false and I tell it you must include an explanation of why the question is true for false. I also tell it to give me supporting links which validate its claim. I've had it make up explanations and invent plausible looking URLs. But I wrote a checker that pulls the content of the URL and asks a different model if the URL supports the claim. LLM's are much better at something like that than pulling a trivia question out of thin air. Reviewing diffs can be much the same. It isn't having to write the code but just evaluate if it matches what the commit comments claim it does. You are right, though. Without good comments on PR's and commits (i.e. context) the diff review is going to be a lot of assumptions made by the AI which is where hallucinations get more common.