Remix.run Logo
▲ catigula an hour ago

Gemini vis-a-vis Claude and GPT seems to have substantial unsolved hallucination problems. Hope it’s solved or it can’t be used for agentic coding.

▲mattlondon 20 minutes ago | parent [-]

People keep saying this, yet the benchmarks suggest it is actually the best at not hallucinating? https://artificialanalysis.ai/evaluations/omniscience

Seems like as soon as there is any post about Gemini there are knee jerk reactions about how either you can't use it because it's not released (you can - it is released), you can't use it because you can't buy it (you can - consumer plans or API or just even OpenRouter), or that it hallucinates worse than other models (not according to https://artificialanalysis.ai/evaluations/omniscience that actually says it is SOTA in this regard).

I've used 3.8 flash a lot as my daily driver (...eagerly awaiting flash 4 or whatever it ends up getting called Zirconium or Caesium or something other nonsense I am sure) and I can't say I've noticed any significant hallucination issues that everyone seems to think are so prevalent.