| ▲ | smallerfish 4 hours ago | |
I tried a prompt that consistently gets Gemini to badly hallucinate, and it responded correctly. Prompt: "At a recent SINAC conference (approx Sept 2025) the presenters spoke about SINAC being underresourced and in crisis, and suggested better leveraging of and coordination with NGOs. Find the minutes of the conference, and who was advocating for better NGO interaction." The conference was actually in Oct 2024. The approx date in parens causes Gemini to create an entirely false narrative, which includes real people quoted out of context. This happens in both Gemini regular chat and Gemini Deep Research (in which the narrative gets badly out of control). Kagi reasonably enough answers: "I cannot find the minutes of a SINAC conference from approximately September 2025, nor any specific information about presenters advocating for better NGO coordination at such an event." | ||
| ▲ | adriantoine 2 hours ago | parent | next [-] | |
I also tried two hallucinations that I've seen in the latest Gemini and Claude. The first one is the famous "how many r in strawberry" and another one is asking about king sacrifice in chess where most LLMs tell you it's a very powerful move and tell you how to do it (obviously a nonsense since the point of the game is to protect the king). Kagi actually got those two correct, explaining how losing your king means losing your chess game, which is quite impressive. | ||
| ▲ | VHRanger 4 hours ago | parent | prev [-] | |
Ah yes we have some benchmarks on this sort of misguided prompt trap, so it should perform well on this | ||