This topic is bringing out the hallucination-panic crowd.

It's perfectly safe to use LLMs to surface underlying data and knowledge without having to trust that it's giving you correct advice.

Use the LLM as search technology: string match -> semantic search -> search intelligence.

This is especially helpful when you don't know the right question to ask. Semantics only help you find text that is semantically near the input query. Semantics won't help you when what you are looking for is connected to what you have by something other than semantics, like causation, mechanism, or interpretation.

Trying to figure out why your crops aren't growing well will likely require you to read several articles across a range of sciences, probably scoped to the specific conditions you are dealing with. A semantic search is not going to surface those first.

Allowing an LLM to chain together several sources for you to read first does not require you to ask it for a conclusion.

The readme on Github does say the system produces hallucinated sources https://github.com/anonnerdfrenzy/survival-llm#how-the-model..., but really you're correct, for this kind of highly targeted use case the LLM's summary could probably be ignored entirely and the system could work extremely well even just as a search engine: index all of the content up front, chunked by paragraph and maybe enriched with a HyDE step, have the LLM facilitate running a handful of hybrid keyword/semantic searches based on the user's question and reranking the results, and show the exact pieces of text in the corpus that are most responsive to their question. No summary needed.

Can a bad LM implementation surface bad references? Yes. Are LMs overwhelmingly useful for research? Yes.

But at this point why even bring a full model into scope in the first place.. just run some cheap embedding retriever

A great strength of LLMs is how they provide useful answers even if the prompt has glaring spelling or grammatical mistakes. If your prompt is not great, as you said, they will reply in a way that helps you identify that flaws, and learn how to steer the model better.

They're amazingly useful tools, but they're nowhere the level believers want them to be, not yet.