You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?

iamflimflam1 a day ago | parent | next [-]

You may benefit from an embedding approach for semantic search. Not sure what an LLM would give you on top of that.

	▲	crsx_ a day ago \| parent \| next [-]
		If the emergency is in a foreign country, being able to communicate with locals would likely be a benefit - and a domain specific trained model could translate better than general purpose translators. In general, I think speech as input/output is under-explored. In the emergency scenario, in a stressful environment, having an expert in your ear you can talk to should work much better than having a big manual book to look up specific cases.
	▲	Lio a day ago \| parent \| prev [-]
		I think an LLM would give a "conversational" experience to search. That's handy for situations where you might not really understand what you need to search for. Any search system that can ask you clarifying questions is going to be a big improvement. Or where you need to combine several steps together but you don't yet know what those steps are. There's probably other technologies that could do that, requiring lower resources but they'll come with different trade-offs around configuration. Just having a Raspberry PI, a offline copy wikipedia and a RAG enabled small LLM would be quite useful or at least entertaining if you have to go off grid.

▲

visarga a day ago | parent | prev | next [-]

> You will probably want a search engine though.

The search engine is indeed the last missing component from a sovereign stack. But I think this could be solved locally with little cost. Instead of indexing content on the web we should be indexing sources themselves - where to look for X? - like forums, blogs, docs, feeds, and specialized search engines. We could collectively amass millions of these search stubs that can be used by local models to go and fetch fresh information from the source directly. This means separating the routing layer from the information layer, we don't need to keep information cached from the whole internet locally. The search stubs could fit in a few GB about same size with the local LLM. The cool thing is that sources change much slower than information itself, so the search stub database could be refreshed at a slower pace. We could combine a few million generic stubs with a few hundred personal stubs generated from our own activities. It is trivial to generate these stubs by piggy backing on frontier models.

	▲	20 hours ago \| parent [-]
		[deleted]

▲

rtpg a day ago | parent | prev [-]

grep works well!