Considering the tendency even now of LLMs to hallucinate, and the hardware requirements for any kind of LLM -why not just gather those resources into some sort of local archive that can be searched without an LLM?

This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.

▲

nico 3 hours ago | parent | next [-]

Not sure about how the code in this project works. But the way I see it, an LLM+chat is a great interface for information. If you then connect it to a db with the knowledge/articles you need, then you get a great way to navigate it, without needing the model to memorize all the info

▲

mapontosevenths 3 hours ago | parent | next [-]

I think that if I were to build this I might skip the LLM and just setup a nice interface to the source files using qmd style search. [0]

That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.

I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.

[0] https://github.com/tobi/qmd

	▲	nico 3 hours ago \| parent [-]
		Awesome, I didn't know qmd style search, thank you! I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples) 0: http://jeffyclassify.com/blog/when-llms-fail-at-classificati...

▲

rafterydj 3 hours ago | parent | prev [-]

I agree with the OP; this technology does not seem an appropriate fit for survival equipment. By definition survival scenarios can be life and death. Even one "You're absolutely right!" level mistake from an LLM could give a user bad information, leading to John Doe eating a mushroom he really should not have eaten.

	▲	tourmalinetaco 2 hours ago \| parent [-]
		I agree with your point, but if John Doe is foraging mushrooms without experience of the local flora (which I doubt this archive contains), then John Doe is kinda asking for it. The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway. It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.

▲

ASalazarMX an hour ago | parent | prev | next [-]

Why not both? After the collapse of civilization, as long as you can maintain the computer running, you could educate the next generation as soon as they could write. Maybe they will bypass the local agent as they learn their way into the library, to save energy.

Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.

▲

datsci_est_2015 an hour ago | parent | next [-]

Both. Both is good.

A field guide to foraging for edible mushrooms will probably be more practical than naïve use of an LLM: “can I eat this mushroom?” is an exercise in trust that I will maybe never be comfortable with.

But “how do I effectively forage for edible mushrooms” is a great stepping stone.

	▲	whoanoway an hour ago \| parent [-]
		[dead]

▲

munk-a an hour ago | parent | prev [-]

Nvidia GPUs in the post-apocalyptic radioactive barrens are going to be stupid pricey.

▲

rebolek 2 hours ago | parent | prev | next [-]

I think that for situations like this we should invent a device that can hold information for an arbitrary amount of time even without electricity. Maybe we could call it "BOOK" or something like that.

	▲	anigbrowl an hour ago \| parent \| next [-]
		This isn't clever or insightful. You can of course condense down critical information into a survival handbook, but in existential crises you don't necessarily know in advance what you'll need to know. Wikipedia (as a benchmark repository of knowledge) is ~5 million pages and would require a decent-size room to store if printed out. By contrast, a laptop + solar panel can fit in a backpack. SD cards/USB sticks with the capacity to store all of Wikipedia are a few $ each. If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?
	▲	dorianmariecom 2 hours ago \| parent \| prev [-]
		and then we have a lot of “books” so we collect them in “shelves” then in “libraries” then we want to search them etc etc

▲

lrvick 2 hours ago | parent | prev | next [-]

> and the hardware requirements for any kind of LLM

You can run bonsai2 27b on basically any gaming GPU from the last 15 years including on game consoles.

Everyone has access to this stuff now if they bothered to look beyond the marketing from big labs trying to collect subscription revenue.

I work on compilers, kernels, and operating systems using exclusively consumer GPUs.

▲

x3haloed 2 hours ago | parent | prev | next [-]

Decent LLMs can run on anything now: https://github.com/he-be/turbo-fieldfare

Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.

Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.

▲

IshKebab 33 minutes ago | parent | prev | next [-]

> the tendency even now of LLMs to hallucinate

That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.

Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.

In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.

Obviously this local LLM is going to hallucinate all over you though.

▲

tourmalinetaco 3 hours ago | parent | prev [-]

It already is a local archive that can be searched independently, but the LLM does RAG and is, according to them, 95% accurate. It can be useful if you don’t remember where to find a source/what words to search for, but even a durable device with Kiwix would be useless in a a lot of survival situations people imagine.

Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.