What sort of tasks are you using self-hosting for? Just curious as I've been watching the scene but not experimenting with self-hosting.

▲

vunderba 2 days ago | parent | next [-]

Not OP but one example is that recent VL models are more than sufficient for analyzing your local photo albums/images for creating metadata / descriptions / captions to help better organize your library.

▲

kejaed 2 days ago | parent [-]

Any pointers on some local VLMs to start with?

	▲	vunderba 2 days ago \| parent \| next [-]
		The easiest way to get started is probably to use something like Ollama and use the `qwen3-vl:8b` 4‑bit quantized model [1]. It's a good balance between accuracy and memory, though in my experience, it's slower than older model architectures such as Llava. Just be aware Qwen-VL tends to be a bit verbose [2], and you can’t really control that reliably with token limits - it'll just cut off abruptly. You can ask it to be more concise but it can be hit or miss. What I often end up doing and I admit it's a bit ridiculous is letting Qwen-VL generate its full detailed output, and then passing that to a different LLM to summarize. - [1] https://ollama.com/library/qwen3-vl:8b - [2] https://mordenstar.com/other/vlm-xkcd
	▲	canyon289 2 days ago \| parent \| prev [-]
		You could try Gemma4 :D

▲

ktimespi 2 days ago | parent | prev | next [-]

For me, receipt scanning and tagging documents and parts of speech in my personal notes. It's a lot of manual labour and I'd like to automate it if possible.

▲

ezst 2 days ago | parent [-]

Have you tried paperless-ngx, a true and tested open source solution that's been filling this niche successfully for decades now?

	▲	codethief a day ago \| parent [-]
		They, too, offer integrations for LLMs these days, presumably for better OCR and classification.

▲

mentalgear 2 days ago | parent | prev | next [-]

Adding to the Q: Any good small open-source model with a high correctness of reading/extracting Tables and/of PDFs with more uncommon layouts.

	▲	mh- a day ago \| parent [-]
		I haven't tried it yet, but I bookmarked this recently: https://github.com/opendataloader-project/opendataloader-pdf

▲

BoredPositron 2 days ago | parent | prev | next [-]

I use local models for auto complete in simple coding tasks, cli auto complete, formatter, grammarly replacement, translation (it/de/fr -> en), ocr, simple web research, dataset tagging, file sorting, email sorting, validating configs or creating boilerplates of well known tools and much more basically anything that I would have used the old mini models of OpenAI for.

▲

irishcoffee 2 days ago | parent | prev [-]

I would personally be much more interested in using LLMs if I didn’t need to depend on an internet connection and spending money on tokens.