The reason to run local models is not for coding mostly it's for learning how to deploy models and tinker with self hosting. It's also for massively crunching data 24/7. Imaging having an agent analyzing constinous log streams etc .. that could be a usescse where even deepseek could add up cost.

▲

epolanski an hour ago | parent [-]

Also privacy and offline capabilities.

▲

DanielHB 37 minutes ago | parent [-]

One thing I realized is just how much offline local models can hurt mass data collection.

For example, I needed to write an invitation letter for immigration control for a relative visiting me. Previously I would have used a search engine for a template. Today I fire up my local qwen 3.5-9b for this kind of stuff and feed it all the private data I need.

Unfortunately it is unlikely the average user will known how to avoid this data collection. Even if the LLM is local you are likely feeding the prompts to remote servers if you harness/chat-interface is not properly vetted.

	▲	epihelix 19 minutes ago \| parent [-]
		I hope for local model chat inexperienced users are just using llama.cpp's built-in web server interface, which gives you everything you need. No need for a harness or any other chat client.