> Firstly, you can run the LLMs on your own machine. So I find the proprietary/moat narrative weak.

You cannot run useful models on consumer hardware, sorry this is wrong and will always be the case. Atleast for 10 years until GPUs with 48GB vRAM depreciate. This is a limitation of llm architecture. You cannot post train a <1T Param model to a place where it competes with frontier model capability. If you think you 70b param models (which still require 5k in GPUs) are useful, you are being dishonest with yourself.

It costs about $60-80k to run a 1T param model at your house like Kimi 2.5 .. which is the only size model that's going to get anywhere close to a foundation model's capability. Nobody is going to spend close to 100k to run a mediocre open source model as opposed to spending $200.00 a month. Its a ridiculous notion.

I run Qwen-3.5 based LLMs in the 20-35 parameter range on my laptop's iGPU and it works great for my use case, which includes coding, search, reasoning, and general tasks. Gemma3 is good too.

There are ones that are distilled with better reasoning models or abliterated for whatever you need, and the multimodal features work... fine.

Just started running local LLMs this week, and it is pretty much overkill for what anyone in my family needs. All it really lacks is some tools for it to use, which I am putting together now.

To be fair, the best model I have used is claude sonnet. I don't really know what I am missing with opus.