I'm still hoping some future LLM is good and small enough for people to run local AI for everyday life.

jrm4 a day ago | parent | next [-]

It's interesting trying to be an AI "moderate," which as the owner of a 16gb card I definitely am. Still working on the balance, but it really does feel like I'm going to land on something like "use external AI services to build a few tools, but use my local rig to actually handle everything personal once those tools are built."

▲

trollbridge a day ago | parent | prev [-]

Already here. 3.8-27b is good enough.

5 years ago it would have been science fiction

▲

72deluxe a day ago | parent | next [-]

Do you find it intelligent enough? What hardware are you using?

▲

criley2 a day ago | parent | prev [-]

For me, qwen 3.8 27b on my m5 max is so slow and dumb that I can't think of a single use for it. Haiku, Luna, or data center scale open models are ten times faster and ten times smarter (and inference so cheap that it's essentially free). It was a nifty toy to brag to friends about but why would I waste my time goofing with something so slow and dumb?

▲

liuliu a day ago | parent | next [-]

Try DeepSeek 4.1 Flash 1017, feels much better: https://github.com/drawthingsai/local-code-community

▲

jrm4 a day ago | parent | prev | next [-]

For roughly the same reason you might slow down and read a book on how to do something as opposed to speedrunning 20 youtube videos?

	▲	criley2 a day ago \| parent [-]
		Are you insinuating that your agents are more effective at 20 tok/s than 200 tok/s? I don't even understand your analogy. I'm not watching the youtube video or reading the book. The model is. And a big computer can watch 20 youtube videos faster than a small computer can watch 1, and do a far better job understanding all of them (bigger model at data center scale is both smarter and faster than smaller model at local scale).

▲

LeBit a day ago | parent | prev [-]

On my Mac Mini M4 Pro with 48G I find Qwen 3.8 27B perfectly usable, useful and fast enough.

Are you trying to one shot create the Linux kernel?