Running models on-device on a Mac is immensely annoying though. Figuring out what will work out of BF16, FP8, BF16+FP8, NVFP4, INT8, GGUF ... the list goes on ... is 'non-obvious' at best. Apple do little to support with tooling. There's MLX, but unless you're happy to transform a model to that format yourself you'll be lagging a long way behind.

Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplifying it but it's still a pain.

▲

gizajob 6 hours ago | parent | next [-]

How is it a pain exactly? It’s just learning and only takes a day or two to get up to speed. We seem to have forgotten that for the past fifty years doing all kinds of tasks on computers has been tedious and involved and time consuming to even get working. My first computer had 48kb of RAM and to play a game you had to load it off cassette for five minutes. That was annoying. Having LM Studio download a model and load so you can chat or attach an agent it is effortless and easy in comparison.

	▲	prodmod 4 hours ago \| parent [-]
		Not to mention you can just ask Codex or Claude Code to set up a local LLM for you. Use one tool to set up the other.

▲

kamranjon 12 hours ago | parent | prev | next [-]

I dunno I use LMStudio pretty regularly and the MLX folks and the community usually have MLX versions of new model releases up within a day or two.

	▲	onion2k 12 hours ago \| parent \| next [-]
		For some models like the popular coding and chat models, things move faster. For things like images, voice, sound etc they definitely lag a long way behind.
	▲	Barbing 12 hours ago \| parent \| prev [-]
		& they clearly delineate all the models that’ll work on your exact machine (but guardrails can be disabled in settings)

▲

throw1234567891 8 hours ago | parent | prev | next [-]

Converting a model to mlx is literally a git clone and mlx_lm.convert. 5 minute operation after clone.

	▲	fellowmartian 5 hours ago \| parent [-]
		Or just ask any model to do it for you.

▲

fghorow 5 hours ago | parent | prev [-]

Counterpoint, I'm just a happy user of this local LLM [1] from antirez (of Redis fame).

[1] https://github.com/antirez/ds4