Remix clone Hacker News

new | show | ask | jobs Github

▲

JSR_FDED 4 days ago

Which of these can I run locally on a 64GB Mac Mini Pro? And how much does quantization affect the quality?

▲

simonw 4 days ago | parent [-]

I use a 64GB M2 MacBook Pro. I tend to find any model that's smaller than 32B works well (I can just about run a 70B but it's not worth it as I have to quit all other apps first).

My current favorite to run on my machine is OpenAI's gpt-oss-20b because it only uses 11GB of RAM and it's designed to run at that quantization size.

I also really like playing with the Qwen 3 family at various sizes and I'm fond of Mistral Small 3.2 as a vision LLM that works well.

▲

JSR_FDED 4 days ago | parent [-]

Thanks. Do you get any value from those for coding?

▲

simonw 4 days ago | parent [-]

Only when I'm offline (on planes for example) - I've had both Mistral Small and gpt-oss-20b be useful for Python and JavaScript stuff.

If I have an internet connection I'll use GPT-5 or Claude 4 or Gemini 2.5 instead - they're better and they don't need me to dedicate a quarter of my RAM or run down my battery.

	▲	mettamage 4 days ago \| parent [-]
		Useful info! I have an M1 Mac with 64 GB and haven't experimented with offline models recently. I'll come back to this when I need my AI Maccie snackie, haha. Offline Apple Intelligence isn't at the level I want it, yet.