Macbook M5 64GB - can run gemma-4-26b-a4b-it-4bit and Qwen3.6-35B-A3B-4bit at about 1500 tps prefix and 45 tps decode on contexts up to 100K tokens using MLX. It's faster than Claude. I was really surprised, chat quality is also similar to Claude for gemma4. Agentic works but does not compare to cloud models, you can still make agents where top level is code.

▲

mzubairtahir a day ago | parent [-]

sorry but asking again: how much memory is actually useable by gpu in macbook? as it is shared(os and apps also have to use same memory)? and it is different than dedicated gpu memory?

	▲	gizajob a day ago \| parent \| next [-]
		It’s completely shared so the OS and everything else takes up maybe 8GB of the RAM. On a 64GB machine you can run models about 45GB in size and still have space for those models run other tasks which themselves might need ram. To a user, the GPU appears to just use the RAM as much as it needs same as any other process running on the system. You can see what space your LLMs are taking up in Activity Monitor (or htop) and how much GPU capacity they’re using (all of it)
	▲	cco a day ago \| parent \| prev \| next [-]
		Rule of thumb is about 70-80%.
	▲	j45 a day ago \| parent \| prev \| next [-]
		You can adjust the percentage available both on the MacOS side and how much the model uses.
	▲	visarga a day ago \| parent \| prev [-]
		[dead]