This is genuinely very helpful. I'm planning a MacBook pro purchase with local inference in mind and now see I'll have to aim for a slightly higher memory option because the Gemma A4 26B MoE is not all that!

▲

egorfine 4 hours ago | parent | next [-]

I have upgraded my M4 Pro 24GB to M5 Pro 48GB yesterday. The same Gemma 4 MoE model (4bit, don't remember which version) runs about 8x faster on M5 Pro and loads 2x times faster in memory.

So yes, do purchase that new MacBook Pro.

▲

croemer 42 minutes ago | parent [-]

You don't know if it's the newer model or the increase in RAM. If someone has already got 48GB it they might not benefit much. You changed 2 things at once.

	▲	egorfine 18 minutes ago \| parent [-]
		Not really: it's the same model size and it fits 24GB entirely.

▲

tomr75 4 hours ago | parent | prev [-]

pretty sure Nvidia GPU is better bang for buck because of usable inference speed..