HuggingFace says this model has 753B parameters, which will need a lot more RAM than a maxed-out MacBook Pro. With 40B active parameters, running from SSD would need patience.

▲

_aavaa_ 5 hours ago | parent | next [-]

For an fp4 quantization it should fit with room to spare for KVCache

	▲	Tepix 4 minutes ago \| parent [-]
		Aren't Macbooks limited to 128GB RAM? FP4 would require >350GB RAM + KV cache, so no.

▲

api 8 hours ago | parent | prev [-]

I’ve wondered for a while if anyone is working on very wide channel parallel (kind of like RAID 0) SSD for this purpose. Couple that with a tensor processor and that would be interesting.

	▲	petu 16 minutes ago \| parent [-]
		There's talks about HBF. F for Flash -- HBM packaging and bus width, but using NAND memory. e.g. https://www.sandisk.com/company/newsroom/press-releases/2026...