This sounds very appealing. What size Mac mini would I need for that?

A PC with an nvidia card with 16gb vram works just fine for Qwen MoE models, and these have worked great as a daily driver for me.

mathgeek 2 hours ago | parent | prev | next [-]

blensor 3 hours ago | parent | prev [-]

I am curious if you implicitly assumed they are Macs or if that's what you are looking for specifically?

I assumed the 27B dense model would be preferable to a MoE model, and that it wouldn’t fit into a consumer graphics card, which leaves the Macs.

Then I assumed for cost and battery/heat reasons that a Mini would be better than a laptop.

	▲	blensor an hour ago \| parent [-]
		The reason why I was curious is that I am running my stuff on a Strix Halo and I get the feeling that this class of devices ( gmktek, minisforum, lenovo, etc. ) seem to becoming a pretty good alternative