I use it with a strix halo server. 35B runs stupidly fast. 27B is about 700 TPS prefill and 30 TPS token generation. Which interestedly is about what Kimi K3 gives me depending on provider.

▲

dionian 2 hours ago | parent [-]

what hardware do you use or recommend for this? never heard of it until today.

	▲	Zetaphor an hour ago \| parent [-]
		Strix Halo is the unified memory platform from AMD. Similar to the DGX Spark from NVIDIA or the M series Macs. I personally have the Framework Desktop, but there's also systems from other brands like Bosgame