You can run the 27B released last week. I haven't tried it yet myself but the 3.6 version runs great on my 5090.

Strongly recommend https://github.com/Neroued/ninfer, which can pull ~180 TPS on 5090 with 3.8, and 500 (!) with 3.6 35B-A3B.

	▲	cpburns2009 3 hours ago \| parent [-]
		I've been waiting for the dust to settle on this model so I can find a good runtime setup. I'm definitely bookmarking this. Thanks!