Cant you run it on a pair of DGX spark? Sure it's expensive now, but at the initial retail price that was what, 8k in hardware? Not exactly unachievable

▲

bashtoni 2 hours ago | parent [-]

No, this is not a small model. I believe 4x Sparks is possible but 4.1 is pretty big for a flash model.

MiMo 2.6 Flash is the most capable model you can currently run on 2x Sparks without resorting to a quantised version of something.

▲

Pragmata 2 hours ago | parent [-]

I meant quantized. You can get an EXL3 quantized version thats almost the same performance.

	▲	bashtoni 32 minutes ago \| parent [-]
		My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial.