I have one. The limitation (beyond total size of the memory) with the Spark is DDR5. "Real" inference hardware is HBM (high bandwidth memory) which is like 10x the performance.

So for prefill -- which is more about compute than bandwidth -- the Spark performs quite admirably. But on decode it's highly bandwidth constrained. Some smaller MoE models (e.g. Gemma4) can do 60-70 tok/second but anything dense, or anything that is actually filling up most of that 128GB is going to choke out around 15 tok/sec. Even at NVFP4.

My experiments with this are at https://github.com/rdaum/eider/

For my current work I get to log into trays on a real GB300. It's somewhat comical that NVIDIA is marketing the little baby on my shelf here as even in the same universe as that. Which is basically like having access to a super computer.