I have a quirky vLLM on k8s on 2x OEM sparks setup with 9 models available to me. I'm not keen to run nightly vLLM, too many issues with it in the past. Going the qwen-next path means displacing things I use daily :/

I have a watchful eye on the diffusion ~ Jev/Kev PR

https://github.com/vllm-project/vllm/pull/57250

▲

girvo 2 hours ago | parent [-]

For what it's worth, Flash Next outperforms every other model that is available to us on the GB10 in all of my testing; though if you have two sparks then the TP=2 version is even better and easier (I don't think you'll need the nightly for that at all, just use the recipe)

I'm so tempted to buy a second one...

	▲	verdverm an hour ago \| parent [-]
		prices have gone up quite a bit... I'm running embedding, reranking, and policy tuned models too, and a Jev/Kev when that's landed. Flash Next is not a substitute for those I have OpenCode/Fireworks to access big models