Realistically, you're looking at least 2x DGX sparks to run this at a 2 bit quant, but quantization really lobotomizes models so it's just better to run DSv4 flash at full precision.

4x DGX sparks should let you run this at 4 bit at least and there are some folks who ran GLM 5.2 on this configuration in r/LocalLlama

▲

teruakohatu 3 hours ago | parent | next [-]

How fast are 2x or 4x DGX?

I only have one and am wondering what the benefits are of getting another. I feel I will be disappointed…

	▲	deepllm 3 hours ago \| parent [-]
		If you can afford it, another DGX spark is worth it imo. Especially since, owning just one, you have a $1000 ConnectX7 card that's unused. You can find speeds here: https://spark-arena.com/leaderboard

▲

disiplus 3 hours ago | parent | prev [-]

i run flash v4 at 2bit, its pretty great and on my tests against full model It didn't lose any capabilities. It just was thinking more. So you don't have the same efficiency.