The DGX Spark is not good for inference though it's very bandwidth limited - around the same as a lower end MacBook Pro. You're much better off with a Apple silicon for performance and memory size at the moment but I'd recommend holding off until the M5 Max comes out early in the early as the M5 has vastly superior performance to any other Apple silicon chip thanks to its matmul instruction set.

▲

llbbdd 2 days ago | parent [-]

Oof, I was already considering an upgrade from the M1 but was hoping I couldn't be convinced to go for the top of the line. Is the performance jump from the M# -> M# Max chips that substantial?

	▲	smcleod a day ago \| parent \| next [-]
		The main jump is from anything to M5; not because it's simply the latest but because it has matmul instructions similar to a CUDA GPU which fixes the slow prompt processing on all previous generation Apple Silicon chips.
	▲	baby_souffle a day ago \| parent \| prev [-]
		> Is the performance jump from the M# -> M# Max chips that substantial From m1? Yes, absolutely. M3 is marginal now but m5 will probably make it definite.