Both this GPU and the Mac are constrained by a lack of GPU compute. There's basically no rack-scale GPUs with 480gb on-die that have a sub-400W compute element. It's going to be incredibly prefill-constrained on denser model inference just like the Apple Silicon iGPUs are.

▲

krasin 6 hours ago | parent | next [-]

This GPU has 1.3 PFLOPS of FP8 compute performance and 1.5 TB/s of memory bandwidth: https://videocardz.com/newz/intel-gpu-d270-is-xe3p-crescent-...

This is almost twice better than RTX 5090. This GPU will not be prefill constrained, I feel.

But I am surprised that they were able to pack so much flops into 300W.

	▲	kadoban 6 hours ago \| parent [-]
		> But I am surprised that they were able to pack so much flops into 300W. I'm not. When you're doing inference you typically reduce power limits anyway if you're optimizing for efficiency. Even if you don't, it doesn't spike to the max of the card. My 3090 I power it down to 175W or so for single-digit reductions in performance for like, 60% of the wattage, something like that.

▲

shusaku 6 hours ago | parent | prev | next [-]

It seems like there’s a market out there that is just happy to run the models locally, even if generation is slow.

▲

jchw 6 hours ago | parent | prev [-]

Am I wrong to assume that large MoE models are likely the point anyway? Seems like this card would love DeepSeek V4.1 Flash.