Who's going to train a nice big coding model based on this? I don't have the GPUs to do it...

I hope Apple Silicon will one day be competitive with GPUs. It dominates much of the CPU market, and it has a GPU, but it's still not even remotely comparable to Nvidia for inference (factor of >5 in some cases, especially in prefill) or even for gaming ("edge computing" in Nvidia terms).

I feel like their GPU architecture is going to need a significant redesign. There are limits to this tile-based mobile architecture.

▲

mycall 21 hours ago | parent [-]

Depends what you are comparing. Apple Macbook Pro M5Max out performs NVIDIA RTX N1X laptops for about the same cost.

▲

LoganDark 20 hours ago | parent [-]

Fastest consumer Apple silicon versus fastest consumer Nvidia. So M5 Ultra vs RTX 5090.

Not comparing price or efficiency or value or anything like that. Just the platforms as a whole.

▲

swiftcoder 20 hours ago | parent [-]

> Fastest consumer Apple silicon versus fastest consumer Nvidia. So M5 Ultra vs RTX 5090. Not comparing price or efficiency or value or anything like that. Just the platforms as a whole. So M5 Ultra vs RTX 5090

The problem here is that they aren't equivalent platforms - no amount of money will buy you an RTX 5090 with 512 GB of RAM.

If you want to buy the 16x RTX 5090s you need to get to the same 512 GB of RAM, we're talking $100,000 just in GPUs, industrial levels of power/cooling/noise, and the interconnect bandwidth is still going to kill it for normal inferencing workloads.

Of course, that 16x cluster is an absolute beast in terms of raw compute - it's just not all that useful for the things people are trying to do with such machines today.

	▲	LoganDark 18 hours ago \| parent [-]
		> The problem here is that they aren't equivalent platforms - no amount of money will buy you an RTX 5090 with 512 GB of RAM. I agree that Apple silicon is far better in terms of memory capacity, which is part of why I'm rooting for it. But an SoC like Apple silicon is roughly analogous to a CPU+GPU when it comes to compute, and the GPU side is quite lacking compared to today's "traditional" GPUs, even though the CPU is somewhat industry leading. Mainly, I'm disappointed with the performance of dense LLMs on Apple silicon compared to Nvidia, due to the large disparity in both memory bandwidth and GPU compute. Apple's latest chips are starting to push into just over a terabyte per second of memory bandwidth, about two-thirds of what the 5090 achieved nearly two years ago now. They are catching up quick and I do hope they pass, but they haven't quite yet. I also hear that Apple silicon is pretty bad at prefill compared to Nvidia even when memory bandwidth is not an issue, but haven't verified. It would make sense though due to the much greater compute.