> The problem here is that they aren't equivalent platforms - no amount of money will buy you an RTX 5090 with 512 GB of RAM.

I agree that Apple silicon is far better in terms of memory capacity, which is part of why I'm rooting for it. But an SoC like Apple silicon is roughly analogous to a CPU+GPU when it comes to compute, and the GPU side is quite lacking compared to today's "traditional" GPUs, even though the CPU is somewhat industry leading.

Mainly, I'm disappointed with the performance of dense LLMs on Apple silicon compared to Nvidia, due to the large disparity in both memory bandwidth and GPU compute. Apple's latest chips are starting to push into just over a terabyte per second of memory bandwidth, about two-thirds of what the 5090 achieved nearly two years ago now. They are catching up quick and I do hope they pass, but they haven't quite yet.

I also hear that Apple silicon is pretty bad at prefill compared to Nvidia even when memory bandwidth is not an issue, but haven't verified. It would make sense though due to the much greater compute.