M5 Ultra will ship before end of year, likely. Though with current RAM shortage, likely max spec will be 256GB and in short supply.

In late 2027 or early 2028, Nvidia will release Vera Rubin DGX Spark, likely with double or better the performance of current Blackwell, though unclear if memory capacity will go up much from current 128GB. Two to four of those will run models like this decently.

In 2028 we should expect Vera Rubin RTX discrete lineup, including the replacement to the RTX PRO 6000. Likely memory spec will be minimum 128GB. Good chance of up to 200GB. Two to four of those will run NVFP4 models in this class very well.

I think there is a gap right now for running large models such as GLM 5.2 in Q4 or Q8. My hope is on Intel Crescent Island 480GB cards. Let‘s see how expensive they‘ll be.

I feel like the models are good enough for a decade of future work. So Once you have a working set up you can keep using it to do the work at the same level. There will be better stuff and may make that type of work obsolete but if you can do useful things it won’t be worth less.

P40 was release 2016 and still selling like hotcakes!

Watch out. Those claims are often made based on KL-divergence over some arbitrary corpus, not performance in the real world or benchmarks.

I’ve found that I need to go a couple steps past whatever quantizations are good enough in the KL-divergence testing to get good performance in real tasks with long context. So when Q4 is claimed to be lossless I end up with Q5 or Q6 for actual long-context tasks.

The natural progression when performance is enough would be price. We were starting to see that but not anymore. I wonder if somebody is afraid the future where generally useful computation is cheap.

It's true we hit limits, but I feel like a lot of it was "limits" in the sense that the tradeoff stopped being worth the cost, so we optimized in other areas.

So we hit limits on clock speed in the early 2000s (ex - the 4ghz wall) but it also turned out that mobile as the driver for sales meant no one really cared much about clock speed compared to performance/watt.

Clock speed mattered, but only relative to how many watts it took to get it (and above 4ghz... too many watts).

But we've seen a 15x improvement over the last 20 years. Performance/Watt is WAY up.

My guess is that LLMs are going to drive another "improvement cycle" in areas that we didn't care much about before.

I've built about 10 personal desktop machines (1 every ~4 years) and I can honestly say that I didn't care much about memory bandwidth prior to 2021.

In the same way that I didn't care much about how many watts my pentium 4 was using in 2005.

But now... now I care a lot about memory bandwidth. I care about memory speeds and total system ram in a manner I really, really didn't before.

So I think we're going to see a big shift to machines built on unified ram with a crazy focus on squeezing memory bandwidth and total ram capacity as far as we can.

My bet is that we'll get a similar 10-15x improvement by 2040 in unified system ram designs.

I fully expect to see 2tb unified ram desktops and 200gb unified ram phones be relatively common on a 20 year timeline, assuming we see similar levels of geopolitical stability (ex - world war 3 throws a wrench into things).

And with MTP (or other speculation techniques) you can ~double that.

But I did my research online and the sun cycle is every 11 years and something something global warming is a hoax every single year now.

I asked and it said “403 forbidden - careful peon attempts to bypass the late stage capitalism api with your monetary offerings in exchange for you daily tokens will get you perma banned right to jail”.