Interesting if true - that Moonshot can train a ~3T SOTA model on only 20K NVIDIA GPUs, while others like Musk (who freely admits to distilling OpenAI's models) struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs).

I guess at least partly a reflection of all the optimizations in the Kimi 3 architecture.

In the recent leaked DeepSeek investor meeting, they also mentioned only having a 20K GPU cluster (unclear if NVIDIA, or Huawei).

From the article:

> A person familiar with Moonshot’s procurement strategy confirmed that the company does indeed have a channel for accessing Blackwell processors via Southeast Asia. They didn’t specify whether this was a rental channel — which is legal, in most cases — or direct purchases, which are a breach of US regulations. The Information reported this week that Moonshot is seeking additional Blackwell processors to train its next model.

> The 20,000 chips Moonshot accesses via Alibaba, meanwhile, are from Nvidia’s earlier generation of Hopper products, the people familiar with the agreement said.

So the 20k GPUs from Alibaba is only a lower bound on how many you need to train a model like Kimi K3.

The advantage of having more GPUs in any case is not so much that you can train bigger models, but that the turnaround time is faster, so you can run more experiments to dial in training choices. It's entirely possible that Musk has more than enough compute, but can't hire the talent to run all those experiments. (That would also explain why he has excess capacity he can rent to Google.)

> they also mentioned only having a 20K GPU cluster (unclear if NVIDIA, or Huawei).

A few quotes from the transcript:

> Our current computing capacity is approximately 20,000 H-equivalent units, most of which have just arrived within the past month or two

> Regarding the Huawei 950, Huawei currently provides us with 16,000 SIM cards

> A Huawei 950 [cluster] with 16,000 cards is equivalent to only a B-series card [cluster] with 4,000 cards.

From a superficial look I think the submitted title ("Moonshot built on 20k Nvidia chip cluster from Alibaba") might have been misleading. Article says "That hardware forms a substantial chunk of the overall computing capacity Moonshot uses for its Kimi models", implying that other hardware was also used.

(We've changed the title to what the article says now.)

It's not that surprising to me. Most of the innovation in Chinese models has been in efficiency gains and optimizations. K3 coming from the factory in MXFP4 weights is a pretty relevant factor. Big performance gap probably also due to Moonshot doing QAT. Throw in the fact that Musk has easier access to compute, and I think you have your answer on the disparity.

Do we believe them? It seems there’s no reason with the resources OpenAI and Anthropic have they wouldn’t be incentivised to build equally efficient pipelines? I think it’s mostly the fine tuning with distillation that gives Chinese labs their big advantage. However I’m pretty sure all the main labs are poisoning the output now when they detect Chinese activity so while K3 looks frontier on tests when you use it it’s miles off Sol and Fable.

One does wonder whether there has been an expiration of the actual weights of Opus at one point.

everything about the current 'a.i' models - east vs west points the wastefulness of western labs - whether in terms of gpu clusters being ran, cost of inference.

the only way is down for the massive valuations and 'a.i' revenue projections.

Genuine question. Is there a time factor part of that equation?

> Musk (who freely admits to distilling OpenAI's models) struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs)

Are you sure they are using all of their compute on training? Didn't they rent out a ton to other AI companies?

> Struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs).

This feels like a very American way of designing things - just throw more horse power at it, bigger is better! The rest of the world is usually a bit more resource constrained and efficient at using those resources.

See also Mustangs vs German sports cars, giant American fridges, giant American suburban McMansions vs livable cities etc etc.

> Musk (who freely admits to distilling OpenAI's models)

Source?