Can someone ELI5 this to me? Nvidia has the market cap of a medium-sized country precisely because apparently (?) no one else can make chips like them. Great tech, hard to manufacture, etc - Intel and AMD are nowhere to be seen. And I can imagine it's very tricky business!

China, admittedly full of smart and hard working people, then just wakes up one day an in a few years covers the entire gap, to within some small error?

How is this consistent? Either:

- The Chinese GPUs are not that good after all

- Nvidia doesn't have any magical secret sauce, and China could easily catch up

- Nvidia IP is real but Chinese people are so smart they can overcome decades of R&D advantage in just s few years

- It's all stolen IP

To be clear, my default guess isn't that it is stolen IP, rather I can't make sense of it. NVDA is valued near infinity, then China just turns around and produces their flagship product without too much sweat..?

> because apparently (?) no one else can make chips like them

No, that's not really why. It is because nobody else has their _ecosystem_; they have a lot of soft lock-in.

This isn’t just an nvidia thing. Why was Intel so dominant for decades? Largely not due to secret magic technology, but due to _ecosystem_. A PPC601 was substantially faster than a pentium, but of little use to you if your whole ecosystem was x86, say. Now nvidia’s ecosystem advantage isn’t as strong as Intel’s was, but it’s not nothing, either.

(Eventually, even Intel itself was unable to deal with this; Itanium failed miserably, largely due not to external competition but due to competition with the x86, though it did have other issues.)

It’s also notable that nvidia’s adventures in markets where someone _else_ has the ecosystem advantage have been less successful. In particular, see their attempts to break into mobile chip land; realistically, it was easier for most OEMs just to use Qualcomm.

It’s several factors and all of your alternatives are true to some degree:

1. An h20 is about 1.5 generations behind Blackwell. This chip looks closer to about 2 generations behind top end Blackwell chips. So ~5ish years behind is not as impressive especially since EUV is likely going to be a major obstacle to catching up which China has no capacity for

2. Nvidia continues to dominate on the software side. Amd chips have been competitive on paper for a while and have had limited uptake. Now Chinese government mandates could obviously correct this after substantial investment in the software stack — but this is probably several years behind.

3. China has poured trillions of dollars into its academic system and graduates more than 3x the number of electrical engineers the US does. The US immigration system has also been training Chinese students but having a much more limited work visa program has transferred a lot of knowledge back without even touching IP issues

4. Of course ip theft covers some of it

What gave you the impression that it's "without too much sweat"? They sweated insanely for the past 6 years.

They also weren't starting from scratch, they already had a domestic semiconductor ecosystem, but it was fragmented and not motivated. The US sanctions united them and gave them motivation.

Also "good" is a matter of perspective. For logic and AI chips they are not Nvidia level, yet. But they've achieved far more than what western commentators gave them credit for 4-5 years ago. And they're just getting started. Even after 6 years, what you're seeing is just the initial results of all that investment. From their perspective, not having Nvidia chips and ASML equipment and TSMC manufacturing is still painful. They're just not paralyzed, and use all that pain to keep developing.

With power chips they're competitive, maybe even ahead. They're very strong at GaN chip design and manufacturing.

Western observers keep getting surprised by China's results because they buy into stereotypes and simple stories too much ("China can't innovate and can only steal", "authoritarianism kills innovation","China is collapsing anyway", "everything is fake, they rely on smuggled chips lol" are just few popular tropes) instead of watching what China is actually doing. Anybody even casually paying attention to news and rumors from China instead of self-congratulating western reports about China could have seen this day coming. This attitude and the phenomenon of keep getting surprised is not limited to semiconductors.

AMDs chips outperform nVidia's (Instinct is the GPU compute line at AMD) and at a lower per watt and per dollar range.

AMD literally can't make enough chips to satisfy demand because nVidia buys up all the fab capacity at TSMC.

My question would be: how did they fab it without access to ASML's high-end lithography machines?

https://www.theguardian.com/technology/2024/jan/02/asml-halt...

The article seems to only depict it being similar to the H20 in memory specs (and still a bit short). Regardless, Nvidia has their moat through cuda, not the hardware.

>- Nvidia doesn't have any magical secret sauce, and China could easily catch up

This is the simple explanation. We'll also see European companies matching them in time, probably on inference first.

> has the market cap of a medium-sized country

"According to investors, today's value of Nvidia's expected future profits over its lifetime equals the total monetary value of all final goods and services produced within a medium-sized country in a year."

Don't compare market cap with GDP, when you spell it out it's clear how nonsensical it is.

> Nvidia has the market cap of a medium-sized country

This makes no sense. Market cap is share price times number of shares, there is no analog for a country. It’s also not comparable to the GDP of a country, since GDP is a measure of flow in a certain time period, whereas market cap is a point in time measurement of expected performance.

Flagship? No, H20 was their cut down chip they were allowed to sell to China.

I'd say there's a mix of 'Chinese GPUs are not that good after all' and 'Nvidia doesn't have any magical secret sauce, and China could easily catch up' going on. Nvidia GPUs are indeed remarkable devices with a complex software stack that offers all kinds of possibilities that you cannot replicate over night (or over a year or two!)

However they've also got a fair amount of generality, anything you might want to do that involves huge amounts of matmuls and vector maths you can probably map to a GPU and do a half decent job of it. This is good for things like model research and exploration of training methods.

Once this is all developed you can cherry pick a few specific things to be good at and build your own GPU concentrating on making those specific things work well (such as inference and training on Transformer architectures) and catch up to Nvidia on those aspects even if you cannot beat or match a GPU on every possible task, however you don't care as you only want to do some specific things well.

This is still hard and model architectures and training approaches are continuously evolving. Simplify things too much and target some ultra specific things and you end up with some pretty useless hardware that won't allow you to develop next year's models, nor run this year's particularly well. You can just develop and run last year's models. So you need to hit a sweet spot between enough flexibility to keep up with developments but don't add so much you have to totally replicate what Nvidia have done.

Ultimately the 'secret sauce' is just years of development producing a very capable architecture that offers huge flexibility across differing workloads. You can short-cut that development by reducing flexibility or not caring your architecture is rubbish at certain things (hence no magical secret sauce). This is still hard and your first gen could suck quite a lot (hence not that good after all) but when you've got a strong desire for an alternative hardware source you can probably put up with a lot of short-term pain for the long-term pay off.

Isn’t NVIDIA fabless? I imagine (I jump to conclusions) that design is less of a challenge than manufacturing. EUV lithography is incredibly difficult- almost implausible. Perhaps one day a clever scientist will come up with a new, seemingly implausible, yet less difficult way, using “fractal chemical” doping techniques.

Perhaps China’s actions are less of a problem for Nvidia and more of a problem for other chip makes. After all, if Alibaba can make this chip, what justifies the valuation of companies like Groq?

Defaulting to China stealing IP is a perfectly reasonable first step.

China is known for their countless theft of Europe and especially American IP, selling it for a quarter of the price, and destroying the original company nearly overnight.

Its so bad even NASA has begun to restrict hiring Chinese nationals (which is more national defense, however illegally killing American companies can be seen as a national defense threat as well)

https://www.bbc.com/news/articles/c9wd5qpekkvo.amp

https://www.csis.org/analysis/how-chinese-communist-party-us...

Just some 2c totally out my head:

- Chinese labs managed to "overcome decades of R&D" because they have been trying for many years now with unlimited resources, government support and total disrespect of IP laws

- Chinese chips may not be competitive at process power/W with Western but they have cheaper electricity and again unlimited loss capacity

- they will probably hit wall at the software/ecosystem level. CUDA ergonomy is something very difficult to replicate and, you know, developers love ease of use

China's corporate espionage might have surpassed France at the winners podium.

It's all stolen IP.

Virtually all products out of china still are.

If you want something manufacturered the best way is still to fake a successful crowd sourcing campaign.

You'll be able to buy whatever it is on AliExpress (minus any safety features) within 6 months.