Remix.run Logo
▲ 01100011 5 hours ago

As the author mentions, the biggest threat is from customers becoming competitors. The problem space is stabilizing and this allows the biggest buyers of GPUs to work with companies like broadcom to make their own chips. Thanks to agents, they can also spin up their own abstraction layers and debug tools. CUDA is becoming less relevant, and the rate at which that is happening is about to go parabolic.

Surely Nvidia won’t stand still and will continue to compete, but the margins will collapse along with volumes.

AMD will probably suffer less, relatively speaking. Their failure to invest in software finally seems to be paying off now that LLMs can pick up their slack. But the growth currently priced in to their stock will never materialize and they’ll be seen as arriving just in time for the lights to go out.

▲irq-1 4 hours ago | parent | next [-]

> CUDA is becoming less relevant, and the rate at which that is happening is about to go parabolic.

CUDA is the 'open' standard that exists today, has market/mind share, and doesn't lock you into a vendor. Look at google to see how difficult it is to make a new chip and surrounding system.

Still, Nvida should work on making the generation of chips and languages that come after CUDA.

▲sjdh76 4 hours ago | parent | next [-]

Yup. There are so many layers the majority doesnt interact with, so they focus on what they know. The current stack has probably 10-12 layers. And NVIDIA touches more layers than anyone else.

1. Physical infrastructure - power, cooling, network fabric, chip fabrication

2. Chips

3. Low level sw - CUDA/ROCm/Metal/XLA etc

4. Kernels - cuBLAS, cuDNN, FlashAttention, FlashInfer, Triton, llama.cpp

5. Kernel gen/optimization - MLIR, TVM, TensorRT

6. ML frameworks pytorch, jax, tensorslow

7. Models - Llama, Qwen, Claude, GPT, Gemini

8. More tuning - vLLM, SGLang, TensorRT-LLM, DeepSpeed, Megatron-LM

9. API/SDK/Platform

10. Apps/Agents/Pipelines/Orchestration etc

▲01100011 3 hours ago | parent | prev | next [-]

Yeah and that’s cool but the large labs don’t need open standards. They are big enough and well funded enough to have their own in-house solutions (and they already have them). They haven’t significantly cut their Nvidia purchases yet though.

▲j45 4 hours ago | parent | prev | next [-]

CUDA is a standard, but I've heard according to some it seems dated too. Either way a lot has been possible because of Nvidia and will continue to be.

It's nice to see companies like Apple adding CUDA support in their own way.

The thing that increasingly intrigues me is how long it might take AI-assisted development to explore what else might be possible in the space.

▲Grimblewald 4 hours ago | parent | prev [-]

True, but leadership changes, how it happened, and that new leadership's affiliations, are giving my group heavy re-thought on what future nvidia plays in our compute needs. The only country we trust less than China is Israel, and this Nvidia coup has not exactly been reassuring. The reason we focus local models is that exact affiliation problem providers with good models have. We're already investigating AMD and how we can transition, hopefully sooner rather than later given israel's history of slipping malware into absolutly everything.

▲psadri an hour ago | parent | prev | next [-]

Apple's strategy here was to buy out the most advanced fab's capacity years in advance. I think NVDA was probably doing the same. Even if you can design a new chip, you can't get it built because all the cutting edge fab capacity is locked up. Ironically that sustains demand for your product, enabling you to continue this pattern! Maybe NVDA's circular investments were to ensure this "demand" is sustained whenever they hit a slow patch.

▲vannevar 4 hours ago | parent | prev | next [-]

>As the author mentions, the biggest threat is from customers becoming competitors.

Maybe in the long term. The biggest threat in the short term is Nvidia's exposure to the data center bubble, which was already showing signs of popping before Jensen's departure. Given Nvidia's role as a key investment bank in the AI trade, making and brokering circular financing deals around data centers, his departure could hasten the data center collapse if the company starts backing away from its key financing role.

▲wilsonjensen 4 hours ago | parent | prev | next [-]

The timing of the WH dropping this makes me suspect Kushner types angling to tank stock, make a lot of bag holders, buy it all up with Saudi or some other country's help

Dylan Ratigan went on that rant a decade ago about it being a mathematical fact American wealth was being extracted through sale of all these corps and offshoring

We are to be serfs of royalty, religious cranks, and the rest of the non-contributor orator class

▲01100011 3 hours ago | parent [-]

Sure, but that’s somewhat orthogonal to whether Nvidia stays in its current position of defacto tech leader commanding top margins.

▲wilsonjensen 28 minutes ago | parent [-]

The outcome of such a scenario 100% matters to your statement and should not be overlooked by investors

▲lumost 4 hours ago | parent | prev | next [-]

A quick prompt to ChatGPT estimates that a B300 should cost around 6-10k in a mature market given typical CPU industry/silicon economics. NVidia has a massive margin for a semi-conductor firm, it would not be surprising to learn that AI is much cheaper than we currently estimate when that margin is removed.

▲Ygg2 2 hours ago | parent | prev | next [-]

> the biggest threat is from customers becoming competitors.

I'd rephrase that a bit. The biggest threat was from their customers commoditizing them (while Google and Amazon could become competitiors, rest are way more iffy). You know, nVidia actually competing on price instead of relying on locking them in nVidia propietary landscape.

https://www.wsj.com/tech/ai/nvidias-days-as-an-open-checkboo...

▲jmyeet 2 hours ago | parent | prev | next [-]

I'm kinda surprised at how many people on this thread think you can now vibecode a GPU and compete with NVidia, basically. This is a wildly optimistic view of AI's capabilities. NVidia has several important moats and a huge lead in this space. That's not going to disappear anytime soon. It has expertise in chip design, the volume relationship with TSMC and the relationship with HBM producers.

I do agree CUDA is becoming less relevant though.

There are two companies I'd be most concerned about if I were NVidia's CEO:

1. Apple. It's huge and has a long history at this point of making very efficient chips that use a shared memory pool. This defeats NVidia's tactic of aggressive market segmentation by the amount of VRAM. You can buy a Mac Studio with 256GB of RAM (well, you have to wait until March currently) for ~$11k. That's a lot cheaper than a single RTX 6000 Pro; and

2. Google. They currently buy or rent a lot of GPU capacity from SpaceX and others as well as make their own data centers. But they also have their own TPUs that they've been developing for years. I would be shocked if they've not investing billions into ramping up that program so they can use homegrown chips for AI. Google is still the best at building their own data centers and their own networking is industry-leading (IMHO).

▲01100011 9 minutes ago | parent [-]

No one is vibecoding chips yet, although models are dramatically accelerating the verification and bring up of custom chips. See OpenAI’s success, for example. People are already working on alternative software stacks on top of various HW and they’re either already there or close to it depending on the company.

Google is putting billions into TPUs and this isn’t a secret.

▲tayo42 4 hours ago | parent | prev [-]

Why do you think custom chips means they'll stop buying GPUs not just use more of everything? That's what I saw when AMD got competitive with Intel and arm showed up.

▲01100011 3 hours ago | parent [-]

Power is the bottleneck. If big labs need to use their more efficient, in-house solutions then they will. These companies are big enough to create their own weather, so to speak.