| ▲ | irq-1 4 hours ago | |
> CUDA is becoming less relevant, and the rate at which that is happening is about to go parabolic. CUDA is the 'open' standard that exists today, has market/mind share, and doesn't lock you into a vendor. Look at google to see how difficult it is to make a new chip and surrounding system. Still, Nvida should work on making the generation of chips and languages that come after CUDA. | ||
| ▲ | sjdh76 4 hours ago | parent | next [-] | |
Yup. There are so many layers the majority doesnt interact with, so they focus on what they know. The current stack has probably 10-12 layers. And NVIDIA touches more layers than anyone else. 1. Physical infrastructure - power, cooling, network fabric, chip fabrication 2. Chips 3. Low level sw - CUDA/ROCm/Metal/XLA etc 4. Kernels - cuBLAS, cuDNN, FlashAttention, FlashInfer, Triton, llama.cpp 5. Kernel gen/optimization - MLIR, TVM, TensorRT 6. ML frameworks pytorch, jax, tensorslow 7. Models - Llama, Qwen, Claude, GPT, Gemini 8. More tuning - vLLM, SGLang, TensorRT-LLM, DeepSpeed, Megatron-LM 9. API/SDK/Platform 10. Apps/Agents/Pipelines/Orchestration etc | ||
| ▲ | 01100011 3 hours ago | parent | prev | next [-] | |
Yeah and that’s cool but the large labs don’t need open standards. They are big enough and well funded enough to have their own in-house solutions (and they already have them). They haven’t significantly cut their Nvidia purchases yet though. | ||
| ▲ | j45 4 hours ago | parent | prev | next [-] | |
CUDA is a standard, but I've heard according to some it seems dated too. Either way a lot has been possible because of Nvidia and will continue to be. It's nice to see companies like Apple adding CUDA support in their own way. The thing that increasingly intrigues me is how long it might take AI-assisted development to explore what else might be possible in the space. | ||
| ▲ | Grimblewald 4 hours ago | parent | prev [-] | |
True, but leadership changes, how it happened, and that new leadership's affiliations, are giving my group heavy re-thought on what future nvidia plays in our compute needs. The only country we trust less than China is Israel, and this Nvidia coup has not exactly been reassuring. The reason we focus local models is that exact affiliation problem providers with good models have. We're already investigating AMD and how we can transition, hopefully sooner rather than later given israel's history of slipping malware into absolutly everything. | ||