Anthropic, at least, has gone to lengths to avoid hardware lock-in or being open to extortion of the nvidia variety. Anthropic is running their models on nvidia GPUs, but also Amazon Trainium and Google's TPUs. Massive scale-outs on all three, so clearly they've abstracted their operations enough that they aren't wed to CUDA or anything nvidia-specific.

Similarly, OpenAI has made some massive investments in AMD hardware, and have also ensured that they aren't tied to nvidia.

I think it's nvidia that has less of a moat than many imagine they do, given that they're a $4.5T company. While small software shops might define their entire solution via CUDA, to the large firms this is just one possible abstraction engine. So if an upstart just copy pastes a massive number of relatively simple tensor cores and earns their business, they can embrace it.