If making infra means designing their own silicon to target only inference instead of more general GPUs I can agree with you, otherwise the long-term success is based on how cheap they can run the infra compared to competitors.

Depending on Nvidia for your inference means you'll be price gouged for it, Nvidia has a golden goose for now and will milk it as much as possible.

I don't see how a company without optimised hardware can win in the long run.

The silicon can be very generic. I don't see why prices of "tensor" computation units can't go down if the world sees the value in them, just like how it happened with CPUs.