Great point.

The parent's argument is that the marginal cost of inference is minimal. However, the fundamental flaw is that he's separating inference from the high cost frontier models. It's a cross-subsidy that can't be ignored.

Without any insider knowledge on the economics of these companies, I suspect it's that the amount of infrastructure you have to build is determined by peak usage rather than average usage. If peak usage is much higher for a small part of one day a week (say on Monday morning as software developers across the US get back to work) the cost of fulfilling demand at all times can be insane. That's why companies are implementing batch/standard/priority pricing for the API.