These are basically all fixed costs, not unit costs. You buy the GPU once and use it for as many calls as you have traffic for, and depreciate it over a fixed lifetime. You have to power it regardless of whether it's fully utilized or not. You have to maintain it by virtue of owning it, not really based on how many queries it has served. Bandwidth is the only one that really scales as a unit cost.

Open question whether this model is actually more economical than using the cloud AI service. The whole reason the industry moved to cloud computing in the first place was because computing had very high fixed costs, and the more these could be amortized over a fully-loaded query stream, the lower the unit costs.

Good points. Speaking from experience, it’s really hard to make it more economical than using a cloud AI service - even if you utilize the GPU to its fullest. There are, of course, other benefits, such as privacy/control/compliance/security, which should be the real reasons to do this, not cost.