| ▲ | BetterThanSober 3 hours ago | |
Accounting for initial acquisition cost and typical depreciation/failure rate? I assume they already consider colo price, electricity, etc | ||
| ▲ | freakynit 3 hours ago | parent | next [-] | |
70%+ net profits margins are for rented infra, that means no major upfont expense. Also, these margins are against 60% utilization only. You can well imagine if you own your infra and can make utilization over 90%, where these go. And then, my link above is for K3 model, which, compared to ds flash models, is highly unoptimized. If I had a few million lying around, I would have invested in inference. | ||
| ▲ | petesergeant 2 hours ago | parent | prev [-] | |
I would be surprised if there was such a glut of undifferentiated inference providers for open models otherwise[0]. Possibly just people trying to minimize the loss from already-committed capital, but I don't see AWS and DO selling inference via OpenRouter unless they're making a decent profit from it. | ||