This is a great solution for a very specific type of team but I think most companies with consistent GPU workloads will still just rent dedicated servers and call it a day.

▲

hyperbovine 8 hours ago | parent | next [-]

I agree, and cloud compute is poised to become even more commoditized in the coming years (gazillion new data centers + AI plateauing + efficiency gains, the writing is on the wall). There’s no way this makes sense for most companies.

▲

NitpickLawyer 8 hours ago | parent [-]

> AI plateauing

Ummm is that plateauing with us in the room?

The advantage of renting vs. owning is that you can always get the latest gen, and that brings you newer capabilities (i.e. fp8, fp4, etc) and cheaper prices for current_gen-1. But betting on something plateauing when all the signs point towards the exact opposite is not one of the bets i'd make.

	▲	lelanthran 5 hours ago \| parent [-]
		> Ummm is that plateauing with us in the room? Well, the capabilities have already plateaued as far as I can tell :-/ Over the next few yeas we can probably wring out some performance improvements, maybe some efficiency improvements. A lot of the current AI users right now are businesses trying to on-sell AI (code reviewers/code generators, recipe apps, assistant apps, etc), and there's way too many of them in the supply/demand ratio, so you can expect maybe 90% of these companies to disappear in the next few years, taking the demand for capacity with them.

▲

ocdtrekkie 8 hours ago | parent | prev [-]

It's the opposite. The more consistent your workload the more practical and cost-effective it is to go on-prem.

Cloud excels for bursty or unpredictable workloads where quickly scaling up and down can save you money.

	▲	langarus 7 hours ago \| parent [-]
		Other benefits: easy access to reliable infrastructure and latest hardware which you can swap as you please. There are cases where it makes sense to navigate away from the big players (like dropbox going from aws to on-prem), but again you make this move when you want to optimize costs and are not worried about the trade-offs.