Cloud is not great for GPU workloads. I run a nightly workload that takes 6-8 hours to run and requires a Nvidia GPU, along with high RAM and CPU requirements. It can't be interrupted. It has a 100GB output and stores 6 nightly versions of that. That's easily $600+ a month in AWS just for that one task. By self-hosting it I have access to the GPU all the time for a fixed up front relatively low cost and can also use the HW for other things (I do). That said, these are all backend / development type resources, self hosting customer facing or critical things yourself is a different prospect, and I do use cloud for those types of workloads. RDS + EKS for a couple hundred a month is an amazing deal for what is essentially zero maintenance application hosting. My point is that "literally anything else" is extreme, as always, it is "right tool for the job".

Literally anything else except GPU. :)

I kind of assume that goes without saying, but you're right.

The company I'm with does model training on cloud GPUs, but it has funding for that.

> RDS + EKS for a couple hundred a month is an amazing deal for what is essentially zero maintenance application hosting.

Right. That's my point, and aside from GPU, pretty much any normal service or app you need to run can be deployed on that.