Neither is the right alternative to compare to. You aren’t going to hit 100% utilization (if you are, ignore me, this doesn’t some to you, and write a blogpost for me to read and share).
The comparison should be against renting in the cloud for the duration of your task for training and research or using pay-per-api-call providers for general inference instead of buying your own hardware (and paying the electricity and cooling bills on top), because let’s face it, the models you want to use are probably the same ones available on inference providers (but, yes, some are more trustworthy than others).
Speaking as someone that does ML/AI research, you are essentially paying a huge premium for being able to just run your Python script at any time without setting up a deployment script and harness to run the job remotely, while your hardware sits essentially idle the rest of the time.
The only way to make the math work is if you rent your hardware in the background for inference while you’re not using it in anger, but despite all the startups and promises that has never become as streamlined as mining bitcoins or shitcoins used to be and they don’t pay out as much as they say they would. Renting your hardware for training is another option but doing that is a lot more involved, options are fewer and farther in between, you won’t get as much utilization out of it, and doesn’t let you feasibly abort running tasks at a moment’s notice.
