I find these local infra costs crazy heavy duty. I always assume these folks are getting their values worth or that infra is serving multiple people's requests at and justifying the cost. Like I once thought of buying a 48-64 GB RAM laptop (not even clsoe to what these models need) and I realised the equivalent cost will serve me for years of ≈ $30 per month API/sub access. So I assume the usage is also heavy duty, or if nothing then for privacy and safety and then I guess the price, if one can pay, will totally be worth it.

You just need to imagine someone making big bucks in tech + being a little bit money ignorant, having a big house and a nice basement and cheap energy prices.

Than it just makes sense that they already do this.

But to be fair, if i move, enough space for battery, solar and a basement which can take proper ML hardware + heat exchanger, is in my future vision.

> I realised the equivalent cost will serve me for years of ≈ $30 per month API/sub access

The advantage is no more 5-hours or weekly session limit anymore. Also no chance your proprietary code ends up in someone repo.

If the only variable is money, local models will never make any sense at these prices of infra/subscriptions.

For a lot of people, there are other variables though.