These are great points.

However, I don’t think these companies provision capacity for peak usage and let it idle during off peak. I think they provision it at something a bit above average, and aim at 100% utilization for the max number of hours in the day. When there is not enough capacity to meet demand they utilize various service degradation methods and/or load shedding.

▲

mcny 4 days ago | parent [-]

Is this why I get anthropic/Claude emails every single day since I signed up for their status updates? I just assumed they were working hard with production bugs but in light of this comment, if you don't hit capacity constraints every day, you are wasting money?

▲

chii 3 days ago | parent | next [-]

This is true for all capital equipment - whether it's a GPU, a bore drill, or an earth mover.

You want to make use of it at as close to 100% as possible.

▲

hvb2 3 days ago | parent [-]

With the caveat that GPUs depreciate a bit faster obviously. A drill is still a drill next year or a decade from now.

	▲	apetrov 3 days ago \| parent [-]
		yes, but the capital is still tied to it. you want it to Have a meaningful ROI, not sitting in a warehouse.

▲

koliber 3 days ago | parent | prev [-]

Just like at an all-you-can eat buffet.