Imagine your job is just replacing millions of Dollars worth of hardware every day. I mean there must be a reason the hold 8 GPUs per rack spare.

Yes the reason is they're pushing the GPUs hard so that they fail, and having hot spares saves the job until helping hands can go replace the bad component.

Theres a ton of people working at (neo)clouds doing exactly that