The data center owners aren't the ones selling new GPUs.

They are selling access to GPU computation. Seeking their used GPUs would flood the market with cheap competitors using their old GPUs.