Especially considering memory/gpu/compute are scarce so these services are likely running with very little buffer.

Any GPU that isn't running at 100% is a wasted GPU.