I've had 100k+ users connected to mid range linode boxes. Do you have that many?

Even still at that point you just round robin to a set of big machines. Easy