Being a big server proponent myself. Usually for one reason or the other there is need to introduce some socket style communication to the frontend and that becomes impossible in a single machine after a certain threshold.

Is there something obvious that I'm missing?

I've had 100k+ users connected to mid range linode boxes. Do you have that many?

Even still at that point you just round robin to a set of big machines. Easy

Load Balancing, Redundancy and Fail-Over