Most of the time, and to a lot of people, it doesn't matter. I have a fast mobile data plan and fast home Internet. But even I have encountered the following circumstances where I wish sites were smaller:

- on an underground subway with slow and spotty connection

- on slow airplane WiFi

- on slow hotel WiFi

- on a slow mobile network while traveling internationally