Looks. Primarily at least, until you get into ultra-performance vehicles not really built for driving on public roads.

I recently upgraded to my "dream" daily driver car that came with 22" rims from the factory. It's definitely the worst part of the vehicle. Unfortunately I can't go much smaller, as 21" are the smallest that fit over the giant brake calipers.

Due to the weight of the vehicle, the larger brakes are likely needed - but the difference between this "performance" version of my car vs. the "luxury" version built on exactly the same body/chassis but with 19" wheels is night and day for day to day ride quality on poorly maintained city streets.

Once you get on fresh blacktop on the highway and start taking corners at a high rate of speed it's a different story, then the car just glides and "sticks" to the road super well due to the giant (wide) wheels. But that accounts for perhaps 5% of my driving if that.