> What about physical buttons off the wheel? A well-designed car might have quite a few: window controls, sound system volume knob, temperature up-down, etc.

Those controls are typically on some surface of the car your hand is braced on. They're also very simple physical controls with a good amount of tactile feedback. It's hard to fuck up a simple push button window control or AC dial. Even on a bumpy road you'd be hard pressed to have trouble with such controls.