> Do I really care about traffic if I’m not the one driving in it?

As someone who took the N across San Francisco every day for 5+ years: Yes, you would. Imagine a 5 mile journey taking 50 minutes. Even if you can nap or listen to a podcast, it's still a waste of time.