That's because they are slowpokes and maniacs: In a decently flowing road, the majority of distinct cars you see are either moving significantly faster or slower than you (and the more extreme the difference the more likely you are to see them). Of cars that go at a similar speed to you, they approach you / you approach them more slowly so you'll see fewer of them.