I wonder whether some fraction of the population has shit night vision and whether this fraction's preferences are driving this trend to too-bright headlights.

That would explain scenes on TV and in movies with very low "key", which to me are awful and frustrating because I cannot see anything, but maybe that is just how the (dimly-lit) scene would actually look in real life to the cinematographer or editor responsible for the visual design of the scene.