I think a lot of the flavor comes from two other factors: fat level (i.e. you can get higher fat than "whole" milk quite easily, although at some point it gets marketed as "cream", which significantly alters the taste) and the cow's diet: you can easily taste the difference between milk marketed as "grassfed" and the cheap stuff in a blind tasting. (Same is true for eggs, I suspect, though I've never tried that particular experiment myself.)

Having it be unhomogenized is at least as big a factor. Side by side it's very hard to tell unhomogenized pasteurized milk apart from raw milk that came from the same dairy. Something about having the fat globules loose in there makes it taste richer, and gives it a different texture.

You can buy unhomogenized milk pretty easily at health food stores etc. People generally don't prefer it though because shaking it is annoying, it curdles more easily in cooked applications, and spoils faster for some reason. But for people with a culinary preference for the "raw" texture it's an option. But most people wanting raw milk want it for political/ideological reasons not gustatory ones. An insane sentence I could not have imagined needing to write 20 years ago.