Having it be unhomogenized is at least as big a factor. Side by side it's very hard to tell unhomogenized pasteurized milk apart from raw milk that came from the same dairy. Something about having the fat globules loose in there makes it taste richer, and gives it a different texture.

You can buy unhomogenized milk pretty easily at health food stores etc. People generally don't prefer it though because shaking it is annoying, it curdles more easily in cooked applications, and spoils faster for some reason. But for people with a culinary preference for the "raw" texture it's an option. But most people wanting raw milk want it for political/ideological reasons not gustatory ones. An insane sentence I could not have imagined needing to write 20 years ago.

> Something about having the fat globules loose in there makes it taste richer, and gives it a different texture.

You can also just mix skim milk and cream right before drinking. Tastes richer for a given fat content.

(We started doing this when we had people who wanted skim, 1%, 2%, and whole all sharing a fridge. Though later we ended up with people only wanting whole and the small taste improvement wasn't worth the hassle of mixing anymore.)