But Spotify has that as well. Tons of user curated playlists. And although user playback data is harder to parse through, it's also pretty straightforward to build some clustering algorithm where if you both like X then you might like Y as well.

My theory is that they don't have the incentive. Apple Genius was ridiculously good at music discovery, too. I shudder to think how much I spent on iTunes songs via genius over its run. But now that apple/spotify/etc get my monthly dollars either way, there's no huge incentive for them to create the discovery systems.

Spotify is pay to win (play) - especially user curated ones playlists.