I really want to like ai music but none of them seem to be trained on reward models that reward "ambiance" or any sort of interesting sound design (which this paper obviously doesn't even concern), which might be reflective of the people training them not having niche music tastes

Why do you want to like it?