You cannot uninstall Music.app indeed. There is one toggle that hides all of Apple Music. It completely hides it, to the point that a link online to the service will error out.

That setting was removed[0].

[0] https://discussions.apple.com/thread/253315335?sortBy=rank