Audio has always been overengineered and brittle. Vanilla alsa was the sweetspot, but things like pulseaudio and all the projects that followed it to "fix" it have too many things that can go wrong.

I don't seem to have any issues with audio anymore since Pipewire became default on Ubuntu, as a non-professional but fairly demanding user with a bunch of wired headphones plus bluetooth. I definitely used to have plenty of annoyances!