Sounds like you're using Bluetooth headphones and the game is attaching to the microphone which will automatically switch the audio codec from audio mode into headset mode. I'd suggest trying to completely disable the microphone of the headset so the game won't even try to attach to it.

Yep that's it! I ended up just buying a headset for gaming, since I use the other one for mostly music anyways. Solved the issue there. There were some workarounds I could try but I needed a new gaming headset anyways, the padding on my old one basically just fell apart after almost 7 years.