Would that be an appropriate mitigation for this issue though, preventing IP leaks?

I use Windscribe vpn and it has an option for a split tunneling of apps, so I have a bunch of apps I just let through. But, when I use the 'block connections without VPN' in always on VPN, it blocks those tunneled apps.