I'm doing the same but I worry about windows compromise messing with the bootloader so then encrypted linux drive won't save me. Probably too paranoid though?

If you use secure boot and don't let your keys near Windows, you should be fine even if your Windows install is compromised. Unless you don't trust Microsoft themselves, in which case you'd need to re-enroll keys whenever switching operating systems, which is possible, but very tedious.