I need the notifications though.

But you can set them without content. That actually works with signal because all it sends through Google Firebase is a notification to wake up the app. If you have the content turned on the app basically fills the content in the notification locally. But you can turn that off.