Use the PIV applet for SSH and signing Git commits instead? Git supports S/MIME and SSH can use keys over PKCS#11 basically out-of-box on OSs that don't ship gpg-agent (that just interferes with SmartCard usage in general).