And what about the CA?

It's no different compared to regular SSH private keys. You need to protect it from compromise.

However, it provides you an additional layer of protection, because it does not need to be on the critical path for every SSH connection. My CA is a Nitrokey HSM, for example. I issue myself temporary certs that are valid only for 6 hours for ephemeral private keys.