What kinds of side channels are you thinking of? Given the key exchanges have a straightforward sha256/sha512 combiner, it would be surprising that a flaw in one of the schemes would give a real vulnerability?

I could see it being more of a problem for signing.

Yeah, key agreement in the context of SSH is quite forgiving of timing side channels as SSH uses ephemeral keys. There's no prospect of repeatedly re-doing the key agreement to gather more statistics on the counterparty's timing.