> they will get primaried

Primaries are kinda insane though. It basically means that a small minority of voters control who actually is allowed to stand for election under a party banner. Like, I understand how it ended up this way, but it's having really bad consequences.

That being said, if you could fix gerrymandering, a lot of the issues with primaries go away, as there would be more competition in the actual election which would dis-incentivise proposing extremist candidates in the primary.