Where does this idea come from that we somehow can't take power away from people without criminal punishment after the fact?

Nobody should have this power, and then abuse of power wouldn't be an issue.