Update: it's starting to look like CPU C-states were the problem.