Energy recovery ventilation is the answer to this.

HRVs only deal with temperature, but then you have humidity that is non-controlled: moisture coming in during the summer, and getting vented out in the winter (too-dry air coming in).

ERVs handle both.