To rephrase it slightly. It's not a perfect vacuum, but compared to terrestrial conditions it's much closer to the former than the latter. The physics naturally reflects that fact.

To illustrate the point with a concrete example. You can heat something with the thermal transfer rate of aerogel to an absurdly high temperature and it will still be safe to pick up with your bare hand. Physics says it has a temperature but our intuition says something is wrong with the physics.