I've found the same with direct sunlight exposure. My distance vision is much sharper if I've been outside a lot recently. It seems similar to how exercise works elsewhere in the body. You can definitely get a neuromuscular reaction if the incident angle of the sunlight is direct enough. The trick (as with all forms of exercise) is moderation.

Might it be that when outside you tend to look farther than when inside? So distance vision gets used more and body adapts. Similar to how kids that spend time outside are less shortsighted.