I moved to a downtown neighborhood a couple years ago, and got rid of my car when I did. This meant way more everyday walking, and I noticed an almost immediate improvement in how I felt. I'm not even walking as much as you are and it still made a big difference.

On days when I end up walking further, like a long distance grocery trip or whatever, I'm tired that day but usually do feel better the next day.