I once worked a night shift stocking job just after college. I was in phenomenal shape without hitting the gym. I was at my lowest weight with a ton of lean muscle as a result of moving heavy loads and stocking paint every night. I also did the math at one point, and given the size of the warehouse I was probably walking quickly 8 miles during my shift. It became a chore to eat enough calories every day. Somewhere around 4000 calories/day, you may still be hungry, but you are generally full. Also food sort of becomes boring, and the desire to eat just isn't as strong.

That said, it was 4-6 hours 4 nights a week. That is a lot of time to spend to burn all those calories. It is really not hard to eat an extra 100 calories per day, but it takes a lot of effort to burn an extra 100 calories. It's the asymmetry here you absolutely have to respect. Further, at least for me, there is another asymmetry in terms of satiation vs hunger. It is much easier to be slightly satiated than it is to be slightly hungry. What this means, is that there is a tendency to be driven to eat slightly more than your body needs. This is partly why the GLP-1 drugs seem so effective, is that they seem to flip this asymmetry in the other direction, which means weight loss is the default, instead of weight gain.

Reminds me of when I got my first job at 17...I worked at a large department/grocery store. My job primarily consisted of pushing karts from the corrals back to the store.

In an 8 hour shift, I likely walked ~15 miles, with half of that time pushing up to a dozen karts. For lunch, I'd go to the McDonald's and get a Super Size (Since this was when that still existed) Double Quarter Pounder meal with a Coke. I'd chug the whole coke and then refill it. This meal was easily 3,000 calories, and I'd eat it 3 times a week.

After about two months on the job, I'd STILL lost about 5 lbs.

Yup, we have a limit in our ability to take in and metabolize calories, although it is somewhat flexible in that if you do it enough you'll get better at it. Look at the people who do the long thru-hikes. Stuff like the Appalachian Trail (Georgia to the highest point in Maine), PCT or CDT (both run from the Mexican border to the Canadian border). They will hit saturation on the ability to take in calories, although enough time out there can increase the ability to metabolize fat (very useful in that it has about 2x the calories per pound of other food)

But you are right that it's very definitely about the balance.

You make a good point about the time investment. You were able to do your exercise while working, but very few people will spend 16-24 hours a week working out.