And then there's Michael Phelps, living proof you can outrun (or at least outswim) just about any diet you can imagine. He's obviously an extreme, but he's not the only example.

The point of the saying is to get most people to understand that the biggest factor in the average person's weight loss is their diet, not their exercise levels.

Pointing out that elite endurance athletes and olympic athletes can have high calorie requirements isn't helpful or the point. Yes, energy must come from somewhere, but even among the generally fit and active portion of the population, only a vanishingly small number of them exercise at the intensity and time requirements to burn so many calories as to not easily have all of that thwarted with a single meal.