A drowning child is of fairly limited threat to an even halfway competent adult swimmer. Even at maximum panic/flailing, they just don't have the mass or strength to prevent you from at least treading water.

I'm a good swimmer, and 50 pounds of thrashing, scratching and climbing feels dangerous.