I would've loved that as a kid. For that matter, I did actually fill some balloons with hydrogen using vinegar and something (might've cut some magnesium off one of those primitive fire starters, or I might've used a weaker metal, can't quite remember). I think I used the glass tube Costco used to use as a vanilla bean container to strap the balloon onto.

Zinc and vinegar is what you're looking for. Our high school science class had us fill plastic pipette bulbs to make rockets.