I thought that in an expanding universe, mass isn't conserved. I don't see why charge would be conserved in that case.

Mass isn't close to being conserved. If an electron and positron annihilate, they become two massless photons, for example. You don't need to invoke an expanding universe.

In an expanding universe, energy isn't conserved. (Unless we do the trick we always do with energy and just define it to be conserved https://youtu.be/lCgKTpIzjRk?t=174 )

I don't know exactly why you think this applies to charge conservation. The U(1) symmetry is where charge conservation comes from and it isn't affected by an expanding universe.