Signal scales with exposure time, noise scales with the root of exposure time. So although the noise increases with time, the signal-noise ratio increases faster. When dealing with astronomy photos, you want to maximise SNR.

(This is a simplification, there are many types of noise including shot noise and thermal noise, but in general this is the rule of thumb you use)