That's none of their business.

There are already laws in place against the kinds of behavior that neighbors are afraid will happen.

Noise, litter, etc, "nuisance" laws are on the books, but mostly depend on people following them voluntarily. The local authorities don't have the time/staff to investigate and resolve them all the time.