Zoning on average is a blight.

Industrial / High Rise is the only thing that should need permitting.

Fourplex / Duplex / Single Family / Small Offices / Schools should not.

Schools, kindergartens and daycares cause a shitload of noise from the children playing and the traffic. Small offices also induce lots of traffic, and small merchants even more.

You do not want to have that outside of zoning control because that is how you end up with a road designed to handle the need of a dozen homes (i.e. 24-30 cars a day) suddenly dealing with ten times that load - not just because of noise, smell and traffic jams but especially because road surfacing quality is usually "the cheapest you can get away with for the expected load", the road will go bad way faster than expected.

Zoning is not your enemy, zoning is your friend. Particularly if you value peace and roads you can drive on.