This may come as a surprise to you, but people like living in pleasant surroundings.

Just because I own the land does not mean I can open an abattoir next to an elementary school.

Using land in different ways results in externalities that affect those around it.

The people of a community should have some right to protect themselves from those externalities. How that happens in practice is a deeply flawed, messy, ugly process, but collectively deciding where to draw the line is part of living together as a community.

The next town over from where I live has basically no rules. No zoning, if you want to turn your property into a junkyard go right ahead! Even still, people are successfully fighting against a trash company putting in a landfill. I believe the levers they're pulling are a state wetlands permit and a state solid waste permit. The system is working.