> Why did it fund sprawl?

Road funding is a big reason. Federal, state, and local taxes are used for roads, and more driving. Parking minimums required land to be dedicated to parking, further encouraging car usage for transportation and spreading out development with parking lots in between developments.

> Why didn't anyone choose to develop density on existing sites?

Existing sites would have had to not be developed enough to trigger a rezoning. If a different use was being proposed for land, then a zoning hearing would be needed, and parking minimums would have to be enforced. Thus requiring adjacent lots to be bought and redeveloped into parking unless exceptions were made. You can see remnants of this in some cities where amongst historic buildings and skyscrapers there are large surface parking lots.

> We built that stuff just fine from 1870 through the 1940s. What changed?

Quite a few things. Parking minimums as mentioned, euclidian/single use zoning, etc. I think one of the core things that changed is something that Strong Towns mentioned. Up until the early 1900s, municipal planners would try to project how much tax revenue per acre of land was being generated and how much tax expenditures were made for those areas. Over time, tax per acre or per parcel was deprioritized, and level of service for roads was used as an economic metric. More vehicles in an area means more economic activity (in theory), so municipalities started optimizing for more vehicular movement.

In the end, it was a lot of government regulation that resulted in this. From the federal level, to states, counties, and municipalities. It worked for a large portion of the voting populace, so it was generally favored.