the real reason midtown is where it is, is because that is where the train stations are due to the city banning surface running railways south of 42nd St. (The remaining surface railways north of that point have been covered up.)

As a result, Midtown is now one of the few places in New York's metropolitan that can reach the millions of people in the five boroughs and the suburbs all at once, which means the labor market is substantially larger than what it is in Lower Manhattan or Downtown Brooklyn, and makes it massively more attractive for employers.