And the moment you start building large structures on it everyone and their dog is going to show up and NIMBY it.

Spoken like someone who has never been to the mojave. This is the home to the largest wind turbine installation in the country. But again, if the worry is the solar panels acting like sails, probably not a great site given that wind.