You can -- the real problem here is that each app could violate your privacy in different ways. Unless you break TLS and inspect all the traffic coming from an app (and, do this over time since the reality of what data is sent will change over time) then you don't really know what your apps are stealing from you. For sure, many apps are quite egregious in this regard while some are legitimately benign. But, do you as a user have a real way to know this authoritatively, and to keep up with changes in the ecosystem? My argument would be that even security researchers don't have time to really do a thorough job here, and users are forced to err on the side of caution.

What they do then is create an app where location is necessary, make that app spin up a localhost server, then add js to facebook and every site with a like button to phone that localhost and basically deanon everyone.