First thing would be that a small geofence (i.e., a narrow church on available data) is entirely orthogonal to having high precision, high quality location data available.

I won't claim with certainty that this is the case, but it seems likely that Factual was overselling their capabilities. That, or they relied specifically on having users grant high precision location data access and had nothing otherwise.

Apps that already need location data are probably the most likely sources of collecting such data - food apps, dating apps, chat apps you have sent your location in, ...

"Apps that already need location data are probably the most likely sources of collecting such data"

Yes, and many companies have access to both feeds.....