> Shouldn't the solution be to gate bulk access for automated services for a price?

Fine in theory but determined scrapers will use residential proxies in bulk.