| ▲ | trollbridge 18 hours ago | ||||||||||||||||||||||
Yeah, it serves the purpose of blocking this kind of proxy traffic that isn't in Google's personal best interests. Only Google is allowed to scrape the web. | |||||||||||||||||||||||
| ▲ | 1vuio0pswjnm7 11 hours ago | parent | next [-] | ||||||||||||||||||||||
"Only Google is allowed to scrape the web." If I'm not mistaken, the plaintiffs in the US v Google antitrust litigation in the DC Circuit tried to argue that website operators are biased toward allowing Google to crawl and against allowing other search engines to do the same The Court rejected this argument because the plaintiffs did not present any evidence to support it For someone who does not follow the web's history, how would one produce direct evidence that the bias exists | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | a456463 17 hours ago | parent | prev | next [-] | ||||||||||||||||||||||
Yup exactly. Google must be the only one allowed to scrape the web. Google can't have any other competition. Calling it in "user's best interest" is just like their other marketing cons: "play integrity for user's security" etc | |||||||||||||||||||||||
| ▲ | miki123211 9 hours ago | parent | prev | next [-] | ||||||||||||||||||||||
Google does not use residential proxies. This does nothing against your ability to scrape the web the Google way, AKA from your own assigned IP range, obeying robots.txt, and with an user agent that explicitly says what you're doing and gives website owners a way to opt out. What Google doesn't want (and I don't think that's a bad thing) is competitors scraping the web in bad faith, without disclosing what they're doing to site owners and without giving them the ability to opt out. If Google doesn't stop these proxies, unscrupulous parties will have a competitive advantage over Google, it's that simple. Then Google will have to decide between just giving up (unlikely) or becoming unscrupulous themselves. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | vachina 12 hours ago | parent | prev | next [-] | ||||||||||||||||||||||
This is demonstrably false by the success of many scrapers from AI companies. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | viraptor 15 hours ago | parent | prev [-] | ||||||||||||||||||||||
Have you got any proof of Google scraping from residential proxies users don't know about, rather than from their clearly labelled AS? Otherwise you're mixing entirely different things into one claim. | |||||||||||||||||||||||
| |||||||||||||||||||||||