Users are OK with acting as proxies because they don't understand all the shady stuff their proxy is being used for. Also consumer ISPs generally ban this.

> These SDKs, which are offered to developers across multiple mobile and desktop platforms, surreptitiously enroll user devices into the IPIDEA network.

?

Am I the only one cynically thinking that "Russia, Iran, DPRK, PRC, etc" is the "But think of the chiiildren!!!" excuse for doing this?

And when Google say

"IPIDEA’s proxy infrastructure is a little-known component of the digital ecosystem leveraged by a wide array of bad actors."

What they really mean is " ... leveraged by actors indiscriminately scraping the web and ignoring copyright - that are not us."

I can't help but feel this is just Google trying to pull the ladder up behind then and make it more difficult for other companies to collect training data.

No, the question is not just disclosure. People have their bandwidth stolen, and sometimes internet access revoked due to this kind of fraud and misuse - disclosure wouldn’t solve that

The proxies is the reason why you get spam in your Google search result, spam in your Play store (by means of fake good reviews), basically spam in anything user generated.

It directly affects Google and you, I don’t see why they should not do this.

Okay. You get right on that. In the meantime, would you rather they did nothing? What do you actually want, in concrete terms?

> many anti-scraping services explicitly allow access to Google and maybe couple of other search engines.

because google (and the couple of other search engines) provide enough value that offset the crawler's resource consumption.

You say this like robots.txt doesn't exist.

"Only Google is allowed to scrape the web."

If I'm not mistaken, the plaintiffs in the US v Google antitrust litigation in the DC Circuit tried to argue that website operators are biased toward allowing Google to crawl and against allowing other search engines to do the same

The Court rejected this argument because the plaintiffs did not present any evidence to support it

For someone who does not follow the web's history, how would one produce direct evidence that the bias exists

Yup exactly. Google must be the only one allowed to scrape the web. Google can't have any other competition. Calling it in "user's best interest" is just like their other marketing cons: "play integrity for user's security" etc

This is demonstrably false by the success of many scrapers from AI companies.

Google does not use residential proxies.

This does nothing against your ability to scrape the web the Google way, AKA from your own assigned IP range, obeying robots.txt, and with an user agent that explicitly says what you're doing and gives website owners a way to opt out.

What Google doesn't want (and I don't think that's a bad thing) is competitors scraping the web in bad faith, without disclosing what they're doing to site owners and without giving them the ability to opt out.

If Google doesn't stop these proxies, unscrupulous parties will have a competitive advantage over Google, it's that simple. Then Google will have to decide between just giving up (unlikely) or becoming unscrupulous themselves.

Have you got any proof of Google scraping from residential proxies users don't know about, rather than from their clearly labelled AS? Otherwise you're mixing entirely different things into one claim.

Play Protect blocks malicious apps, not network traffic, so no, it obviously doesn't interfere with Google's apps.

AFAIK it also left SmartTube (an alternative YouTube client) alone until the developer got pwned and the app trojanized with this kind of SDK, and the clean versions are AFAIK again being left alone. No guarantee that it won't change in the future, of course, but so far they seem to not be abusing it.

That takes buying low and selling high to a whole new level

Google's definition of a "bad actor" is someone who wants to use Google without seeing the ads. Or Kagi. Or an AI other than Gemini.

100%

Yeah, they make you do a Skype interview (or probably Zoom interview nowadays). You could call this KYC or collateral, depending on your view of the company. It does limit the nefariousness of their clientele but I doubt they do much, or any, monitoring of actual traffic after onboarding (not for compliance reasons, anyway).

I’ve certainly never been asked to do KYC with Luminati after using them for hundreds of terabytes over the years.

It’s not like I’m using some bigco email address or given them any other reason to skip KYC either.

No, he is referencing Google going after the Chinese company, not the Israel based one. That does not mean there is bias with the commenter at all, just that the companies operate differently and are treated differently. The country of origin is important as Israel based companies are more integrated into the western business world, and tend to at least try to show an effort in keeping spam and other things off their platforms. Now I do agree that they are both bad companies that should not be allowed to operate the way they do. I would say the same thing about the other 1000 scrapers hitting websites everyday as well (including Google).

What they did not comment directly on, is how many apps / games they might have actually removed from the Playstore with the removal of the SDKs, which would be the actual interesting data.

Personally, I don’t think either of them are actually meaningfully bad. A bit naughty, maybe?

I do think the disparity in attention is fascinating. These new Chinese players have been getting nonstop press while everyone ignores the established giant.

> that's how you get legal threats from media companies or the police called to your house.

Or residential proxies get so widespread that almost every house has a proxy in, and it becomes the new way the internet works - "for privacy, your data has been routed through someone else's connection at random".

They provide an SDK for mobile developers. Here is a video of how it works. [0] They don't even hide it.

[0] https://www.youtube.com/watch?v=1a9HLrwvUO4&t=15s

> that they have to objectively ruin the site for everyone using it. See the API changes and killing off of third party apps.

Third party app users were a very small but vocal minority. The API changes didn't drop their traffic at all. In fact, it's only gone up since then.

The datacenter IP address blocks aren't just for scrapers, it's an anti-bot measure across the board. I don't spend much time on Reddit but even the few subreddits I visited were starting to become infiltrated by obvious bot accounts doing weird karma farming operations.

Even HN routinely gets AI posting bots. It's a common technique to generate upvote rings - Make the accounts post comments so they look real enough, have the bots randomly upvote things to hide activity, and then when someone buys upvotes you have a selection of the puppet accounts upvote the targeted story. Having a lot of IP addresses and generating fake activity is key to making this work, so there's a lot of incentive to do it.

Fix your government.

Have you considered that it’s because a new industry popped up that decided it was okay to slurp up the entire internet, repackage it, and resell it? Surely that couldn’t be why sites are trying to keep non humans out.

How much can you make if you run all of them at the same time?

Doesn't the ISP detect them?

Most of them abuse the ip pool attached to lambda from my experience.

That's already the case (irrespective of residential proxies) because content only serves as bait for someone to hand over personal information (during signup/login) and then engage with ads.

Proxies actually help with that by facilitating mass account registration and scraping of the content without wasting a human's time "engaging" with ads.

Amazon.com now only shows you a few reviews. To see the rest you must login. Social media websites have long gated the carrots behind a login. Anandtech just took their ball and went home by going offline.

Anyone know how to create a usable reddit account from the .onion domain?

Agreed. With things people paid for and using our wifi data to build their "positioning dbs" that you can't block or turn off on your phone, without "rooting" your own device.

How is Find My a proxy network?

Have you tried it? Every new account will be shadowbanned and if it's shared you often get blank page 429. None of this was true before the API shutdown.

Yes you do.

Private VPS for personal VPN in Netherlands (digital ocean), then Hungary (some small local DC) — both are blocked from day one.

> You've been blocked by network security. To continue, log in to your Reddit account or use your developer token. If you think you've been blocked by mistake, file a ticket below and we'll look into it.

Try browsing from any Mullvad vpn. You will be "blocked by network security"

I have never interacted with a reddit employee who wasn't actively gaslighting me about the platform. Do you even use the site? I talked to a PM recently who genuinely thought the phone app was something people liked.

everything on Reddit is so locked down it’s useless. even if you do get to post something useful some basement dwelling mod will block it for an arcane interpretation of one of the subreddits 14 rules.

Have you tried using it logged out on a vpn? It is impossible.

there are several times where I've had to disable PIA to access reddit's login page

Network operators have zero reason to care, they get paid per the GB for the bandwidth.

Google does much worse in Google–branded devices and apps, like the wifi location data harvesting.

> trust a company like LG to secure their devices.

They have an interest in securing their devices so they can sell proxy service themselves.

The actual secret is to use IPv6 with varied source IPs in the same subnet, you get an insane number of IPs and 90% of anti-scraping software is not specialized enough to realize that any IP in a /64 is the same as a single IP in a /32 in IPv4.

If they're CGNAT then unless Starlink actively provides assistance to block them it won't matter.

As someone who wants the internet to maintain as much anarchy as possible I think it would be nice to see a large ISP that actively rotated its customer IPv6 assignments on a tight schedule.