Remember the 90s when viagra pills and drug recommendations were all over the place?

Yeah, I use that as a safeguard :D The URLs that I don't want to be indexed have hundreds of those keywords that are leading to URLs being deindexed directly. There is also some law in the US that forbids to show that as a result, so Google and Bing are both having a hard time scraping those pages/articles.

Note that this is the latest defense measurement before eBPF blocks. The first one uses zip bombs and the second one uses chunked encoding to blow up proxies so their clients get blocked.

You can only win this game if you make it more expensive to scrape than to host it.

▲

n1xis10t 13 hours ago | parent [-]

Which law is that? Do you have a link to it?

	▲	cookiengineer 13 hours ago \| parent [-]
		The things I could find on justice.gov and other official websites, maybe there's more in the web archive? - https://www.justice.gov/archives/opa/pr/google-forfeits-500-... - https://www.congress.gov/110/plaws/publ425/PLAW-110publ425.p... - https://www.fda.gov/drugs/prescription-drug-advertising/pres... edit: Oh it was very likely the Federal Food, Drug and Cosmetic Act that was the legal basis for the crackdown. But that's a very old law from the pre-internet age. - https://en.wikipedia.org/wiki/Federal_Food,_Drug,_and_Cosmet... edit 2: Might not have been clear for the younger generation, but there was a huge wave of addicted patients that got treated with Oxycodone (or OxyContin) subscriptions at the time. I think that might have been the actual cause for the crackdown on those online advertisements, but I might be wrong about that.