| ▲ | mitxela 2 hours ago | |||||||
Scrapers scrape all sorts of nonsense, notably every possible git diff from git hosts. They clearly aren't concerned with rationality. | ||||||||
| ▲ | XorNot 2 hours ago | parent [-] | |||||||
Scrapers scrape every link they can find. The git host example is probably the one which Anubis is the dumbest defense for: the main reason to have those links is for easy machine interaction. So that's dead when you implement it. So why even host the links? It's a git repo: send them a local got client and let them clone the repo in browser or something. | ||||||||
| ||||||||