| ▲ | drclegg 7 hours ago | |
> So fuzzycanary also checks user agents I wouldn't be so surprised if they often fake user agents to be honest. Sure, it 'll stop the "more honest" ones (but then, actual honest scrapers would respect robots.txt) Cool idea though! | ||