▲ | lolinder 2 days ago | |||||||
> I know its not completely true, I know read-mode can help you bypass the ads _after_ you already had a peek at the cluttered version What about reader mode that is auto-configured to turn on immediately on landing on specific domains? Is that a robot for the purposes of robots.txt? https://addons.mozilla.org/en-US/firefox/addon/automatic-rea... And also, just to confirm, I'm to understand that if I'm navigating the internet with an ad blocker then you believe that I should respect robots.txt because my user agent is now a robot by virtue of using an ad blocker? Is that also true if I browse with a terminal-based browser that simply doesn't render JavaScript or images? | ||||||||
▲ | mattigames 2 days ago | parent [-] | |||||||
If you are using an ad-blocker by definition you are intentionally breaking the intended behavior by the creator of any given website (for personal gain), in that context any discussion about robots.txt or any other behavior that the creator expects is a moot point. Autoconfig of reader mode and so on its so uncommon that is not even in the radar of most websites, if it was browser developers probably would try to create a solution that satisfies both parties, like putting the ads at the end and required to be text-only and other guidelines, but its not popular, same thing happens with terminal-based browsers, a lot of the most visited websites in the world don't even work without JS enabled. On the other hand, this AI stuff seems to envision a larger userbase so it could become a concern and therefore the role of robots.txt or other anti-bot features could have some practical connotations. | ||||||||
|