| ▲ | input_sh 4 hours ago | |
Yes, we know, its purpose is to guide the bots, not forcibly block them. That said, one of the biggest websites in the world not respecting it is definitely a noteworthy story. Hopefully another one of the biggest websites in the world (formerly known as Twitter) eventually respects it as well instead of not even disclosing itself via a user agent and pretending to be Safari running on iOS. | ||
| ▲ | namegulf 3 hours ago | parent [-] | |
Why down vote a comment? You're talking about one (yes, biggest) but millions of other bots don't follow must be a bigger story. | ||