Remix.run Logo
Symbiote 4 hours ago

Meta is an exception. They intentionally ignore robots.txt rules.

(No second chances, I blocked their whole IP range.)

inigyou an hour ago | parent [-]

Everyone ignores robots.txt because every robots.txt blocks everyone.

nubinetwork 27 minutes ago | parent [-]

I allow Google, Bing, Yahoo, Kagi, and Apple on my personal blog... who else do I need to allow? Claude, OpenAI, Alibaba, and perplexity can go to hell.