| ▲ | fragmede 10 hours ago | |
https://firehose.directory/robots.txt isn't one, but comes up in Google despite that. https://atproto.at/robots.txt is good and findable with a direct search https://atproto.wiki/robots.txt is empty but findable on Google. https://pulsar.feeds.blue/robots.txt is 404 and thus not indexed by Google. https://leaflet.pub/robots.txt is good. A logged in google search for this finds this for me. Tell the people at https://pulsar.feeds.blue to fix their robots.txt if they want to get indexed. | ||