| ▲ | hskalin 4 hours ago | |
But what do these bots gain from this? | ||
| ▲ | ddxv 4 hours ago | parent | next [-] | |
It's all OpenAI / Anthropic / Singapore/China based crawlers. I guess they gain data that I provide for free. I get very little crawling by Googlebot / Bing etc which barely even index my site, I only have a single page "indexed" by Google. | ||
| ▲ | AlienRobot 3 hours ago | parent | prev | next [-] | |
Have you ever use an AI chatbot? It's basically a democratized scraper. Type something stupid, it "visits" 200 websites and regurgitates some random gibberish. | ||
| ▲ | bredren 4 hours ago | parent | prev | next [-] | |
Possibly content that appears only briefly? I'm not sure otherwise. It seems wasteful. | ||
| ▲ | esseph 4 hours ago | parent | prev [-] | |
The most recent info of every single fucking thing on the web. Everything. Data locusts. | ||