Remix.run Logo
ddxv 5 hours ago

I'm in a similar boat. Probably 99.999% is bots. I have nearly 1m unique "visitors" according to cloudflare and my real users are in the dozens a day. That being said, I love the open internet and am holding on to keeping as much open as I can.

fooey 2 hours ago | parent | next [-]

I shutdown all my little informational hobby projects this year that I've tinkered with for decades

They all shifted from mostly paying for themselves (or being so cheap it didn't matter), to essentially producing zero income while resource usage leapt up in magnitudes

I couldn't justify the stress and hassle of making sites, that a few dozen people a day might find useful, into some complex hyperscalable obligations just to feed the bots

hskalin 4 hours ago | parent | prev [-]

But what do these bots gain from this?

ddxv 4 hours ago | parent | next [-]

It's all OpenAI / Anthropic / Singapore/China based crawlers. I guess they gain data that I provide for free. I get very little crawling by Googlebot / Bing etc which barely even index my site, I only have a single page "indexed" by Google.

AlienRobot 3 hours ago | parent | prev | next [-]

Have you ever use an AI chatbot? It's basically a democratized scraper.

Type something stupid, it "visits" 200 websites and regurgitates some random gibberish.

bredren 4 hours ago | parent | prev | next [-]

Possibly content that appears only briefly? I'm not sure otherwise. It seems wasteful.

esseph 4 hours ago | parent | prev [-]

The most recent info of every single fucking thing on the web. Everything. Data locusts.