Remix.run Logo
ihuman 4 hours ago

There's a difference between someone running a scraping tool occasionally and bots constantly and rapidly re-scraping the same site over and over again

Lalabadie 4 hours ago | parent | next [-]

The author's website is responsible for storing its own data. AI services currently treat the entire web as their storage and cache layer.

qbane 4 hours ago | parent | prev [-]

This is an important context: the site is more likely to be targeted by scrapers because it is a curated collection of scraped information.

0cf8612b2e1e 4 hours ago | parent | next [-]

Do the bots care? Seemingly very little intelligence in many of them. Could be as simple as the site has more pages, so more traffic.

Loot first, ask questions later.

everybodyknows 4 hours ago | parent | prev [-]

A curated collection of people who give away money. Was ever sweeter honey ever found in a pot?