Remix.run Logo
singpolyma3 4 hours ago

How?

oblio 3 hours ago | parent [-]

Because OpenAI built their entire business around shamelessly scraping anything that had bits on it.

singpolyma3 3 hours ago | parent [-]

Maybe. But scraping isn't abuse. Seems a bit different?

PunchyHamster an hour ago | parent [-]

Given that the scraping doesn't do any rate limiting and pisses on robots.txt, yes it is abuse

singpolyma3 17 minutes ago | parent [-]

Is there any evidence OpenAI has been ignoring robots.txt for scraping purposes? AFAIK the main sources of that traffic are still unknown.