Remix.run Logo
xnx 5 hours ago

Does Internet Archive have a distributed residential IP crawler program? I would enthusiastically contribute to that.

There must be some mechanism to prevent tampering in such a setup.

progval 4 hours ago | parent | next [-]

The Internet Archive does not, but Archive Team does: https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior

xnx 4 hours ago | parent [-]

Yes! I'm running an instance right now.

gzread 4 hours ago | parent | prev | next [-]

No, IA does everything above board and even honors invalid DMCA takedowns.

Retr0id 4 hours ago | parent | prev [-]

> There must be some mechanism to prevent tampering in such a setup.

Trivial as long as they terminate the TLS on their end, not yours. So you'd just be a residential proxy.