| ▲ | typpilol 6 hours ago | |
Do we know how much data the internet archive has? Is it even viable to replicate to multiple regions if it's 1000s of PB? | ||
| ▲ | cookiengineer 6 hours ago | parent [-] | |
Anna's archive has the metadata on it. IA was around 300TB last time I checked. libgen was around 190TB. For my own at home cluster I decided to go for 512TB but I can't host nor upload in these bandwidth requirements from here. I started to build sth like a torrent splitter tool yesterday because I realized that all torrent clients just crash when you try to open, modify, or seed those torrents. Edit: correction, the IA is ~15PB big, brewster kahle mentioned it in the documentary (2014) | ||