Remix.run Logo
JonChesterfield 4 hours ago

This is really important. I am already tired of hoarding rclone copies of huggingface torrents and periodically checking them for bitrot. Pirateface is not a very helpful name for it though and it doesn't seem to have scripted torrent creation either.

Is hosting the same thing at 'academictorrents' actually a viable thing? In terms of peers from either send data to one another?

edit: looks like it can treat huggingface as a backstop for torrents that are otherwise not shared which is interesting, whole load of checksum nonsense I hand rolled disappear if bittorrent handles that. Except it doesn't work?

    Magnet soon No seeders yet - a torrent mints once a seeder packages this model.
So there's some per-torrent work to be done, but I don't know what that is, and I don't see how it can be based on files I have locally _and also_ be an exact match to files on huggingface. So I'm missing something here.

edit2: Looks like an implementation error. I can create a torrent from local files and upload it, but it won't have the huggingface backstop, and I can't specify it, so that doesn't actually achieve the claimed result. Before creating community torrents in that fashion would actually be of use, the submission page needs to allow pointing at the upstream.

Also, having everyone DIY a set of files -> torrent information is insane, this should not be a SKILLS.md, it should be a bash script that makes the thing.