Remix.run Logo
finaard 7 hours ago

Depends - if parallel decompression of any bzip3 archive is possible I'd consider it fair, if it requires special flags on archive creation I'd consider it unfair. I didn't see any description about that on that page.

pbzip2 only can do parallel decompression on archives created with pbzip2, otherwise it'll fall back to single thread. There nowadays seems to be lbzip2, though, which claims to be able to add SMP support for standard bzip2 archives.

I'll need to try that next time I'm working with large archives - I learned about the pbzip2 limitations the hard way last time I was shuffling around a few multi-10GB archives, and was trying to speed things up fully utilising my 32 core threadripper.

sedatk 4 hours ago | parent [-]

> pbzip2 only can do parallel decompression on archives created with pbzip2

Oh I didn’t know that.