Previously:

“Hi, tool author here.” A useful explanation of Burrows-Wheelers transform as used by bzip3: https://news.ycombinator.com/item?id=42902407

“bzip3 is not yet listed on the large text compression benchmark” It is now: https://mattmahoney.net/dc/text.html

(2 years ago, 176 comments) https://news.ycombinator.com/item?id=42899713

(4 years ago, 104 comments) https://news.ycombinator.com/item?id=31324439

▲

andruby 4 hours ago | parent | next [-]

> “bzip3 is not yet listed on the large text compression benchmark” It is now

And it comes in relatively well, in my opinion. I'm a compression amateur, but bzip3 is the first entry I recognize as a general purpose compression program.

	▲	idoubtit an hour ago \| parent \| next [-]
		Yes, bzip3 compresses that 1GB text file into 170 MB while zstd needs 213 MB (SI units, from my own tests with the "enwik9" file of the GP's page, and the latest releases of both programs, same commands). But the decompression (memory and speed) is a different story. On my desktop, decompressing with zstd requires 128 MB and 1 s, while bzip3 uses 3.2 GB and 90 s. Without `-b 511`, bzip3's ratio decreases and decompression requires about the same memory as zstd. But the decompression is still two orders of magnitude slower.
	▲	3 hours ago \| parent \| prev [-]
		[deleted]

▲

p-e-w 5 hours ago | parent | prev [-]

> “Hi, tool author here.” A useful explanation of Burrows-Wheelers transform as used by bzip3

FWIW, the Burrows-Wheelers transform is also used by bzip2, so this isn’t a new feature even though that quote kinda sounds like it is.

	▲	altairprime an hour ago \| parent [-]
		It’s also used in this paper blending an aspect of LZ77 with BWT: https://arxiv.org/abs/1502.05937 https://github.com/nicolaprezza/lz-rlbwt Which leads me to wonder if someone did, in fact, solve how to ‘combine zstd and bzip’ (so to speak) since the last time.