with zstd at level 16 with default params (dict size, ...). Serious compression starts at level 19 and with much higher dict sizes.

how is this an honest benchmark:

    bzip3 ... 12178M memory
    zstd  ...   687M memory

▲ myrmidon 7 hours ago | parent [-]

There is a comparison with "zstd -19" on the Silesia corpus, showing better compression ratio for bzip3 (47.2 vs 53MB) while being ~5 times faster (and using only half the memory).

Even if the examples are highly cherry-picked, it is quite suprising to me that such pareto-dominance is possible at all.

edit: Tested it myself and found that it often also does slightly worse than zstd -19 in compression ratio but faster (it was slower in one case on "uncompressible" input).

Compression performance vs "zstd -19" seems to depends a lot on actual input data in a very unpredictable way. I'd assume the benchmarks that they show are definitely somewhat cherry-picked.

	▲	dist-epoch 7 hours ago \| parent [-]
		having used zstd, it has terrible defaults, optimized for speed and low-memory. You need to change it's params (not just level and dict size) to get high performance. probably somebody should use a coding agent to do auto-research to optimize params for each compression algo, while matching one fixed goal - time, memory or size