I feel like we need a video on Dynamic Markov Chains. It's a method to create a markov chain from data. It's used in all the highest compression winners in the Hutter Prize (a competition to compress data the most).

▲

atiedebee 7 days ago | parent | next [-]

You mean the algorithm used in hook[0]? These are not really top performers anymore. PPM has generally performed better and nowadays it's LLMs and context mixers that are at the top of text compression[1]

[0]: https://mattmahoney.net/dc/dce.html#Section_421 [1]: https://mattmahoney.net/dc/text.html

	▲	AnotherGoodName 7 days ago \| parent [-]
		DMC is used in pretty much all of these, just not alone (that alg column is too small to capture this). As in go into the first open source entry which is #2 in this list, cmix, unzip the files, go into paq8.cpp and search for DMC. See "Model using DMC (Dynamic Markov Compression)" and associated code. In these cases DMC is one model mixed in with others and the best model for the current context is used. Hook exclusively uses DMC for outstanding results but the others use DMC as one of the prediction models.

▲

jadbox 8 days ago | parent | prev [-]

Make your own video then :)