Nope; there is a bit more nuance and the distinction is important.

Compression is functionally equivalent to prediction when the data distribution is exactly representative of all future problems. The story changes drastically if you want generalization -- because the test distribution could be arbitrarily different, even if it had the same support! Eg: you observe a rare edge case in your training data and (lossy) compression could simply ignore it. But if you wanted generalization in that particular part of the space -- either because an adversary was testing you, or for design freedom where you choose to build in that specific corner -- then you don't just want data compression, but good prediction performance on a test distribution which peaks in that corner.

Assuming that the training data distribution is exactly the distribution you will ever care for is implicitly doing a lot of the heavy lifting in the claim that compression = prediction, and I'm peeved at how much this statement is unthinkingly repeated like a manifesto.

There is nothing natural about the training data distribution, especially if the data generation process is exploratory while the downstream usage will be exploitative.

▲

porphyra 2 hours ago | parent | next [-]

How does that invalidate the "compression is prediction"? If the future data is different and you failed to generalize, then the failure to predict means you got worse at compressing and have to spend more bits storing the new information. Conversely, if the future data is the same as that you've seen previously, you could predict it very well, and compress better as a result.

▲

gr_norm an hour ago | parent | next [-]

A maximally efficient compressor for the existing data distribution is not in general (and often will not be) maximally efficient for future data. The former may only be enabled by convenient local optima of the input distribution that a compressor accounting for the latter could not take advantage of.

For instance, consider the distribution of strings drawn from the language '0+'. Now consider the same for the language '[01]+'. A compressor looking at only the strings of the first language within those of the second can do a much better job if it does not have to account for future data.

This also relates distantly to the idea of overfitting in machine learning.

	▲	AlotOfReading an hour ago \| parent [-]
		It might not be optimal, but it's not wrong to call it the best available guess. That's basically assuming Occam's razor / Solomonoff induction. Hutter published a bunch of work about what it means to have an "optimal" compressor and famously spent the past couple decades running a compression contest on the idea that it'd lead to insights in AI.

▲

tcgv an hour ago | parent | prev | next [-]

"Compression can be prediction" would be more accurate.

▲

GornRok an hour ago | parent | prev [-]

[dead]

▲

variadix an hour ago | parent | prev | next [-]

I’m not exactly sure what distinction you are trying to make, but a compressor that generalizes from currently observed day to potential future data _is_ a better compressor, as it will have better predictions as it observes more of the source data distribution. Maybe you are making a point about whether compressors can “learn” after compressing the input data, but this also isn’t a fundamental property of compressors. Also compressors do have to deal with non-stationary data sources, so it also isn’t correct to say a compressor only models stationary distributions.

▲

canjobear 31 minutes ago | parent | prev | next [-]

Compression is still prediction, the predictions just might not be good.

▲

jbs789 2 hours ago | parent | prev | next [-]

That’s interesting.

Also sparked the thought that the assumption only holds if the future looks like the present.

▲

vanviegen an hour ago | parent [-]

If your compression algrotihm is deep enough (think LLM), it will capture a lot of abstraction, making it compress well even in future cases that differ from the passed but fit the scheme in some other way.

	▲	goatlover an hour ago \| parent [-]
		But not cases that don't fit the scheme, and the future always has some of those. We are continually surprised by what comes next, and have been notoriously bad at predictions that aren't clear trends (until those trends change).

▲

schopra909 2 hours ago | parent | prev [-]

100% agreed.