I'm more interested in the converse question: how well does an LLM perform as a compressor, compared to gzip (ignoring its insanely lower speed)?

gkbrk 5 hours ago | parent | next [-]

Top contestant in the Hutter Prize uses a neural network for compression. So fair to say, LLMs would perform pretty well compared to gzip.

	▲	computably 4 hours ago \| parent \| next [-]
		Even ignoring speed per GP, the Hutter Prize's metric includes the size of the decompressor. LLMs would be disqualified for being larger than 1GB.
	▲	londons_explore 4 hours ago \| parent \| prev [-]
		And the hutter prize disallows GPU's. If you allow use of a powerful GPU, you can do quite a bit better.

anax32 4 hours ago | parent | prev | next [-]

hallucinations are lossy compression artefacts

asdfsa32 5 hours ago | parent | prev [-]

lossless vs lossy is the question.

How lossy? Because I can lossy compress anything into 0 bits.

	▲	sire-vc 3 hours ago \| parent [-]
		0 bits is just a very small bloom filter.