> Gzip scales linearly. I can gzip a one terabyte file.

In part because gzip only has a 32KiB window size, and I think it'd be at least quadratic within that window if you were going for optimal compression.

▲

bob1029 4 hours ago | parent | next [-]

I'll concede the window part, but Gzip runs within the physical confines of a single cpu core and is typically entirely resident in local caches. The point is not just the quadratic scaling but also what it scales with.

Show me an LLM that can run at 300 megabytes per second. Even dedicated ASICs with weights burned in will never move this fast.

	▲	pishpash 3 hours ago \| parent [-]
		It won't move as fast as gzip provided the same resources, but to say that LLM's can never run at 300mbps on any hardware is ... a bold bet.

▲

Sesse__ 5 hours ago | parent | prev [-]

Match-finding does not need to be quadratic. However, truly optimal gzip block splitting is very slow, indeed.