| ▲ | zergrush 4 hours ago | |
Viewing LLMS as compression is flawed because it shouldn't be possible to solve new novel problems. Zipping every single file in existence and then asking Winzip to produce a novel working zip file clearly doesn't work. Clearly LLMs are learning from data it is fed and a lot of what we do before LLMs also involved "plagirism". Even the greatest writers and artists would copy each other to master their styles and extract the dimensions. I think framing LLMs as compression is not only silly it demonstrates a poor understanding of the inner workings of LLMs | ||
| ▲ | Joel_Mckay 4 hours ago | parent [-] | |
Except a statistically salient search context is in proximity to something similar, and manual content curators drop their thumb on the scales weights to boost desirable outcomes in the quantized model. https://www.youtube.com/watch?v=LPZh9BOjkQs&list=PLZHQObOWTQ... The encoding of the LLM is almost always lossy compression, but we agree it is not real intelligence. As a sentient turnip, I likely wouldn't understand such things, but do recognize copyright bleed-through when it includes known obscure works. =3 | ||