Remix.run Logo
▲ thrw045 2 hours ago

Maybe I misunderstand the technicalities behind compression and prediction but there is something wrong with them in terms of LLMs. You can't really compress knowledge all the time.

You can for example talk to ChatGPT about religion - just tell it that you are on a hard journey and you hope God is on your side or something like this, and it will become a full on preacher of the gospels. In other words it hasn't really compressed the scientific knowledge with the religious knowledge, it contains both of them. If there was this objective type of compression and prediction, you would predict that the LLM would reach some kind of universal information state, but that doesn't seem to be the case?

The other thing is, maybe there is a way to compress answers to questions that are objective. Like if I ask ChatGPT to extract the audio from a Youtube video and save it as a wav file. Relatively objective right? Well sort, but there's a gazillion ways you can do that. You can write a script in an endless amount of programming languages, you can segment the task into an almost innumerable set of sub steps (for example different scripts for different steps in the process), so what exactly does compression of that information mean when it can be compressed and predicted in so many different ways?

I don't really have a total answer here, I'm just thinking about all of these things and wondering.

▲ACCount39 an hour ago | parent | next [-]

Why not?

If our objective is, for example, "compress the sum of all of humankind's knowledge", then we have a bit of a conundrum: that objective contains human knowledge of math, but also of religion, and of soap opera plots.

You, a human, could easily make an argument that one of those things is more important than the others, and thus should dominate. But an optimal compressor does not care. All of those things are in its objective. It weights them accordingly. It will compress both math and soap opera into its "universal information state" - and if it finds patterns that happen to help compress both, then all the better for it.

As such, if we take "compressor" as LLM and "objective" as its training datasets, a capable LLM would be able to both solve math problems and give a preacher-like sermon. But it would not let its knowledge of religion intrude into its performance in math, or the other way around, to an unwarranted level. The compressor lives by "non-overlapping magisteria."

> You can write a script in an endless amount of programming languages, you can segment the task into an almost innumerable set of sub steps (for example different scripts for different steps in the process), so what exactly does compression of that information mean when it can be compressed and predicted in so many different ways?

It means that, instead of storing every possible permutation, we get a compressed representation that decomposes - separates "segment a video" from "write code in language A" or "use tool B". And then those elements can be recombined into the near-infinite permutations to decompress. The compressor finds the commonalities and exploits them to fit "infinite" into "finite" better.

▲janandonly 2 hours ago | parent | prev | next [-]

By that preacher’s argument you are ignoring all the PhDs who do believe in a God.

They might not be very vocal about their personal beliefs for obvious reasons.

▲thrw045 2 hours ago | parent [-]

OK fair enough, but it's the first example I could come up with. But I think the point still stands in that the LLM contains conflicting information, some about science and philosophy, and some about religion. It doesn't try to consolidate or "compress" conflicting information from what I've found.

▲zuminator an hour ago | parent | next [-]

It's possible that the LLM finds and compresses common elements of what we perceive as entirely disparate or conflicting information.

▲toomuchtodo 2 hours ago | parent | prev | next [-]

“Resolve” might be the word you’re looking for?

▲thrw045 2 hours ago | parent [-]

Sure.

▲rukuu001 an hour ago | parent | prev [-]

You want an objective truth machine?

▲prometheus1992 2 hours ago | parent | prev | next [-]

>>you would predict that the LLM would reach some kind of universal information state, but that doesn't seem to be the case?

maybe a 50T parameter will do better on this benchmark. you can already see signs of the models discussing trade offs in great length and this falls somewhere in that direction.

▲thrw045 2 hours ago | parent [-]

It would be interesting if that were the case.

▲famouswaffles 2 hours ago | parent | prev [-]

>You can for example talk to ChatGPT about religion - just tell it that you are on a hard journey and you hope God is on your side or something like this, and it will become a full on preacher of the gospels. In other words it hasn't really compressed the scientific knowledge with the religious knowledge, it contains both of them.

In the first place, science and religion are not necessarily contradictory. Did you expect it to try to convince you to become an atheist? Newton was devoutly religous and many scientists are religious today.

> If there was this objective type of compression and prediction, you would predict that the LLM would reach some kind of universal information state, but that doesn't seem to be the case?

The objective is to predict, at minimum, internet scale text. Internet scale text is a massive trove of data with highly diverse and sometimes contradictory opinions and ideas, and the goal is to predict all of them well. Congealing on a single opinion or idea is obviously going to run counter to that objetcive.

▲thrw045 2 hours ago | parent [-]

Yeah fair enough, maybe religion vs science wasn't the best example to use. It was the example I could come up with off hand.

> The objective is to predict, at minimum, internet scale text.

But then what does it compress exactly? How do you compress the answer to a question that has multiple answers? How do you pick "the right one"? That's at least what I get from this idea of compression that there is some congealing of information.

▲fc417fc802 16 minutes ago | parent | next [-]

An adjacent comment explains your misconception about the nature of what's being described as compression.

Separately I'd like to add that you seem to be focused on the concept of some sort of universal truth. Meanwhile compression has nothing to do with that. Input is entirely arbitrary - it can contain contradictory statements or even random noise. Obviously truly random noise isn't compressible but for example pseudorandom values derived from a seed are. At its core compression is merely finding a set of rules that can be used to describe the data in question.

▲codersfocus 43 minutes ago | parent | prev | next [-]

> But then what does it compress exactly?

Tokens into dimensions.

A good example is how humans talk about "left and right" in terms of political systems.

Humans are able to intuitively place these complicated multivariable economic-political systems into a one dimensional line that we call left or right wing.

Of course, this deep compression complicates things... why is national socialism considered right, while democratic socialism is considered left.

You can get more signal by increasing dimensions, so let's say instead just left or right, we have 2 axis one for economic and one for political. This would make things more clear. Even clearer if we add another dimension, let's say authoritarian or democratic. We can add yet another, socially liberal or conservative.

You can keep going, and as you add more dimensions it makes things clearer / more intelligent (to a limit.)

AI is doing this to text. It ingests text, and creates a token vocabulary. IIRC most models have a vocab of hundreds to low thousands of tokens.

This token vocabulary (how many distinct tokens it uses) is how many dimensions the AI thinks in, so to speak.

After this, there is the whole training process of how tokens relate to each other.

So your question of

> How do you compress the answer to a question that has multiple answers?

Isn't applicable, as it doesn't compress anything we would call a question or answer.

Instead, answers to questions are an emergent property of how text fragments relate to each other, and is why it can give contradictory answers.

▲michalsustr an hour ago | parent | prev [-]

You don’t - what’s reported is average compression, not for the particular examples you made. Some things are more compressible than others