| ▲ | famouswaffles 2 hours ago | |||||||||||||||||||
>You can for example talk to ChatGPT about religion - just tell it that you are on a hard journey and you hope God is on your side or something like this, and it will become a full on preacher of the gospels. In other words it hasn't really compressed the scientific knowledge with the religious knowledge, it contains both of them. In the first place, science and religion are not necessarily contradictory. Did you expect it to try to convince you to become an atheist? Newton was devoutly religous and many scientists are religious today. > If there was this objective type of compression and prediction, you would predict that the LLM would reach some kind of universal information state, but that doesn't seem to be the case? The objective is to predict, at minimum, internet scale text. Internet scale text is a massive trove of data with highly diverse and sometimes contradictory opinions and ideas, and the goal is to predict all of them well. Congealing on a single opinion or idea is obviously going to run counter to that objetcive. | ||||||||||||||||||||
| ▲ | thrw045 2 hours ago | parent [-] | |||||||||||||||||||
Yeah fair enough, maybe religion vs science wasn't the best example to use. It was the example I could come up with off hand. > The objective is to predict, at minimum, internet scale text. But then what does it compress exactly? How do you compress the answer to a question that has multiple answers? How do you pick "the right one"? That's at least what I get from this idea of compression that there is some congealing of information. | ||||||||||||||||||||
| ||||||||||||||||||||