Maybe I’m confused here, but it’s perfectly reasonable to just guess the same dice roll every time right?

kantahayashi an hour ago | parent | next [-]

Yes. There's no problem with choosing the same face every time. The problem is the probability it attached to the choice. Jev gave face 1 an 83% probability while the true probability is 1/6.

▲

seizethecheese an hour ago | parent | next [-]

Okay, I see, you're expecting Jev to properly give 1/6 probability for each option. This is different from my intuition of how LLMs work, where their probabilities don't really work like this (I would expect LLM to also do something like 0.83 for 1).

	▲	kantahayashi 30 minutes ago \| parent \| next [-]
		That's right. It's normal behavior of LLMs. But what matters is TypeSafe argues it's different exactly on this point. The selling point of Jev is "calibrated probabilities", so I checked it on probability problems.
	▲	maayank 13 minutes ago \| parent \| prev \| next [-]
		Jev and LLMs give other promises. Jev's RLCD training aims to make its probabilities calibrated such that given many cases where it assigns label Y about X% probability, Y should be the correct label about X% of the time.
	▲	robotresearcher 36 minutes ago \| parent \| prev \| next [-]
		Why? LLMs (along with other DNNs) model the distribution seen in their training data. Does the training data have dice roll examples being mainly 1? Maybe so! If that’s the case it’s an interesting example of LLM fragility since it’s failed to reason from the many (millions of?) times it’s seen stated in training data that each outcome has probability 1/6.
	▲	26 minutes ago \| parent \| prev [-]
		[deleted]

▲

sshine an hour ago | parent | prev [-]

Do you provide Jev that the probability is 1/6 and yet it gives back a probability that is way off?

	▲	kantahayashi an hour ago \| parent [-]
		Yes. For example, one of the prompts said "The die is unbiased: each of the six faces has probability exactly 1/6."

▲

alexmolas an hour ago | parent | prev | next [-]

I don't know if it's reasonable. What it isn't is calibrated.

▲

dgritsko an hour ago | parent | prev [-]

Reminds me of this... https://xkcd.com/221/