Yes. There's no problem with choosing the same face every time. The problem is the probability it attached to the choice. Jev gave face 1 an 83% probability while the true probability is 1/6.

▲

seizethecheese an hour ago | parent | next [-]

Okay, I see, you're expecting Jev to properly give 1/6 probability for each option. This is different from my intuition of how LLMs work, where their probabilities don't really work like this (I would expect LLM to also do something like 0.83 for 1).

	▲	kantahayashi 29 minutes ago \| parent \| next [-]
		That's right. It's normal behavior of LLMs. But what matters is TypeSafe argues it's different exactly on this point. The selling point of Jev is "calibrated probabilities", so I checked it on probability problems.
	▲	maayank 12 minutes ago \| parent \| prev \| next [-]
		Jev and LLMs give other promises. Jev's RLCD training aims to make its probabilities calibrated such that given many cases where it assigns label Y about X% probability, Y should be the correct label about X% of the time.
	▲	25 minutes ago \| parent \| prev \| next [-]
		[deleted]
	▲	robotresearcher 35 minutes ago \| parent \| prev [-]
		Why? LLMs (along with other DNNs) model the distribution seen in their training data. Does the training data have dice roll examples being mainly 1? Maybe so! If that’s the case it’s an interesting example of LLM fragility since it’s failed to reason from the many (millions of?) times it’s seen stated in training data that each outcome has probability 1/6.

▲

sshine an hour ago | parent | prev [-]

Do you provide Jev that the probability is 1/6 and yet it gives back a probability that is way off?

	▲	kantahayashi an hour ago \| parent [-]
		Yes. For example, one of the prompts said "The die is unbiased: each of the six faces has probability exactly 1/6."