Remix.run Logo
twoodfin a day ago

If both of these are equally likely, you gain one bit of information, the maximum possible amount. If you already have other information about the situation, you might gain _less_ than one bit on average (because it confirms something you already knew, but doesn't provide any new information), but you can't gain more.

Can’t gain more!

The core confusion is this idea that the answer to a yes/no question can’t provide more than one bit of information, no matter what the question or answer. This is false. The question itself can encode multiple bits of potential information and the answer simply verifies them.

Dylan16807 a day ago | parent [-]

> Can’t gain more!

"you might gain _less_ than one bit on average [...], but you can't gain more."

On. Average.

That's a true statement. Can't gain more than one bit on average.

twoodfin a day ago | parent [-]

I’m not arguing with that, it’s basic information theory.

One bit, however, is not “the maximum possible amount” you can gain from an oracular answer to a yes/no question. The OP covers exactly this point re: the “Guess Who?” game.

Dylan16807 a day ago | parent [-]

The start of this comment thread was a complaint that OP is showing more than one bit expected for certain yes/no answers. Not best case, expected.

That's why people are talking about the maximum expected value.