Remix.run Logo
thaumasiotes a day ago

So, you have n options, you ask a question, and now you're down to m options.

The number of bits of information you gained is -log₂ (m/n).

If you ask a question which always eliminates half of the options, you will always gain -log₂ (1/2) = 1 bit of information.

If you go with the dumber approach of taking a moonshot, you can potentially gain more than that, but in expectation you'll gain less.

If your question is a 25-75 split, you have a 25% chance of gaining -log₂ (1/4) = 2 bits, and a 75% chance of gaining -log₂ (3/4) = 0.415 bits. On average, this strategy will gain you (0.25)(2) + (0.75)(0.415) = 0.8113 bits, which is less than 1 bit.

The farther away you get from 50-50, the more bits you can potentially gain in a single question, but - also - the lower the number of bits you expect to gain becomes. You can never do better than an expectation of 1 bit for a trial with 2 outcomes.

(All of this is in the article; see footnote 3 and its associated paragraph.)

The article explicitly calls out the expectational maximum of one bit:

>> You'll also notice that you're never presented with a question that gives you more than 1 expected information, which is backed up by the above graph never going higher than 1.

So it's strange that it then goes on to list an example of a hypothetical (undescribed, since the scenario is impossible) yes/no question with an expected gain of 1.6 bits.