| ▲ | thaumasiotes a day ago | |
So, you have n options, you ask a question, and now you're down to m options. The number of bits of information you gained is -log₂ (m/n). If you ask a question which always eliminates half of the options, you will always gain -log₂ (1/2) = 1 bit of information. If you go with the dumber approach of taking a moonshot, you can potentially gain more than that, but in expectation you'll gain less. If your question is a 25-75 split, you have a 25% chance of gaining -log₂ (1/4) = 2 bits, and a 75% chance of gaining -log₂ (3/4) = 0.415 bits. On average, this strategy will gain you (0.25)(2) + (0.75)(0.415) = 0.8113 bits, which is less than 1 bit. The farther away you get from 50-50, the more bits you can potentially gain in a single question, but - also - the lower the number of bits you expect to gain becomes. You can never do better than an expectation of 1 bit for a trial with 2 outcomes. (All of this is in the article; see footnote 3 and its associated paragraph.) The article explicitly calls out the expectational maximum of one bit: >> You'll also notice that you're never presented with a question that gives you more than 1 expected information, which is backed up by the above graph never going higher than 1. So it's strange that it then goes on to list an example of a hypothetical (undescribed, since the scenario is impossible) yes/no question with an expected gain of 1.6 bits. | ||