| ▲ | throwuxiytayq 2 hours ago | |
What if the next token represents a wrong or low-quality answer, but would have only been picked 10% of the time, but now it's picked 20% of the time? Doesn't that obviously decrease the model quality, even though "it might have picked that token anyway"? | ||
| ▲ | dan-robertson an hour ago | parent [-] | |
It would be picked 10% of the time with watermarking. The randomness properties of the PRNG will be very similar to other random number generators, it is just chosen to be vulnerable to a particular cryptanalytic attack (that requires a private key known only to anthropic). I think of it like the Dual_EC_DRGB generator rather than a biased coin. | ||