Remix.run Logo
himata4113 4 hours ago

From what I understand when you re-tokenize the output you can simply look at how often certain tokens show up and the position of them, enough of these matches would result it watermarked text.

Let's say we are at token 431 and there is 49% to generate token 1 and 51% to generate token 2, we apply bias to our token 1 which would make it win causing a repeating pattern invisible to the human eye.

Now you apply this to multiple tokens and a reversible source of random you have a pretty strong watermarking system... That is rather annoying to defeat as you essentially have to rewrite most of the text. The alternative is to use a diffusion model and spray some gaps across non-literal information such as ids, links, etc.

4 hours ago | parent [-]
[deleted]