I must wonder whether it's their watermarking initiative[1] forcing certain logit choices to produce watermarked text that ultimately causing the model to behave in a dumb manner.

[1] https://support.claude.com/en/articles/16266773-how-claude-m...

▲

algoth1 5 hours ago | parent [-]

From the little i understand that wouldnt be an issue because the model is ‘just’ using interchangeable words in a mathematical non-random way. Like using the same number of adjectives and the exct same words, but in a order that wouldn’t be mathematically plausible unless it was the watermark

▲

demibabs 5 hours ago | parent | next [-]

I wouldn’t exactly put it like that. It’s moreso the model sometimes outputting non-optimal tokens in a way that’s detectable if you know the algorithm.

It seems possible for that to make the response “drift” far from what it would’ve been, because it’s constant entropy that adds up after time.

(However, according to Anthropic and Google, it doesn’t really impact the quality of responses. I find that a bit hard to believe, although those guys are much smarter than I.)

▲

algoth1 3 hours ago | parent | next [-]

Yeah, it’s hard to believe, specially when you are coding and there’s only one best way to do things, unless it plays with variable naming, or comments

	▲	demibabs an hour ago \| parent [-]
		Well the algorithm only increases the chances of certain words being chosen/not chosen, rather than guaranteeing it. If there’s a clear answer then that nudge won’t do anything. If the model’s most recent output is “for (let i = 0; ”, the likelihood of the next token being “i” is probably millions of times greater than any other possible token. Thus even if “i” is on the red list and has its likelihood decreased, it’s not going to suddenly choose another word. Put another way, on low-entropy tasks like coding, this style of fingerprinting is less effective and needs bigger sample sizes to be recognizable. That said, even small changes can dramatically affect output quality, which is why I’m still a skeptic.

▲

fuglede_ 4 hours ago | parent | prev [-]

One way to watermark (assuming temperature is otherwise positive) would be to output the most likely (or optimal) token every so often.

	▲	cl3misch 3 hours ago \| parent [-]
		But wouldn't you have to know the exact context before this token in order to verify the watermark? I.e. a paragraph wouldn't be sufficient; you would need the system prompt, previous prompts, and even hidden thinking?

▲

samrus 2 hours ago | parent | prev [-]

But its not just swapping the words out post hoc is it. LLMs are autoregressive, so weird word choice before would influence the probability distribution of all future tokens.

I feel like they thought it wouldnt be that bad, or it was a worthwhile tradeoff, but im getting the feeling it might be contributing heavily to opus5's uncanny communication style

As for the verbosity, my conspiracy theory is that they are token maxxing to hack revenue/enshitify the product in prep for their IPO