> My hypothesis is that some of this a perceived quality drop due to "luck of the draw" where it comes to the non-deterministic nature of [LLM] output.

I think you must have learned that they’re more nondeterministic than you had thought, but then wrongly connected your new understanding to the recent model degradation. Note: they’ve been nondeterministic the whole time, while the widely-reported degradation is recent.

▲

bityard 6 hours ago | parent | next [-]

Er, no, I am fully aware that LLMs have always been non-deterministic.

▲

gilrain 6 hours ago | parent [-]

Your argument seems to be that a statistically-improbable number of people all experienced ultimately- randomly-poor outputs, leading to only a misperception of model degradation… but this is not supported by reality, in which a different cause was found, so I was trying to connect your dots.

	▲	zamadatix 5 hours ago \| parent \| next [-]
		Not everyone is reporting and the number of users is not consistent. On the former the noisiest will always be those that experience an issue while on the latter there are more people than ever using Claude Code regularly. Combining these things in the strongest interpretation instead of an easy to attack one and it's very reasonable to posit a critical mass has been reached where enough people will report about issues causing others to try their own investigations while the negative outliers get the most online attention. I'm not convinced this is the story (or, at least the biggest part of it) myself but I'm not ready to declare it illogical either.
	▲	bityard 5 hours ago \| parent \| prev \| next [-]
		No, that is not my argument, in fact I don't have any argument whatsoever. It was just a plausible observation that I felt like sharing. There's nothing further to read into it, I don't have a horse in this race.
	▲	furyofantares 5 hours ago \| parent \| prev [-]
		Not really, they said "some of this a perceived quality drop". That's almost certainly correct, that _some_ of it is that. When everyone's talking about the real degradation, you'll also get everyone who experiences "random"[1] degradation thinking they're experiencing the same thing, and chiming in as well. [1] I also don't think we're talking the more technical type of nondeterminism here, temperature etc, but the nondeterminism where I can't really determine when I have a good context and when I don't, and in some cases can't tell why an LLM is capable of one thing but not another. And so when I switch tasks that I think are equally easy and it fails on the new one, or when my context has some meaningless-to-me (random-to-me) variation that causes it to fail instead of succeed, I can't determine the cause. And so I bucket myself with the crowd that's experiencing real degradation and chime in.

▲

pydry 6 hours ago | parent | prev [-]

I wonder how well the "good" versions worked if you threw awkward edge cases at it.