not rejecting reality, but increasing doubts about the effectiveness of these tests. and yes its subjective n=1, but I literally create and ship projects for many months now always from the same github template repository forked and essentially do the same steps with a few differnt brand touches and nearly muscle memory prompting to do the just right next steps mechanically over and over again, and the amount of things getting done per step gots worse and the quality degraded too, forgetting basic things along the way a few prompts in. as I said n=1 but the very repetitive nature of my current work days alwyas doing a new thing from the exact same start point that hasn't changed in half a year is kind of my personal benchmark. YMMV but on my end the effects are real, specifically when tracking hours over this stuff.

deaux 5 hours ago | parent [-]

You use Claude Code? Then harness changes will have had much more impact than any model "stealth nerfing".

	▲	anonyfox 5 hours ago \| parent [-]
		Both CC but also cursor with raw api calls.