We have this same point made every release. I wonder if models actually are getting better or if these companies are just secretly hamstringing the previous models before releasing a similarly performing model

▲

WillPostForFood 3 hours ago | parent | next [-]

Models are actually getting better.

▲

taneq 3 hours ago | parent | prev | next [-]

I think it takes a while to start finding the weak points in a model. It seems fantastic until you use it enough to find out that it hallucinates APIs, or can’t count letters, or uses too many em-dashes, etc. you don’t really know someone (or something) until you’ve lived with them (or it).

	▲	gazebo2 2 hours ago \| parent [-]
		I think there's definitely some quantization or nerfing happening post-release -- I definitely noticed that with Gemini Flash 3.7, it was super fast and accurate at the start and a few weeks later seemed to run in circles and overthink even a "what are you doing" message, all while doing almost identical work. I find it weird that we're so far into the LLM era and the post-launch degradation is still something of a conspiracy theory.

▲

becquerel 2 hours ago | parent | prev | next [-]

Try using a model from a year ago.

	▲	selcuka 33 minutes ago \| parent [-]
		What would that prove if we are testing whether they "are just secretly hamstringing the previous models" as GP claimed?

▲

KerrAvon 2 hours ago | parent | prev [-]

While IME 4.7 and 4.8 were not appreciably better than 4.6, Opus 5 was noticeably better, and Opus 5.5 is significantly better than Opus 5.