I have read so many anecdotes about so many models that "were great" and aren't now.

I actually think this is psychological bias. It got a few things right early on, and that's what you remember. As time passes, the errors add up, until the memory doesn't match reality. The "new shiny" feeling goes away, and you perceive it for what it really is: a kind of shitty slot machine

> personally am frustrated that there’s no refund or anything after a month of degraded performance

lol, LMAO. A company operates a shitty slot machine at a loss and you're surprised they have "issues" that reduce your usage?

I'm not paying for any of this shit until these companies figure out how to align incentives. If they make more by applying limits, or charge me when the machine makes errors, that's good for them and bad for me! Why should I continue to pay to pull on the slot machine lever?

It's a waste of time and money. I'll be richer and more productive if I just write the code myself, and the result will be better too.

I think you’re onto something but it works the opposite way too. When you first start using a new model you are more forgiving because almost by definition you were using a worse model before. You give if the sorts of problems the old model couldn’t do, and the new model can do them; you see only success, and the places where it fails, well, you can’t have it all.

Then after using the new model for a few months you get used to it, you feel like you know what it should be able to do, and when it can’t do that, you’re annoyed. You feel like it got worse. But what happened is your expectations crept up. You’re now constantly riding it at 95% of its capabilities and hitting more edge cases where it messes up. You think you’re doing everything consistently, but you’re not, you’ve dramatically dialed up your expectations and demands relative to what you were doing months ago. I don’t mean “you,” I mean the royal “you”, this is what we all do. If you think your expectations haven’t risen, go back and look at your commits from six months ago and tell me I’m wrong.

Claude has been constantly terrible for the last couple of weeks. You must have seen this, but just in case: https://x.com/claudeai/status/1965208247302029728

Except this is a verifiable thing that actually is acknowledged and even tracked by people.

You are saying that you are writing mock data, boiler plate code all yourself? I seriously don't believe that. Llms are already much much faster in these tasks. There is no going back there.

This is equivalent to people reminiscing about WoW or EverQuest saying gaming peaked back then…

I think you’re right. I think it’s complete bias with a little bit of “it does more tasks now” so it might behave a bit differently to the same prompt.

I also think you’re right that there’s an incentive to dumb it down so you pull the lever more. Just 2 more $1 spins and maybe you’ll hit jackpot.

Really it’s the enshitification of the SOTA for profits and glory.