Selection bias + perceptual adaptation is my experience. Selection bias happens when we play the probabilities of using an LLM and we only focus on the things it does really well, because it can be really amazing. When you use a model a lot you increasingly see when they don't work well your perception changes to focus on what doesn't work vs. the what does.

Living evals can solve for the quantitative issues with infra and model updates, but not sure how to deal with perceptual adaptation.

And survivor bias.

People who like the tool at first use it until they stop liking it -> "it got worse"

People who dislike the tool at first do not use it -> "it was bad"