I think the point is that a subjective experience, without accompanying data, is useless for making any factual claims wrt both ghosts and reported productivity-boosts from LLM usage.

Getting photos of ghosts is one thing, but productivity increases are omething that we should be able to quantify at some level to demonstrate the efficacy of these tools.

That's a silly thing to request from random people in the comments of an HN thread though ha

Measuring programming productivity is hard. For example, take testing. It is certainly useful. At the same time, you can waste time on it in some situations.

When, what, how to test may be important for productivity.

I don't know whether LLMs are in the same category.

Funnily enough, the one study that was done, indicates people misjudge the utility and impact of LLMs. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...

Nobody is saying LLM's can never be helpful, it's skepticism towards certain claims made around agentic workflows re. programming, such as claims of massively increased productivity or the idea that agents will replace most if not all programmers.