Measuring programming productivity is hard. For example, take testing. It is certainly useful. At the same time, you can waste time on it in some situations.

When, what, how to test may be important for productivity.

I don't know whether LLMs are in the same category.