> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.

METR reran the study early this year and, while they caveat it, this time they found a speedup, which is consistent with subjective estimates of productivity also having increased -- the simplest explanation is that subjective estimates exaggerate, but there's still a speedup with current models: https://metr.org/blog/2026-02-24-uplift-update/#wider-adopti...

(Nobody seems to cite the followup since it's not such a fun counterintuitive finding.)

▲

krupan 5 hours ago | parent | next [-]

"Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current productivity effect of AI tools."

That's more than a caveat. That says, this study is unreliable. And why? "The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI."

So, maybe people are fooling themselves, just like this article suggests (I almost wrote asserts, but saying, "I think..." isn't really an assertion)

▲

kalkin 4 hours ago | parent | next [-]

The people who chose to use AI would be the ones who would be fooling themselves, right? But they measured actually positive productivity there.

METR's concern makes most sense if it's about the inability to measure a consistent trend over time, not about the validity of the study for the population that participated.

▲

dcow 5 hours ago | parent | prev | next [-]

> The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI.

Why do you think this is? Who wants to spend an afternoon gluing API calls together when the robot can do it in 5 minutes?

▲

leecommamichael 5 hours ago | parent | prev [-]

Don't let the downvotes get you down. They don't know who funds or constitutes METR. If you actually read these papers, you'll find they cite themselves 4-5 times each publication. They also consider "hosting an HTTP server using Python" to be a "long time horizon" task.

▲

kalkin 4 hours ago | parent [-]

The OP cited a METR study... if you don't think they're credible, fine, but it's hard for me to see how it could be intellectually honest to cite them when their research is helpful to your argument and discredit them when it's not. One might even call this approach "fooling yourself".

	▲	leecommamichael 3 hours ago \| parent [-]
		The OP is not citing their claims, but their commentary. Do you understand why I find that to be different than citing their findings? We're not talking percentages here.

▲

notnullorvoid 3 hours ago | parent | prev | next [-]

Have you looked at the raw data of the follow-up? It doesn't paint a pretty picture, many tasks attempted but not completed, refusal to attempt without AI, higher overall estimates and completion times than the previous study (both for AI and Non AI).

There's no definitive conclusion about productivity that can be drawn from the followup. However I think it's reasonable to argue that AI use overtime has made devs worse without AI.

▲

Aurornis 5 hours ago | parent | prev [-]

By now, anyone writing articles in 2026 but ignoring the 2026 study is either intentionally trying to mislead people, or so uninformed on the topic that they're just regurgitating old anti-AI content.

If you want to talk about studies, we need to be talking about current studies with current tools. Not studies about how the situation was in the past with different tools.