That early 2025 METR study was particularly interesting because participants self-evaluated themselves as 20% faster, but the measurements showed they were actually 19% slower.

All the reports of productivity since then are self-reported, or using questionable measures such as SLOC and PRs, so it’s reasonable to say that productivity improvements are still unknown.

Unfortunately, METR hasn’t been able to replicate the study because they couldn’t find enough willing participants.