Remix clone Hacker News

new | show | ask | jobs Github

▲

thebigspacefuck 6 hours ago

The METR study showed even though people feel more productive they weren’t https://arxiv.org/abs/2507.09089

▲

knes 6 hours ago | parent [-]

the MTR study is a joke. it surveyed only 16 devs. in the era of Sonnet 3.5

Can we stop citing this study

I'm not saying the DORA study is more accurate, but at least it surveyed 5000 developers, globally and more recently (between June 13 and July 21, 2025) which means using the most recent SOTA models

	▲	rsynnott 5 hours ago \| parent \| next [-]
		> I'm not saying the DORA study is more accurate, but at least it surveyed 5000 developers, globally and more recently It's asking a completely different question; it is a survey of peoples' _perceptions of their own productivity_. That's basically useless; people are notoriously bad at self-evaluating things like that.
	▲	capnrefsmmat 5 hours ago \| parent \| prev \| next [-]
		It didn't "survey" devs. It paid them to complete real tasks while they were randomly assigned to use AI or not, and measured the actual time taken to complete the tasks vs. just the perception. It is much higher quality evidence than a convenience sample of developers who just report their perceptions.
	▲	bopbopbop7 6 hours ago \| parent \| prev [-]
		Yea cite the study funded by a company that invested billions into AI instead, that will surely be non biased and accurate.