> anecdotally based on their own subjective experience

So the “subjective” part counts against them. It’s better to make things objective. At least they should be reproducible examples.

When it comes to the “anecdotally” part, that doesn’t matter. Anecdotes are sufficient for demonstrating capabilities. If you can get a race car around a track in three minutes and it takes me four minutes, that’s a three minute race car.

▲

tshaddox a day ago | parent | next [-]

The term "anecdotal evidence" is used as a criticism of evidence that is not gathered in a scientific manner. The criticism does not imply that a single sample (a car making a lap in 3 minutes) cannot be used as valid evidence of a claim (the car is capable of making a lap in 3 minutes).

▲

Ianjit 19 hours ago | parent | prev | next [-]

Studies have shown that software engineers are very bad at judging their own productivity. When a software engineer feels more productive the inverse is just as likely to be true. Thats why anecdotal data can't be trusted.

▲

jimbo808 a day ago | parent | prev | next [-]

I have never once seen extraordinary claims of AI wins accompanied by code and prompts.

▲

llmslave2 a day ago | parent | prev [-]

Anecdotal: (of an account) not necessarily true or reliable, because based on personal accounts rather than facts or research.

If you say you drove a 3 minute lap but you didn't time it, that's an anecdote (and is what I mean). If you measured it, that would be a fact.

▲

ozim a day ago | parent | next [-]

I think from your top post you also miss “representative”.

If you measure something and amount is N=1 it might be a fact but still a fact true for a single person.

I often don’t need a sample size of 1000 to consider something worth of my time but if it is sample N=1 by a random person on the internet I am going to doubt that.

If I see 1000 people claiming it makes them more productive I am going to check. If it is going to be done by 5 people who I follow and expect they know tech quite well I am going to check as well.

▲

llmslave2 a day ago | parent [-]

Checking is good, you should probably check.

Every person I respect as a great programmer thinks agentic workflows are a joke, and almost every programmer I hold in low regard thinks they're the greatest things ever, so while I still check, I'm naturally quite skeptical.

	▲	rjh29 a day ago \| parent \| next [-]
		Doesn't help that many people use AI to assist with autocompleting boilerplate crap or simple refactors, where it works well, or even the occasional small feature. But this is conflated with people who think you can just tell an AI to build an entire app and it'll go off and do it by itself in a giant feedback loop and it'll be perfect.
	▲	ozim 20 hours ago \| parent \| prev [-]
		There are already people I follow who are startup owners and developers themselves saying they are not hiring “respectable developers” who are bashing agentic coding, they much rather hire junior who is starry eyed to work with agents. Because they see the value as they are running companies.

▲

hackable_sand 20 hours ago | parent | prev [-]

In this case it's more like someone simulated a 3-minute lap and tried to pass it off as a real car with real friction.