i wonder the replication rate is for ML papers

PaulHoule 2 days ago | parent | next [-]

From working in industry and rubbing shoulders with CS people who prioritize writing papers over writing working software I’m sure that in a high fraction of papers people didn’t implement the algorithm they thought they implemented.

	▲	avdelazeri 2 days ago \| parent [-]
		Don't get me started, I have seem repos that I'm fairly sure never ran in their presented form. A guy in our lab thinks authors purposefully mess up their code when publishing on GitHub to make it harder to replicate. I'm starting to come around on his theory.

▲

KingMob 2 days ago | parent | prev | next [-]

And most medical studies. It's just as bad as social psych, if not worse, because there's real money at stake in churning out new drugs.

▲

intalentive 2 days ago | parent | prev [-]

Nowadays everyone publishes their code. There’s typically a project page on github.io, a paper on arxiv.org, and a public repo.