Remix.run Logo
NewJazz 2 days ago

I think one would wish the famous ones to be more often replicable.

tomjakubowski 2 days ago | parent | next [-]

Nonreplicable publications are cited more than replicable ones (2021)

> We use publicly available data to show that published papers in top psychology, economics, and general interest journals that fail to replicate are cited more than those that replicate. This difference in citation does not change after the publication of the failure to replicate. Only 12% of postreplication citations of nonreplicable findings acknowledge the replication failure.

https://www.science.org/doi/10.1126/sciadv.abd1705

Press release: https://rady.ucsd.edu/why/news/2021/05-21-a-new-replication-...

esperent a day ago | parent | next [-]

This is at least partially a failure in publication. Once a paper is published, it's usually left up in the same state forever. If it fails to replicate, that data is published somewhere else. So when someone references the paper, and the diligent reader follows up and reads the reference, it looks convincing, just as it did when first published. It's not reasonable to expect the reader, or even the writer, to be so well versed in all the thousands and thousands of papers published that they know when something has failed to be replicated.

What we need is for every paper to be published alongside a stats card that is kept up to date. How many times it's been cited, how many times people tried to replicate it, and how many times they failed.

nitwit005 2 days ago | parent | prev [-]

This feels like some sort of truth telling paradox, where if you assume the study is true, then seeing a citation like this means it's likely not true.

dlcarrier 2 days ago | parent | prev | next [-]

Isn't the unexpected more famous than the expected?

sunscream89 2 days ago | parent | prev [-]

There may be minute details like having a confident frame of reference for the confidence tests. Cultures, even psychologies might swing certain ideas and their compulsions.