Have they solved the issue where papers that cite research already invalidated are still being cited?

cogman10 7 hours ago | parent | next [-]

AFAIK, no, but I could see there being cause to push citations to also cite the validations. It'd be good if standard practice turned into something like

Paper A, by bob, bill, brad. Validated by Paper B by carol, clare, charlotte.

Paper A, by bob, bill, brad. Unvalidated.

▲

gcr 7 hours ago | parent [-]

Academics typically use citation count and popularity as a rough proxy for validation. It's certainly not perfect, but it is something that people think about. Semantic Scholar in particular is doing great work in this area, making it easy to see who cites who: https://www.semanticscholar.org/

Google Scholar's PDF reader extension turns every hyperlinked citation into a popout card that shows citation counts inline in the PDF: https://chromewebstore.google.com/detail/google-scholar-pdf-...

	▲	rtkwe 4 hours ago \| parent [-]
		That is a factor most people miss when thinking about the replication crisis. For the harder physical sciences a wrong paper will fairly quickly be found because as people go to expand on the ideas/use that data and get results that don't match the model informed by paper X they're going to eventually figure out that X is wrong. There might be issues with getting incentives to write and publish that negative result but each paper where the results of a previous paper are actually used in the new paper is a form of replication.

▲

reliabilityguy 7 hours ago | parent | prev [-]

Nope.

I am still reviewing papers that propose solutions based on a technique X, conveniently ignoring research from two years ago that shows that X cannot be used on its own. Both the paper I reviewed and the research showing X cannot be used are in the same venue!

▲

b00ty4breakfast 7 hours ago | parent [-]

does it seem to be legitimate ignorance or maybe folks pushing ahead regardless of x being disproved?

	▲	freedomben 7 hours ago \| parent \| next [-]
		IMHO, It's mostly ignorance coming a push/drive to "publish or perish." When the stakes are so high and output is so valued, and when reproducability isn't required, it disincentivizes thorough work. The system is set up in a way that is making it fail. There is also the reality that "one paper" or "one study" can be found contradicted almost anything, so if you just went with "some other paper/study debunks my premise" then you'd end up producing nothing. Plus many inside know that there's a lot of slop out there that gets published, so they can (sometimes reasonably IMHO) dismiss that "one paper" even when they do know about it. It's (mostly) not fraud or malicious intent or ignorance, it's (mostly) humans existing in the system in which they must live.
	▲	reliabilityguy 6 hours ago \| parent \| prev [-]
		Poor scholarship. However, given the feedback by other reviewers, I was the only one who knew that X doesn’t work. I am not sure how these people mark themselves as “experts” in the field if they are not following the literature themselves.