I have worked in this particular sausage factory. Multiple funded random replications are the only thing that will save science from this crisis. The scientific method works. We need to actually do it.

Replications don't have to be in the journals either. As long as money flows, someone will do them, and that is what matters. The randomization will help prevent coordination between authors and replicators.

In a better world, negative studies and replications would count towards tenure, but that is unlikely to occur. At least half of the problem is the pressure to continuously publish positive results.

▲

bonoboTP an hour ago | parent [-]

Regardless of what gets taught in school about science being objective and without ego, or having a culture of adversarial checks on each other etc., the reality is that scientists are humans and have egos and have petty feuds.

Publishing a failed replication of the work of a colleague will not earn you many brownie points. I'm stating this as an observation of what is the case, not as something that I think should be the case. If you attack other researchers like this and damage their reputation - even if for valid scientific reason - you'll have a hard time when those colleagues sit on committees deciding about your next grant etc.

Of course if you discover something truly monumental that will override this. But simply sniping down the mediocre research published by other run-of-the-mill researchers will get you more trouble than good. Yes it's directly in contradiction to the textbook-ideal of what science should be, as described to high school students, but there are many things in life this way.

Of course it can be laudable to go on such a crusade despite all this, and to relentlessly pursue scientific truth, etc. but that just won't scale.

▲

obviouslynotme 43 minutes ago | parent [-]

You are absolutely correct. Even distributing the replication around the world will only help so much. It's a small world out there and only smaller in the specializations.

That's why replication has to be required and standard. It will hurt to tear off the bandaid, but once the culture shifts, people will hesitate to publish mediocre research in the first place. Without mediocre research flooding the zone, real numbers will dominate and inflated expectations will wither.

	▲	bonoboTP 23 minutes ago \| parent [-]
		> That's why replication has to be required and standard. "has to be required"... This is a passive construct. Who will do the requiring and what precisely will motivate them to such a change and what will get them the buy-in from the other players in this whole ecosystem, especially the ones who provide the money? What if it turns out that those people who do the funding actually in the deepest of their deepest are fine with "groundbreaking" research results that simply sound like being "groundbreaking" research results to such an extent that their prestige and social status rises enough and are seen as someone who funds such research, instead of truly caring about the actual contents of said research? There is much more demand (backed with money) for (plausibly-claimable) innovation and breakthroughs than supply of real novel thought. It's a bit like the anecdote that all the True Cross relics across Catholic churches weigh more than the cross Jesus carried (not really true as a fact though). As long as there is such strong demand, the system will adapt to allow for the supply finding its way.