Remix.run Logo
▲ ModernMech 2 hours ago

Reviewers don't get paid but that doesn't mean they have zero incentive to do a good job.

First, reviewers do get compensated, just not with money but with reviews from other researchers when they submit their papers. Not every review is good, but I've gotten enough helpful reviews from highly competent researchers whose time is very valuable which I didn't have to pay a cent for, that I'm incentivized to do the same for others. Also I want the system to work, so that's also an incentive.

> You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through.

This is not how to build a career in research, only high impact journals and conferences matter.

> The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.

Peer review isn't supposed to detect fraud, it's not designed to do that, and the people reviewing aren't trained in fraud detection, nor should they be.

The system is supposed to detect fraud through reproducible studies, but if you hop over to this thread (https://news.ycombinator.com/item?id=50538303) you'll see people complaining all over that the results are obvious so what was the point of the research. Then you might appreciate how hard it is to make a career off reproducing results. If we think fraud in research is such a problem, we have to actually allocate funds toward reproducing results.

But to your point, peer review does stop plenty of shoddy research, just not all of it. It's not reasonable to expect peer review to stop all shoddy research, because we might be working on different and conflicting definitions of shoddy.

▲marginalia_nu 2 hours ago | parent | next [-]

Well in the scenario you're a fraud or a hack, you're likely aware that you're a fraud or a hack, so you're unlikely to pursue a career in the traditional high impact journal sense. Though you may still wish to build credibility via volume of work, or to perpetuate some agenda, perhaps manufacturing a scientific basis for the magnetized water spiral business you're running.

We know replication doesn't prevent fraud either. Like in many fields most studies simply don't replicate. It's a whole thing.

It's very difficult to get institutions to work at scale when they assume every participant is acting in good faith. That may have worked 150 years ago when scientists were wealthy and doing the work for the love of the game (and also ironically not subject to peer review), but it doesn't work in the human centipede of fraud era of society.

▲cozzyd an hour ago | parent | prev | next [-]

We get compensated with a line item on our NSF synergistic activities document.

▲azan_ an hour ago | parent | prev [-]

> First, reviewers do get compensated, just not with money but with reviews from other researchers when they submit their papers.

It would make sense if publishing process was free and not extremely expensive. Publishers are sitting on pile of gold made from exploiting scientists and funneling public funds.