| ▲ | marginalia_nu 2 hours ago | |||||||||||||||||||||||||||||||||||||
To be fair the peer review model has been falling apart long before AI generated text was a thing, mostly due to the exploitative nature of academic journals leading to incentive structures that don't align with producing good science. Reviewers don't get paid and have zero incentive to do a good job. You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through. The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review. | ||||||||||||||||||||||||||||||||||||||
| ▲ | jrflo an hour ago | parent | next [-] | |||||||||||||||||||||||||||||||||||||
I have several papers published and this has been my experience too. One paper got submitted to dozens of different journals for several months as-is until it went through. Other times reviewers clearly did not read the paper in depth, and their critiques would have been answered if they read more than 2 pages in. More importantly, I've never felt that the review process actually added anything, especially for engineering journals. No reviewer ever sees the actual prototype, algorithm, raw data, etc. The only thing they can critique is your methods and literature review. No one checks if the thing actually works as you're claiming it does, which is 80-90% of the hard part. Certain fields, like mathematics, peer review is absolutely critical. But we've copy/pasted this process onto other fields where it's way less useful, which is why many fields are moving towards Arxiv instead of waiting for long, useless peer review process. | ||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||
| ▲ | ModernMech 2 hours ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||
Reviewers don't get paid but that doesn't mean they have zero incentive to do a good job. First, reviewers do get compensated, just not with money but with reviews from other researchers when they submit their papers. Not every review is good, but I've gotten enough helpful reviews from highly competent researchers whose time is very valuable which I didn't have to pay a cent for, that I'm incentivized to do the same for others. Also I want the system to work, so that's also an incentive. > You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through. This is not how to build a career in research, only high impact journals and conferences matter. > The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review. Peer review isn't supposed to detect fraud, it's not designed to do that, and the people reviewing aren't trained in fraud detection, nor should they be. The system is supposed to detect fraud through reproducible studies, but if you hop over to this thread (https://news.ycombinator.com/item?id=50538303) you'll see people complaining all over that the results are obvious so what was the point of the research. Then you might appreciate how hard it is to make a career off reproducing results. If we think fraud in research is such a problem, we have to actually allocate funds toward reproducing results. But to your point, peer review does stop plenty of shoddy research, just not all of it. It's not reasonable to expect peer review to stop all shoddy research, because we might be working on different and conflicting definitions of shoddy. | ||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||
| ▲ | BurningFrog 20 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||
Sounds like AIs should do the reviews! | ||||||||||||||||||||||||||||||||||||||
| ▲ | dominotw an hour ago | parent | prev [-] | |||||||||||||||||||||||||||||||||||||
That sounds like pull request review that is now totally useless. | ||||||||||||||||||||||||||||||||||||||