The quality of a journal’s papers is largely determined by the quality of its peer review. This screening mechanism is designed to ensure that the data and conclusions presented in the papers are reliable and reasonable. But how do we determine the quality of a journal’s peer review process? Some reviews are rigorous, some pro forma. Journalistic sting operations have shown that nonsensical papers can pass peer reviews at many journals. Either the peer review did not exist or it failed. Until recently there was no way to determine at a glance whether a journal had a good peer review system in place. Now, perhaps there is.
PRE: A Peer Review Quality Rating
Peer Review Evaluation (PRE), owned by STRIATUS/JBJS, Inc., publisher of The Journal of Bone & Joint Surgery, is a system for evaluating and rating the peer review quality of individual papers and the journals that publish them. It’s a sort of a Good Housekeeping Seal of Approval stamp that a journal will display to show that its peer review is up to snuff. I logged onto the PRE website at pre-val.org to find out how it works. Not being a computer scientist, I struggled a little with the concepts of metadata and application program interface but believe I grasped the essentials.
How PRE Rates Peer Review
Once a journal signs up for the PRE service (there is a fee), it provides PRE with certain information about its review process. Consider the following for example:
- How many rounds did an article go through?
- Who participated in the review process (e.g., reviewer, associate editor, editor)?
- Who were the reviewers?
- What were the reviewers’ comments?
The computer program, presumably with the assistance of a live person, takes this data and assigns a score indicating the quality of the review process.
I’m in favor of an impartial peer review rating system such as PRE and think it would be a real benefit to readers and journals. But there are some questions I have about the PRE process. Mainly, how does PRE verify the data given to them by the publisher? For example, PRE can’t just take the journals’ word that distinguished professor ABC reviewed an article; the professor will have to be contacted to verify this. Would this verification violate the confidentiality that some reviewers expect? From the PRE website, it appears that divulging reviewers’ identities is optional, so I’m not sure verification would take place.
Our Experts’ Opinions on PRE
PRE will become more and more valuable because it will become the symbol not only for reliable articles, but also for highly reputable and trustworthy journals.
PhD, Molecular and Cellular Biology (10+ years of Scientific and Editing Experience, US)
The peer review process is designed to assure the originality, quality, and validity of articles published in scholarly journals. It has long been accepted as a system that guarantees the authenticity and honesty of the work done. However, with the increase in the incidences of faked data and plagiarism that have been brought to light recently, it seems there is now a need to assure the validity of the peer review process itself. There have been cases of bogus “peer reviewers” being used and some articles that have not been legitimately peer-reviewed have been published by reputable scholarly journals. The PRE system of validating the status of peer review for articles will serve as a valuable partner to the ORCID system for contributor identification. The use of the PRE label both on journals and on specific articles assures readers that the work has been reliably peer reviewed. In addition, PRE provides useful statistics about the rigor of each review, including the number of review rounds, how many and what kind of reviewer worked on the paper (editors, statisticians, general reviewers), whether the paper was screened for plagiarism, and the original submission date. PRE will become more and more valuable because it will become the symbol not only for reliable articles but also for highly reputable and trustworthy journals.
I do have concerns regarding how effective the proposed service will be and how easy it will be to truly ensure that peer review truly occurs as intended.
PhD, Molecular and Cellular Biology (10+ years of Scientific and Editing Experience, US)
There is no doubt that transparency in the peer review process is urgently needed. Specifically, and worryingly, peer review is often a biased process. Whether intentionally or not, reviewers often view manuscripts from their friends or that benefits their own research favorably, while delaying the publication of studies from competitors or that might contradict their own findings. Conversely, scientists writing articles often exclude competitors from reviewing their work, adding further bias to the process. As such, an evaluation system that normalizes the process among reviewers and journals cannot come quickly enough. Having said that, I do have concerns regarding how effective the proposed service will be and how easy it will be to truly ensure that peer review truly occurs as intended.
