Peer review is a way by which manuscripts can be assessed for their quality. Reviewers scrutinize the draft of a journal article, grant application, or other work and provide feedback to the author(s) for improving the text. Reviewers don’t just read the text, but also evaluate whether the research presented is sound, whether the methods used in the research are in keeping with basic scientific protocols, and whether the analysis of the results is valid. In addition, reviewers need to determine whether the subject matter (e.g., a research study) is in keeping with the journal’s particular field of study and is a novel scientific concept to warrant publishing it.
Even with the system in place, there are conflicting views about peer review and their merits. Most researchers believe that the current peer review system is lacking. However, they also agree that peer reviews are valuable in helping improve their papers, which help get them published. Still, a 2008 study revealed approximately one-third of those asked thought that the system could be improved.
Reviewer Anonymity
As mentioned, peer reviewers also assess grant applications and this process has its flaws. In an article in Times Higher Education, the author describes the grant review process as one of stifling innovative research and paring down each application into a numerical score. Funding sources presume that the reviewers are unbiased and knowledgeable. The funding agencies often base their decisions on these scores without having ever read the actual research proposal. The suggestion of having reviewers reveal their identities would remove any doubt of reviewer qualifications and hopes to hold the reviewers more accountable for these types of scoring protocols.
In an article published on F1000 Research, an open-research publishing platform, author Tony Ross-Hellauer describes some of the criticisms of the anonymous peer review as follows:
- Unreliable and inconsistent: Reviewers rarely agree with one another. Decisions to reject or accept a paper are not consistent. Papers have been known to be published but then rejected when resubmitted to the same journal later.
- Publication delays and costs: At times, the traditional peer review process can delay publication of an article for a year or more. When “time is money” and research opportunities must be taken advantage of, this delay can be a huge cost to the researcher.
- No accountability: Because of anonymity, reviewers can be biased, have conflicts of interest, or purposely give a bad review (or even a stellar review) because of some personal agenda.
- Biases: Although they should remain impartial, reviewers have biases based on sex, nationality, language or other characteristics of the author(s). They can also be biased against the study subject or new methods of research.
- No incentives: In most countries, reviewers volunteer their time. Some feel that this is part of their job as a scholar; however, others might feel unappreciated for their time and talent. This might have an impact on the reviewer’s incentive to perform.
- Wasted information: Discussions between editors and reviewers or between reviewers and authors are often valuable information for younger researchers. It can help provide them with guidelines for the publishing process. Unfortunately, this information is never passed on.
Because of these obvious dissatisfactions with the peer review process, a change to “open peer review” or “OPR” has been suggested. The premise is that an open peer review process would avoid many of the issues listed above.
What Is OPR?
OPR was first considered about 30 years ago but became more popular in the 1990s. Originally defined only as revealing a reviewer’s identity, it has now expanded to include other innovations. Although suggested as a means by which to help streamline the process and ensure honest reviews, the actual definition OPR has eluded those in the research and publishing fields. A plethora of different meanings prompted a study on the accepted definitions of the term.
In an article published on F1000 Research, author Tony Ross-Hellauer delves into the several different definitions of OPR and created a “corpus of 122 definitions” for the study. Ross-Hellauer reminds us that there is yet no standard definition of OPR. On the contrary, many definitions overlap and even contradict each other as follows:
- Identities of both author and reviewer are disclosed,
- Reviewer reports are published alongside articles,
- Both of these conditions,
- Not just invited experts are able to comment, and/or
- Combinations of these and other new methods.
These definitions are very open ended and those discussing OPR use one, some, or all of these in combination.
The Study on OPR
Ross-Hellauer reviewed the literature (e.g., Web of Science, PubMed, Google Scholar, and BioMed) for articles that mentioned “open review” or OPR and found 122 definitions of the term! The author then reviewed and classified all 122 definitions according to a set of traits that were new to the traditional peer review process. He concluded by defining seven traits of OPR and offered these as a basis for the definition of OPR as follows:
- Open identities: Authors’ and reviewers’ identities revealed.
- Open reports: Reviews published with the article.
- Open participation: Readers able to contribute to review process.
- Open interaction: Reciprocal discussions between parties.
- Open pre-review manuscripts: Manuscripts immediately available before formal peer review.
- Open final-version commenting: Review or commenting on final “version of record” publications.
- Open platforms: Review facilitated entity other than the venue of publication.
It appears from various studies that OPR is a valuable revision to the old peer review process and its anonymity. Although most agree that peer review has always been valuable, it is not without its faults. There is hope that the OPR system will eliminate much of these criticisms.
