Wouldn’t it be great during peer review if the editors and reviewers could sit down together and discuss the paper among themselves, hashing out any issues with the manuscript as a group before sending it back to the author? Many people think so and advocate such an interactive or collaborative process. Others are doubtful, reasoning that independent reviews, unbiased by other’s opinions, would be more useful. So, which adage applies to peer review—“Two heads are better than one” or “Too many cooks spoil the broth”?
Testing Collaborative Peer Review
Elsevier recently carried out an experiment to find out if collaborative peer review has advantages over the traditional review process. Using the Mendeley program, they allowed editors and reviewers to anonymously read each other’s comments during review and discuss the paper among themselves. Reviewers, editors, and authors were almost unanimous in reporting that the collaborative process gave a better review. The interactive discussion clarified doubtful points among the reviewers and provided a stronger consensus of opinion to the editor. Authors thought the collaborative review gave better direction on how to revise the manuscript. Downsides? There was only one. The interactive process took longer than the standard individual review process. It was more work for editors and reviewers.
Let’s find out what our experts think about Collaborative Peer Review.
Our Experts’ Opinions on Collaborative Peer Review*
Why not give editors the best information possible?
Collaboration: A Valuable Tool: I think collaborative peer review is a fine idea. Some manuscripts I’ve submitted have come back with wildly divergent comments. For one communication, two reviewers recommended against publication for opposite reasons: one said the proposed mechanism was not novel enough, the other said the mechanism was unprecedented. A collaborative review process would have allowed the reviewers to resolve these contradictory impressions among themselves instead of sending the editor a baffling pair of opinions. (My paper ended up being accepted.)
When I served as president of a condominium board of directors, I saw firsthand the advantages of group discussion. Decisions that eventually came out of the interaction were almost always slightly different and significantly better than the ideas of any one person going into the discussion. Once all the facts and opinions were on the table, they could be evaluated and the best plan put together.
Discussion vs. Execution: Critics of collaboration often have in mind the dangers of a plural executive. When a plural executive is split down the middle, no action can be taken. But this is not the situation with peer review. In the interactive review process, there is a single executive—the editor—who makes the final decision. The review process, collaborative or otherwise, simply provides the editor with information on which to base his decision. Why not give him the best information possible?
PhD, Organic Chemistry
6+ years of Scientific and Medical Writing experience, US
…likely to prevent unfairly positive or negative reviews…likely to require multiple rounds of editing, thereby increasing the time commitment
Collaborative review is a very intriguing idea, which has obvious pros and cons. In my experience, one of the problems associated with traditional peer review is reviewer bias. For example, many researchers recommend their friends and former colleagues as reviewers while excluding potential competitors in their own research field. The use of a collaborative review process is likely to prevent unfairly positive or negative reviews, thereby creating a more “level playing field” for researchers worldwide. It is also likely to lead to more thorough and thoughtful reviews.
Conversely, a collaborative review process might lead to one reviewer influencing or changing the opinion of other reviewers. While this could be a positive thing, there is also the potential of ulterior motives.
In addition, a collaborative review process is likely to require multiple rounds of editing, thereby increasing the time commitment to potential reviewers significantly. Because most peer-reviewers find that their time is already too stretched, journals might find it harder to find appropriate reviewers if the rejection rate of requests to review increases.
PhD, Molecular and Cellular Biology
10+ years of Scientific and Editing Experience, US
…great potential for improving the quality of papers published through clearing up misconceptions among reviewers
At face value, the idea of collaborative peer review seems like a good one, especially in the process of presenting the author and editor with a cohesive set of recommendations. Though it currently takes more time and is more work, the practice is still in it’s infancy. It could likely take the same amount of work or less with some tweaking of the model and styles of practice. For example, in the pilot for Neuron, the editors saw the most value in using the discussions for the more promising articles, so it might be wise to implement a structure which bars the papers with less potential from the further processes.
It would be interesting to see a study on peer reviewing and the impact of peer effects on a collaborative review process. While the effects may be negligible due to the high amount of expertise in their field, anonymity of the reviews, and discussion only after their initial summaries are submitted, it is still possible that legitimate concerns may be dropped from the conclusion due to the effect of peer behavior.
All in all, I would say that collaborative peer reviewing has great potential for improving the quality of papers published, through clearing up misconceptions among reviewers, as well as through providing better feedback to authors, who would then produce better revisions.
MS, Information Technology
11+ years of English-Japanese Translation experience, Japan
An open discussion would make it easier for the editor to identify whether a technical issue raised represents a significant flaw
Collaborative peer review can be lengthy; however, the time required for the reviewers to discuss the manuscript is time that, as author, you might not need to spend replying some contentious reviewer’s comments as any discrepancies would have been dealt with at the editorial level. I do wonder whether transparency would be also easier to achieve following a collaborative peer review approach (e.g., someone with a strong conflict of interest might not find it straightforward to “kill” a paper if in addition to providing an assessment of the manuscript, the reviewer needs to defend in front of other reviewers why the manuscript should not be published). An open discussion would also make it easier for the editor to identify whether a technical issue raised represents a significant flaw or could be addressed with few caveats. Overall, authors and reviewers seem to be positive and keen to engage in open discussions, and collaborative peer review would help the editor to take an informed decision. I’ll give it a try!
PhD, Biology
12+ years of Scientific and Editing Experience, UK
…participation in real-time discussion offers immediate intellectual rewards that may heighten engagement in the science at hand
Open access interactive peer review will likely produce dramatic improvements in the quality of scientific publications and may prevent fraud, although requirements of confidentiality may make the difference between open criticisms and politically oriented appraisals. In the current electronic climate, the collaborative features of Mendeley are obvious developments for scientific discourse, offering rapid global communication of opinions and even ideas, and demanding the commentary integrity of public forums that do not limit exposure to discerning contributors. Accordingly, proponents of collaborative peer review claim that the process will avoid dilution of the scientific literature with poorly executed studies that penetrate the traditional review process.
Moreover, participation in real-time discussion offers immediate intellectual rewards that may heighten engagement in the science at hand, promising a scientific utopia of scintillating analytical conversations such as those enjoyed by intellectually and financially elite thinkers of the 17th century. However, similar to the traditional review process, the ensuing quality control will continue to reflect time spent by unpaid and unrewarded reviewers with potentially persuasive and materialistic motivations. Thus, the expected improvements of scientific rigor and objectivity remain to be seen.
PhD, Cancer
12+ years of Scientific and Medical Writing experience, AU
…it should be an option available for the papers that could specifically benefit from additional collaboration
While using Mendeley for collaborative peer review requires some additional time for editors to prepare and for reviewers to complete their work, the benefits gained through having an interactive review process seem worth the effort. Based on a series of pilot studies using this new system, authors agreed that the discussions held during the review process helped to clarify how they should revise their papers. Editors also found the process helpful for reaching agreements on the questions that arose requiring further discussion.
The interactive discussions were found to be most valuable for papers that were thought to have potential for that journal by editors and reviewers. There was a higher level of interest in interaction among reviewers for papers the more that the reviews differed, and this new system gives reviewers the ability to go further into those discussions.
Since using this interactive review method is more time and labor intensive, instead of becoming the new standard it should be an option available for the papers that could specifically benefit from additional collaboration in the discussion process.
MA, Interdisciplinary Studies
6+ years of Research and Academic Editing experience, US
