The 2020 Peer Review Week, from 21 to 25 September, is dedicated to “Trust in Peer Review.” Nearly everyone in the academic and scientific communities would agree on the importance of the peer review process. As well as ensuring the quality of published research, peer review can provide authors with important and constructive feedback. For journals, peer review protects their reputation by ensuring the work they publish is of a high standard. An article that receives poor reviewer reports is unlikely to proceed to publication without substantial changes.
Positive peer review reports are obviously good news for authors. On the other hand, negative reviews may be distressing, even if they are ultimately helpful. But what happens if the reviewer reports are mixed – some good, some bad? Even more confusing, what happens when two reviewers make different, but conflicting, suggestions on how to improve the paper? How should the author deal with this situation?
Why Are the Reviews Different?
While all peer reviewers are (or should be) experts in their field, this does not mean that they have all had the same experience. The reviewers will have had different careers, possibly in different cultures and countries. It is not surprising that there might be differences in their reports! Also, reviewers might place different levels of importance on various aspects of the paper. Some, for example, might consider poor writing or language – or even writing in a style that is different to their own – to be a serious problem. Others, though, will overlook this, as long as the science is sound.
Occasionally, you might come across a biased reviewer. This might be a researcher whose work conflicts with your own in some way, or perhaps believes that the research direction you have chosen is misguided. In this case, the researcher – whether knowingly or not – may let their own prejudices influence their review. If you think this might be the case, it is definitely worth letting the editor know, making sure to clearly explain your reasons.
Finally, most researchers are very busy people. Sometimes, peer reviewing slips down the list of priorities. A review that is rushed or incomplete may not be a true reflection of the value of your paper.
What Will the Editor Do?
The first person to receive the mixed peer review reports will be the journal editor. How the editor responds can depend on just how different the reviews are. The editor will also carefully consider whether the differences between the reviews focus on major or minor issues.
Sometimes, if the reports are drastically different, the editor might invite one or two more peer reviewers, to see if a “majority view” can be reached. Alternatively, the editor might give more weight to the more experienced or well-published reviewer, or the one that they have had a longer relationship with.
Editors, too, are experts in their fields, and will be experienced in using their own judgement in responding to mixed reviews. It is important to remember that it is the editor, not the reviewer, who makes the final decision on whether to accept or reject a paper. This decision will depend on many factors.
So, You’ve Had Mixed Reviews…
Receiving mixed reviews can be difficult for an author. Sometimes, editors might suggest which comments to prioritize – and it’s a good idea to follow these suggestions! If this is what happens, it means your article still has a good chance of being published.
Sometimes, though, editors do not, or cannot, offer any such advice. If you find yourself in this situation, however, there is no need to despair.
The first thing you should do is to make sure you have enough time to properly consider and respond to the reviewer comments. Be realistic about what you can achieve in the time allowed. If necessary, ask the editor for more time. Try to do this as soon as you can – don’t wait until the deadline has already passed!
Next, you might try listing out all the reviewers’ comments according to whether they require minor or major revision. This will also help you to spot any contradictory comments. If there are any comments you don’t understand, ask the editor for clarification. Try to plan for approaching the revisions in a logical order.
Revising Your Work
Now you are ready to start making changes to your paper. First, tackle the easy revisions. Then consider the more major comments – these will likely take more time to address. Remember, you can always ask an experienced colleague for help or advice. Make sure you are satisfied that you have thoroughly addressed these non-controversial comments before moving on.
You should now be left with only the contradictory comments. Try to approach one issue at a time. Read each comment carefully and take the time to think about which suggestion to follow. It is a good idea to make notes of your reasons for choosing one suggestion over another, in case the editor asks you to explain your choice. This will also help you to be sure of your decision.
Try to treat each set of contradictory comments independently. For example, if you chose Reviewer A’s suggestion for one issue, this does not mean that Reviewer A is the “right” choice throughout; on a different point, you might go with Reviewer B.
You should consider writing a covering letter to the editor to explain that some of the comments were contradictory, and that you used your own judgement to decide which suggestion to follow. For the most major changes, you might briefly explain your choices.
The last thing you should do is to carefully read through your paper to check that it still makes sense after the revisions. It can be easy to make silly mistakes or omissions when you are making substantial changes. Try to read the paper as if you are preparing it for submission to a journal for the first time – it needs to be just as polished and complete.
Have you ever had a paper with mixed peer review reports? What action did you take? Share your thoughts and experiences in the comments below.
