The peer review process is vital for the production of scientific knowledge. But this cornerstone of science may be crumbling, increasingly weakened by a confluence of factors. Just think, every year, the world’s 6–9 million researchers publish 2.5 million scientific publications in 28,000 academic journals, all of which must undergo peer review. The process seems straightforward enough:
(1) Submit to a journal
(2) Paper goes to referees
(3) Author gets a first decision of reject or revise
(4) Author resubmits a revision
(5) Author awaits a final decision after peer review and revision.
The author experiences along the way are less straightforward. Ideally, the peer review process should serve to ensure scientific integrity and enhance the papers’ quality. However, the process seems slow and inefficient to many; after all, a paper could be reviewed within 1-2 days. Indeed, another pressing factor is the poor incentives to overworked scientists for properly peer reviewing in academia, whose institutions increasingly operate on a business model, prizing output and competition alongside metrics and high rankings. For Ph.D. and young post-doc authors, a long waiting period to the first decision on a paper can affect their career advancement.
Moreover, among different fields, there is much variation in the duration of peer review (e.g. 572 vs. 249 days to full publication for conservation/ecology vs. genetics/evolutionary journals). Increasingly, a greater publication pressure has led to more rejections (3–6) for the average manuscript. In addition, it is harder for editors to find competent experts willing to peer review, worsening problems of its duration and quality. Not surprisingly, suggestions to improve peer review are plentiful: ranging from open access and preprints for accelerating scientific publications, to changing academic incentives, reviewers being paid money or getting public recognition.
New Study of Author Views
Hence, it would be valuable to know the author perception of modern peer review. To this end, a study by J. Huisman and J. Smits was recently published, entitled “Duration and quality of the peer review process: the author’s perspective” In it, a sample of 3500 review experiences of authors between 2013 and 2016—with 2516 from non-English speaking countries—were obtained from the SciRev.sc website and subjected to quantitative and qualitative analyses.
A structured questionnaire asked the author(s) the day-duration of different steps in the peer review process, as well as the number of referee reports they got and their (perceived) difficulty and quality. Two other data collected were an overall rating, on a scale of 0 (bad) to 5 (excellent), and the authors’ explanation, in their own words, for giving this rating. The data were analyzed for scientific fields separately and together, and according to the peer review outcome (accepted vs. rejected), with consideration to how the data may have been influenced by a journal’s impact factor. A multivariate regression evaluated the variation in the overall rating by authors, with missing values addressed by dummy variables.
The data set came from real authors (identities verified), for any type of research paper submitted to a peer-reviewed scientific publication, but being voluntary, it may not be 100% representative. Nonetheless, as used it allows for a broad insight into the peer review process as experienced by authors globally and in different fields. Now, what did they find out?
Key Points and Results
The work by J. Huisman and J. Smits has a number of interesting findings. First, the mean time to complete the peer review process is 17 weeks, but it varies two-fold across all fields. It was shorter for medical, public health and natural scientists (12, 13, and 14 weeks, respectively) and longer for psychologists, social scientists, and economists (20, 23, and 25 weeks, respectively). Second, there was also a two-fold variation (mean 39 days; range 29–64 days) among these fields in the time authors took to revise and resubmit their papers. Third, journals with a higher impact factor had a significantly shorter time to first decision, and hence total duration too, along with a faster immediate rejection.
Fourth, the dreaded “desk rejection”—which ought to be a quick end to your paper—can take >2 weeks at one-third of journals, and >4 weeks at one-sixth of journals (up 25% in certain fields)—which is a waste of time, and suggests editorial inefficiency. Fifth, the average number of referee reports was 2.2, with remarkably negligible variation across all fields. Sixth, not surprisingly, authors rated as more positive those experiences that led to accepted papers and that had a shorter peer review. Seventh, but counter-intuitively, the highest positive ratings were in fields linked not to a short but rather to a long peer review process.
Some Conclusions…
Based on their results, J. Huisman and J. Smits concluded that
(1) Undoubtedly, among the 10 major fields; there is currently considerable variation in temporal aspects of the peer review process.
(2) It is not fair to blame a long duration of peer review on referees taking their “sweet time” to do their part since some editors are also slow to do theirs.
(3) Authors should pay attention to the immediate rejection time at the journal, since this may indicate editorial inefficiencies there too.
(4) Evidently, the duration of the peer review in of itself does not determine author perception and judgment of it, but rather it is likely the expectations of the process in a particular field. (5) A journal’s impact factor is clearly linked to aspects of peer review, and it enhances the positive perception by authors.
(6) The qualitative results suggest that editors are not doing enough to communicate with authors nor to stay impartial and to adjudicate responsibly—related to this are contradictory reviews, which most authors understandably deplore.
Parting Tips
To shorten the duration of peer review, all academics should do them sooner rather than later; resist the “more-is-better” mantra, and carefully select suitable target journals. In order to increase the quality of peer review, referees can hone their skills at reviewing manuscripts; journals should tangibly reward their editors and reviewers for their efforts, and academic institutions should embrace “slow science” so that scholars produce fewer papers but more of a higher quality.
References
- Duration and quality of the peer review process: the author’s perspective. Retrieved from https://link.springer.com/article/10.1007%2Fs11192-017-2310-5
- New Scientist (2017, February 15) We need slow science to sow the seeds of future prosperity. Retrieved from https://www.newscientist.com/article/mg23331132-200-we-need-slow-science-to-sow-the-seeds-of-future-prosperity/
your article on data science is very interesting thank you so much.