Any process that has an element of subjectivity to it will be open to claims of bias. From Olympic figure skating judges to academic journal editors, no matter how profound the expertise, they are human and therefore capable of being influenced by bias. Regardless of whether the bias is intentional or unintentional, the result is the same: a decision is made based on elements beyond the established criteria of the field.
In the area of scholarly publishing and academic research, editors make two critical decisions with regard to manuscripts submitted to their journals: 1) which manuscripts are accepted for submission or “desk rejected” (rejected at the outset by editors) and 2) who is selected to review the manuscript. These decisions have broad-ranging effects on the author, the quality and reputation of the journal, and the overall body of published scientific knowledge and research. Therefore, eliminating bias from the editorial process is critical.
Effect of Editor-Author Relations on Manuscript Handling Times
In this study, authors Sarigöl, Garcia, Scholtes, and Schwietzer (2017), pose the following question, “To what extent is the academic peer review process influenced by social relations between authors of a manuscript and the editor handling the manuscript?” Fundamentally, these researchers were looking to see if journal editors had a prior relationship with the author of a submitted manuscript, and if so, did this relationship have an effect on whether or not the manuscript handling time was shorter than average.
A prior relationship could mean that the editor had co-authored an article with this author (in the past) or had previously reviewed and accepted an article from this author. It was not possible to tell from the data if there were any relationships aside from these (e.g., membership on the same scientific board or collaboration in research projects). The study used 82,742 articles, published between 2007 and 2015, as meta-data, from the open access journal PLOS ONE.
The submission-to-acceptance time of manuscripts was the key factor in determining whether or not a prior author-editor relationship had a positive impact on manuscript handling time. Indeed, the findings of this research support the authors’ hypothesis and show that even when considering the corrections for other factors like time, experience, and performance, prior editor-author relations have a significant influence on the manuscript handling times effectively speeding up the editorial decision by an average of 19 days.
Distribution of Editorial Power and Manuscript Decision Bias
The focus of this second study was slightly different from the previous one, although its main theme was the same: Is there bias in the editorial process? Specifically, the research analyzes “the activity of nearly 7,000 editors at the mega-journal PLOS ONE over a 10-year period from 2006-2015,” and looks for an imbalance in the distribution of power among editors, specifically as it relates to quality output and handling times.
Researchers analyzed the citation impact of each editor’s articles and the handling time between manuscript submission and manuscript acceptance. They looked for correlations between citation impact and the length of time an editor had been in service at the journal and how swiftly the article moved from submission to acceptance. The authors also analyzed the number of articles processed by each editor, looking for patterns in distribution.
Findings from this research do support the idea that there is editorial bias at PLOS ONE with regard to social ties between editors and authors, as well as an unequal distribution of power among the editors. The “top-10 editors were responsible for 3,366 articles—corresponding to 2.4% of the 141,986 articles that were analyzed,” and the journal’s top editor reviewed approximately 27 times the number of articles as the average editor.
In addition, the research supports the idea that the longer an editor has been working for the journal, the less time they devote to the review process (i.e., quicker handling times), resulting in lower-quality outputs—articles accepted by these editors have lower citation impact than those from other editors who handle fewer articles and take more time to review. While the researchers’ analysis of this impact borders on ageism, the data does support the finding.
Recommendations and Concerns
The authors of both published studies make recommendations for more transparency and oversight in the editorial process of academic publishing. Citing their use of data processing techniques, Sarigöl et. al. recommend more robust editorial policies to combat bias: “Our approach thus offers a mechanism for journals and regulators to monitor such undesirable differences, motivating future data-driven editorial policies that can ensure a fair, transparent, and unbiased handling of submissions.”
Petersen recommends that journals record and evaluate editors’ activity levels in order to monitor for quality and distribution of power and, in the case of electronic-only journals, impose restrictions on the number of articles an editor can process a)at one time and b)over the course of a year. “By implementing such editorial policy changes at PLOS ONE, it would certainly make for an interesting policy experiment, providing an additional opportunity to observe shifts in editorial behavior, and possibly strengthening the case for tying the observed behavioral trends to outright misconduct.”
The case for monitoring editorial bias in academic publishing is certainly justified. Journal editors are responsible for building the body of work in scientific research—they determine what work is important enough to be published and what work meets the highest standards of quality. While there is something to be said for expertise and experience—an editor’s prior experience with an author may well validate a quicker manuscript handling time if that experience was positive—the potential for bias based on other factors must be avoided. The quality of academic research at large is dependent upon a good judgment based on transparent criteria.
References
- Emre Sarigöl, David Garcia, Ingo Scholtes, Frank Schweitzer (2017, March 3) Quantifying the effect of editor–author relations on manuscript handling times. Retrieved from https://rd.springer.com/article/10.1007%2Fs11192-017-2309-y
- Alexander Michael Petersen (2017, January 19) Quantifying the Distribution of Editorial Power and Manuscript Decision Bias at the Mega-Journal PLOS ONE. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2901272
