The COVID-19 pandemic has affected almost every industry, around the world. The academic publishing industry is no exception. Whether it is dealing with the sudden change in research priorities, or the influx of COVID-focused articles, publishers have been forced to adapt – and fast.
The pandemic has produced a flood of information and misinformation – and put many sources of funding at risk. Preprint servers, which share research papers before they have been peer-reviewed, have become hugely popular. Amid all these changes in the world of scientific publishing, researchers may need to change the way that they read papers.
The Surge in COVID Preprints
In the few short months since the pandemic began, more than 6,000 COVID-related preprints have been posted. While some of these contain important, even vital, information, others are clearly poor quality. Crucially, papers posted on preprint servers have usually not been through the peer review process. This can result in the spread of misinformation, either intentionally or unintentionally. For example, a now-withdrawn preprint noted similarities between the new coronavirus and HIV. The article triggered widely-repeated claims that the new coronavirus is man-made, despite the objections of scientists and others.
The purpose of peer review is to ensure that published research is “good science.” Without this, how can scientists and the general public decide which research to trust?
Preprints: To Trust or Not To Trust?
When overwhelmed with information, people tend to search for ways to sort through the data. Faced with the multitude of COVID preprints, researchers may seek out names and institutions they are familiar with. This is understandable, as these people and places have earned their reputations as trusted sources. However, this approach has a downside.
According to Theo Bloom, the editor of the journal BMJ, this shortcut can favor bias over new ideas. This is a particular risk when it comes to articles on COVID-19. The pandemic began in China and quickly spread to other Asian countries and Europe, before reaching the US. This means that some of the most important COVID research came from scientists based in China and elsewhere in Asia, who are less likely to be familiar to an international audience. These papers helped to identify the virus, explain how it spreads and how it affects the body. Discounting these papers simply because the authors are unfamiliar could come at a great cost.
In another example widely reported in the media, a paper from China noted possible benefits of the malaria drug hydroxychloroquine in treating COVID-19. Journalists (and a certain President) seized on this claim and it was widely repeated. However, when large-scale studies were carried out, they concluded that hydroxychloroquine is not useful for COVID, after all.
With so many COVID articles available, and numerous high-profile cases of misinformation, how can researchers decide which papers are valid? The key is in the approach to reading the papers.
How to Read Preprints (and Papers)
According to Jonathan Kimmelman, a Professor of Bioethics at McGill University, researchers should start with a very simple question: “Can I trust what I’m reading here?”
Kimmelman agrees with Bloom: trusting papers only from recognised authors and institutions is a common practice, but perhaps not helpful. As well as ignoring researchers who are little-known internationally, this method also neglects young or junior researchers. These scientists may not have built a lengthy publication list – but that does not mean that their ideas are not useful or valid. On the other hand, even experienced and well-known researchers can make mistakes.
Rather than looking at the author or their institution, Kimmelman searches for signs of quality in the papers themselves. For example, studies with human subjects should be registered in appropriate government databases such as clinicaltrials.gov. Researchers must propose a specific hypothesis and give details of their experimental design. When reading a paper, you can check the reported methods against the database: if they don’t match, beware. In the case of the hydroxychloroquine study mentioned above, the paper differed significantly from the stated experimental plan.
Unfortunately, as there have been a few instances of misinformation in Chinese papers – such as the hydroxychloroquine study – some researchers view all information from the country with mistrust. This is highly unfair on many diligent researchers. Heping Zhang, a professor of biostatistics, says he has direct experience of important and reliable studies being rejected because the authors are Chinese. The lesson here is to try to approach all papers with an open mind, and judge them solely on the quality of the science. Bloom agrees, and notes that even the most famous journals must sometimes issue retractions.
Overcoming Bias
One way that researchers are tackling the problem of bias in publication is by forming international research teams. Building bridges in this way gives researchers access to new ideas and resources from around the world. For researchers at less well-known institutions, a co-author from a famous university can help their paper to get recognition. During the early weeks and months of the COVID pandemic, scientists in Hong Kong and the UK co-authored papers by Chinese researchers, thereby increasing their reach to the international readers.
How Can Open Abstracts Help?
There is, of course, one very common way that researchers quickly assess an article: they read the abstract. Abstracts are an important part of most journal databases. In the biomedical sciences, abstracts have been openly available in PubMed for some time. This is a great benefit to researchers that those in other disciplines do not have.
While abstracts are only a very short summary of a paper, they can help researchers decide whether the article is suited to their needs. As with the whole paper, readers can search the abstract for signs that the article is “good science.” For example: is there a clear hypothesis? Do the methods seem logical? Is the sample size appropriate? These are all questions that an open abstract might be able to answer.
These are only a few of the improvisations that researchers should consider while reading through research papers. Have you noticed the way that you approach papers during the pandemic has changed? What are your views on the quality of preprints? Let us know your thoughts in the comments below.
