Solving the Reproducibility Crisis in Scientific Research

A search for truth drives scientific research. One of the tests for research findings is that it must be reproducible research. If you are the only person who can observe a result then that result is invalid. It is important that members of the scientific community invest in replicating original research. As more and more scientists are able to show that a result is reproducible, it increases its credibility.

Reproducing research results is important. This idea has led to The Reproducibility Project: Cancer Biology. The project has $1.3 million in funding. Researchers and journals may face a challenge in their quest to replicate studies. As time passes, there may be less interest in the results of a replication study. The passage of time also means that the tools used to create the original data may no longer be available.

The science reproducibility crisis has many causes. Mentors need to train researchers how to manage, organize, and analyze their large data sets. The reproducibility crisis may exist because not being able to handle large data makes it hard for scientists to review and re-analyze each other’s data.

What is the Solution?

Transparency in research is critical for research reproducibility. Scientific transparency is the ideal but this may be difficult if there isn’t grant money to support replication studies. There is a proposal that aims to increase accountability in research. New incentives should encourage scientists to share all their findings, not just the exciting ones. Biochemist Bruce Alberts says we also need to remove the stigma associated with the retraction of a research paper. We need to encourage researchers to tell journals when they find an error in their previously published work.

Pre-registration of experiments may also address these issues of research ethics. Pre-registration would allow a scientist to publicly announce their research question and the methods they have chosen to allow them to test their hypothesis. When they have completed the experiment, they publish all their findings. By stating up front what they are going to do, it makes it harder for a scientist to hide unfavorable results.

Challenges in Irreproducible Research

Agreeing on the definitions of reproducibility is also important. Does reproducibility mean being able to get consistent results across experimental locations? Does it mean getting the exact same results as the original study? Or does it mean that the original study has been published with enough details for someone to be able to repeat it? There are many definitions chosen based on the true focus of the question.

The Open Science Collaboration recently conducted replication studies for 100 original research psychology articles. Only 36% of the replication studies had significant results, compared to 97% in the original papers. This may be discouraging but replicating these studies has created a scientific community framework that could help build more reproducible science. The project involved 270 authors and 86 volunteers.

The Open Science Approach

An Open Science approach could begin to change the current culture where only novel results are published. The Open Science Collaboration also built software called the Open Science Framework to make it easier for all the co-authors to contribute data. Its built-in control system makes it easy to see each person’s contribution. The framework would also make pre-registering studies easy to do. This software could be a step towards a greater transparency in research.

Reproducible science is a mark of credibility. The scientific community must test original research by trying to replicate its research findings. Reproducible research is an ethical issue especially when policies and treatments are based on scientific research. An Open Science culture may ultimately help scientists maintain high ethical standards. For us to achieve this goal, we must move towards greater scientific transparency.