Top 5 Factors Affecting Reproducibility in Research

How can science go wrong? This thought often comes to researchers. When you discuss with your PI about designing an experiment, and your PI informs you about how replicating the experiments and its results is crucial. You manage to plan the experiment, successfully complete it, and acquire results. Post analysis of these results from repeated experiments, you found that you are not able to reproduce the same results that you received the first time you did the experiment! Do you think your science has gone wrong? Well, read on to find out how reproducibility in research impacts your research and how you can improve at replicating and reproducing your results!

What is Research Reproducibility?

Reproducibility in research is a major factor that determines the uniqueness of research study. It means obtaining consistent results using the same data and protocol as the original study. For example, researchers confirm the validity of a new discovery by repeating the experiments that produced the original results. Moreover, other researchers in the field are also able to repeat the same experiments producing the results similar to the original results.

This exercise of producing valid experiments and results build authentic scientific discoveries which are more reliable in the scientific fraternity and help develop robust literature for future research studies of the same field. However, many research studies provide significant results but their findings cannot be reproduced. It disappoints the researchers and raises the question on whether the research question or the hypothesis of the research study is valid; and should the researcher plan a different methodology to approach the research question.

Although a challenging and an often discussed topic, scientific reproducibility has been an issue that institutes and funding agencies are trying to resolve. The American Society for Cell Biology (ASCB) identifies methods and best practices that would enhance the reproducibility in fundamental research. Furthermore, ASCB has given different terms explaining reproducibility as follows:

Direct Replication – Wherein the original experimental design and conditions are used to reproduce previously observed results.

Analytic Replication – A series of scientific findings are reproduced by reanalyzing the original data set.

Systemic Replication – Published findings are reproduced under different experimental conditions.

Conceptual Replication – A phenomenon is validated by evaluating a different set of experimental conditions.

Reproducibility Issues

Lack of reproducibility in research cannot be traced back to a single cause, but there are various reasons where research cannot be reproduced. The lack of research reproducibility has a negative impact on scientific output efficiency, scientific progress, and utilization of resources. Moreover, it impacts the trust of common people in scientific research. These problems are difficult to quantify but there have been estimates of financial expenditures wasted due to inappropriate research study designs. Furthermore, contributing to non-reproducible research gives incorrect results, leading into non-publication of the studies.

5 Factors Affecting Reproducibility in Scientific Research

1. Lack of Access to Raw Data and Methodologies

To reproduce original results, researchers must have access to the original data, protocols, and key research materials. Reproducibility in research is greatly hindered if there is lack of access to raw data and scientific methodologies. Systems such as data repositories and biorepositories should be made available to decrease the issue of data sharing.

2. Invalidated Biological Materials

Reproducibility in research can be complicated in case they cannot track the original source of biological materials. Moreover, research results can be significantly affected and replication of data from such experiments are reduced if contaminated biological materials are used ignorantly. Additionally, in studies related to primary cell culture, variations in gene expression, growth rate and migration occurs due to serial passaging which leads to non-reproducible results.

3. Lack of Knowledge to Analyze Data

Scientific research has led to creating complex data sets. These data sets are only useful to the research when analyzed and represented in the best statistical formats. However, many researchers lack the knowledge or tools to analyze or interpret the data correctly. This limitation causes misinterpretation of results and obstructs the research reproducibility.

4. Incorrect Laboratory Practices

Poor laboratory practices can result in poor experimental designs. Designing the laboratory experiments is a crucial step in producing an authentic research study. Incorrect research study or experimental practices can affect the results significantly. These designs without clear experimental parameters and methodologies might decrease the reproducibility. Improper laboratory practices may also lead to contamination of biological materials.

5. Undervaluing Negative Results

Scientific research is not only about the positive results or proving a hypothesis right, but also about justifying the negative results. Researchers are rewarded for publishing novel findings but not encouraged to publish the negative results. In fact, there is limited knowledge among the early career researchers regarding the platform to publish negative results. Awareness of publishing negative results could enhance the researchers’ efforts and avoid repetition of work which is otherwise difficult to reproduce and replicate.

Best Ways to Improve Reproducibility in Research

1. Open Science Approach

Open science gives access to scientific articles, published data, and collaborative research. It is an umbrella term that involves events aiming to provide open source of information, open collaboration, open peer review, open educational resources, research crowdfunding etc. Access to scientific literature and raw data aids researchers’ in creating reliable and replicable results. Thus, improving the reproducibility in research.

2. Use of Validated Biomaterials

Data reproducibility can be improved by using validated biological materials. Use of cell lines and microorganisms that confirm the phenotypic and genotypic traits  improve the researchers’ experimental outcome and results. Moreover, lack of contaminants will lead to authenticated and reliable data results, which are more likely to reproducible.

3. Train Researchers on Statistical Methods and Study Design

Researchers must be aware of the statistical methods and strictly adhere to the best practices in statistical analyses. Moreover, they should be trained to plan and structure the experiments properly. This could enhance the validity and authenticity of their statistical data and increase the chances of scientific reproducibility in research.

4. Practice Giving Thorough Description of Methods and Methodology

It is a good practice to give detailed methods and methodologies to other researchers for reference and reproduction of results. Researchers should also report their objectives related to experimental parameters, such as use of equipment, instruments, standards, number of replicates, interpretation of results, statistical analysis etc.

5. Promote and Publish Negative Data

Among many research institutes, negative data that does not support the hypothesis is not supported or encouraged to be published. Moreover, the researchers are advised to design experiments and continue to perform the research work until they achieve results that match the hypothesis. However, publishing negative data helps interpreting positive results from related studies and does not waste the researchers’ efforts and negative data.

Reproducibility is a mark of credible research. The scientific community is inclined towards achieving reproducibility in research and expects researchers to take responsibility for the accuracy in results. Furthermore, funders, publishers and ethical bodies should raise awareness about lack of reproducibility and promote better research practices among the early career researchers.

Have you encountered any reproducibility or replicating data issues during your research project? How did you overcome or manage it? Write to us or email us and discuss how can we deal with the problems related to reproducibility in research.