The Reproducibility Project: Cancer Biology to Replicate Only 18 Studies Now

In order to be able to translate basic biomedical research into clinical research, it needs to be robust and reproducible. This has major implications in the pharmaceutical industry.

The requisite for replication is distinct in cancer biology, with active efforts ongoing to attenuate disease progression via targeted therapy. As a result, the Reproducibility Project: Cancer Biology (RP:CB) began in October 2013, to test the replicability of 50 high-impact studies. Five years later, the focus has narrowed down to 18, with replication results for 10 studies already published on eLife.

Replication Crisis

Most studies require exhaustive details including restoration of experimental conditions in an industrial setting. While others are demarcated by research misconduct, including plagiarism and image manipulation, broadly setting back scientific advancements. The hurdles preventing experimental reproducibility include a lack of detailed protocols and the inaccessibility of reagents in an industrial setting. Failed attempts to replicate these experiments in the industry highlight the importance of experimental research. The RP:CB project is a collaboration with Science Exchange that found contract labs to reproduce key experiments from publications. The team expected to complete the replication experiments by 1 year. However, the results could not .

Replication Complication

A reason to scale down some replications resulted from the time it took to troubleshoot experiments. For example, technical details including cell density should be optimized experimentally to maintain a defined size to obtain reproducible results. While such processes of optimization are innate to a research laboratory setting, industrially such efforts are laborious, costly and time-consuming. The increasing costs and delays led organizations to contact original authors for materials and more information. The proposed replications also required mandatory peer review, further delaying the research outcomes. As a result, the organizers decreased the number of papers to 37 in late 2015 and to 29 in January 2017. Thereby, industrial efforts to replicate results led to hours wasted on remaking original cell lines and plasmids (intracellular DNA inserts) instead.

Research Outcomes

Replication results were obtained for 10 of the 18 studies in cancer biology, with mixed results, accessible on eLife.  Five of the studies were mostly repeatable, three inconclusive, and two studies were negative. However, other laboratories confirmed the original findings for the two negative studies. In fact, several other research groups confirmed the results of many of the 50 papers selected for replication. To summarize the project, the RP:CB team is now writing up the remaining eight studies including a meta-analysis to summarize the project. The team would publish the 11 incomplete studies soon. Although not as analytical as full replications, these will present enough information.

Lessons Learnt

A key lesson is to communicate clearly and pay attention to detail with technical protocols. This will enable labs within a variety of settings to staunchly build upon preceding work. Such attempts can improve research reproducibility and advance the scientific niche to develop further discovery. Regulating the following factors within academic labs might be helpful for similar projects in the future:

• Meticulously disclosing protocol details, potentially accessible through open access journals or repositories.
• Depositing original materials developed within experimental labs including plasmids and proprietary cells in life science repositories such as Addgene.
• Effective communication by sharing precise details without overlooking experimental conditions and nuances.
• Validating biochemical assays in academic labs to negate random noise and obtain exact results in practice.

Impact on Cancer Biology

The vigorous debate surrounding reproducibility of research findings in cancer biology is supported in detail by the RP:CB project. It is possible to integrate software to automate the process in theory and simplify reproducibility experimentally. Outcomes aim to optimize cancer therapeutics. Taking care of the above-mentioned factors could result in better and more fruitful outcomes for similar projects.

What are the steps do you think a researcher should keep in mind to ensure the research reproducibility? Please let us know your thoughts in the comments section below.