Importance of Replication Studies

What is Replication of a Study?

Replication of a study is repeating a study’s procedure and observing if the prior findings repeat in similar conditions. A study is replicated when the results of original study are as closely related to the newly collected data.

What is the Main Purpose of Replication Studies?

The credibility of scientific studies is established only if it is replicable under similar or closely related conditions. Findings collected from such studies gives greater validity to the originally conducted research. Furthermore, it means that the original study is more likely to be generalized for larger applications and future research scope.

What are the Types of Replication Studies?

Replication studies are broadly classified as:

1. Exact or Direct replications– Direct replication is the repetition of an experimental procedure to the exact degree as possible. It means that exactly same equipment, material, stimuli, design and statistical analysis should be used.
2. Conceptual replications– Conceptual replication is when research is conducted by using different methods to repeat the original study. Despite difference in methods, the new data is similar to the original study findings.

An Unfair Reputation?

A replication study attempts to validate the findings of a prior piece of research. By doing so, that prior research is confirmed as being both accurate and broadly applicable, since the replication study typically changes one or more variables of the original study, such as sample population, industry sector, etc.

From the perspective of contributing to scientific research, replication studies are important for the continued progress of science. Without validation, how do future researchers know whether to build on the findings of that original work?

However, from the perspective of the individual researcher’s career, replication studies do not carry the same weight as an original piece of work. Research Journals aren’t eager to publish such work because it doesn’t qualify as new research, and is therefore of less interest to their readers.

The Challenges of a Replication Study

Being able to replicate a simple study from a decade ago and observe how things may have changed over the last ten years, has saved the fortunes of more than one doctoral student struggling to find a research topic. However, replicating more recent research, especially projects making controversial claims, can prove to be extremely challenging.

If the original research team is unwilling to share all of their data, any colleagues looking to validate their work through replication are already placed at a disadvantage. Blaming the protocol of the replication study rather than reexamining the original results can easily dismiss any discrepancies from the original findings. For this reason, controversial studies can stand for years without retraction, even with multiple failed validation studies.

A Research Quandary

Operating in a highly competitive publish or perish environment, new researchers are presented with a quandary here. Research Supervisors encourage the use of replication studies to both provide a valuable contribution of validation to science, and to expose their students to multiple research methodologies.

However, those new researchers are eager to build their academic track records and therefore want to do research that will get published. Since replication studies typically don’t get published, that doesn’t help their track records.

Time to Overcome an Industry Bias

Even the best scientists make mistakes, and replication studies provide a valuable contribution in catching such mistakes before flawed studies get too widely dispersed. No one disputes the need for validation, especially in this climate of highly competitive research that is expected to produce maximum results as quickly as possible.

However, we are currently trapped in a vicious cycle of “prestige-chasing” that has to change. The less chance a proposed study has of being published in a prestigious journal, the less likely it is that the study will receive funding. Even if it does receive funding, researchers are unlikely to compete to participate in the study if they don’t see a chance of getting it published.

The days of open access journals and pay-for-print journals that charge article processing fees (APFs), have rapidly expanded the number of journals out there. Unless some of them start setting aside space for replication studies, the resources assigned to research validation will continue to decline, and flawed research that cannot be validated will be allowed to persist without challenge or retraction.