What Happens When Someone Fails to Reproduce My Research Study?

When you are conducting research under the pressure to “publish or perish,” the likelihood that your findings will not be reproducible at the first attempt. The chances of such an incident may be much higher than most research scientists would care to admit. If you receive notification that someone failed to reproduce your research, the situation may not be as bleak as you would think.

You Are Not Alone

An immediate emotional reaction to the news is completely understandable. Especially if you are still in the early days of your research career. That degree of emotion will also correlate to the extent to which your original findings had been publicized, and the corresponding rank of the academic journal in which they were published. The first thing to do is to take a deep breath and remind yourself of just how many variables are in play here. Let’s start with the researchers who attempted to reproduce your research study:

• Are they connected with the journal that originally published your research, or with a known institution? In some cases, unknown researchers do this to try and claim some media attention. They allegedly disprove a notable study only to disappear back into the ether when pushed for specifics.
• Did they follow your methodology to the letter or did they have to make any accommodations for budget, time, population sample, etc.?
• If it was a quantitative study, have they triple-checked their results?
• If it was a qualitative study, how did they handle observer bias?

Your Culpability

Sometimes, the efforts of the researchers attempting to replicate your research study appear to be in earnest. In such cases, the next step of the investigation turns the spotlight on you and, if appropriate, your lab and your team:

• Could the researchers access to all the raw data from your original study, or was some held back as a result of institutional policy?
• If you weren’t the Principal Investigator (PI) on the study, did your PI get adequate time to review the project before releasing the results for publication?
• Did the study stay on track and meet the planned deadline, or was there a last minute rush?
• Was there a substantive peer review or were the results published in an open access journal that didn’t quite take peer review seriously?
• Are the issues, as identified by the researchers, correctable?

An Imperfect World

Your last line of defense, though by no means the most ethical line to take, is to count on the unwillingness of journals to publish negative results.  Fortunately for you, replication studies have a hard time finding a receptive journal to publish. This is because those journals are under as much pressure to publish positive results as you are under to publish on a regular basis.

The final decision to be made depends on the alleged error and the potential consequences for human well-being. In some cases, human test subjects were involved and follow-on protocols have been implemented based on your initial findings. The response, in such cases, has to prioritize the health and well-being of those test subjects and any other patients involved.

On the other hand, it may be simply a foundational study. Any and all personnel connected to any such type of secondary study that incorporated that foundational data should be notified immediately. This ensures that the error isn’t allowed to compound.

Has your study been reported irreproducible? Have you come across any study in which the findings could not be reproduced? Please share your thoughts with us in the comments section below.