Randomized clinical trials are widely accepted as the gold standard approach to test new line of drugs, therapies, or treatments. They reduce bias by randomly allocating patients to either a treatment or a control group. In this scenario, variables other than the planned medical intervention remain constant between the two groups. This way any observed differences can be attributed to the treatment being tested. However, outcome switching is one of the fallacies of these clinical studies and impacts the regulatory approval decisions.
Pre-identification of Clinical Trial Outcomes
Transparent conduct of randomized clinical trials is essential, because the results influence life-or-death decisions and therefore, need to be trusted. Researchers must give details of the research protocol and identify the specific outcomes that will be investigated before the trial begins. Recommendations for planning a clinical trial are detailed in the SPIRIT and CONSORT guidelines.
Adhering to pre-identified outcomes avoids “cherry picking” results considered to be more interesting by the researcher while ignoring other important findings that do not align with the researcher’s expectations. This would undermine the credibility and reliability of the trial results. However, frequent discrepancies have been identified between the outcomes listed in trial protocols and those published in subsequent journal articles. These discrepancies can be dangerous.
Implications of Outcome Switching: Paroxentine
One highly publicized example of the harms of outcome switching was a clinical trial conducted to assess the safety and efficacy of paroxentine, a drug used to treat social anxiety disorder. The published study concluded that paroxentine was well tolerated and effective in adolescents with depression and the drug was approved for use. However, the study did not report the pre-identified outcome measures, and these did not reflect positively on paroxentine. Nineteen additional outcome measures were added during the course of the study, and only four of these showed positive results for paroxentine. These four positive outcomes were the only ones included in the published article. Clearly, the data as a whole told a different story to the one made public. A reanalysis of this study concluded that paroxentine is neither safe nor effective, and actually poses serious risks, including a higher chance of suicide. Yet, this drug has been prescribed to millions of people worldwide.
How Common is Outcome Switching?
You might think that outcome switching must be rare, given the damage it can cause. However, recent reports are showing that it is a common practice. This is most likely because researchers are under tremendous pressure to publish interesting findings in high-impact journals. For example, a team of social psychologists in Charlottesville, Virginia attempted to replicate the findings from 98 articles published in three different psychology journals. Only 39% of these replication attempts were successful.
In a new initiative, the COMPare team checked 67 trials that were published in five top medical journals between October 2015 and January 2016. On average, each trial reported only 58% of its pre-identified outcomes and added 5.3 new outcomes. Only nine trials were perfect. ClinicalTrials.gov was introduced in 2000, requiring researchers to register the study design and outcome measures of all planned clinical trials, before they were performed. The number of successful results for heart disease treatments fell from 57% before 2000 to just 8% after 2000, according to a study published in PLOS ONE. This provides overwhelming evidence that outcome switching has an impact on clinical trial results.
Preventing Outcome Switching: COMPare
The practice of publishing the favorable outcomes of clinical trials and ignoring important findings that do not fit with a preconceived hypothesis must be avoided. The CEBM Outcome Monitoring Project (COMPare) is now taking measures to prevent outcome switching through full exposure and transparency.
To identify cases of outcome switching, the COMPare team monitors all clinical trials published in the top five medical journals (NEJM, JAMA, The Lancet, Annals of Internal Medicine, and BMJ). COMPare are comparing the results published in each article with the registered pre-trial outcomes and any cases of outcome switching are immediately reported to the journal. All raw data are published online to ensure transparency of the procedure. These findings are currently being submitted for publication in academic journals. COMPare will then discuss the journals’ responses and use specific examples to demonstrate how misreporting takes place. These positive steps may encourage journals to take the problem of outcome switching more seriously in the future.
