Article Text

PDF

The worm turns: publication bias and trial registers revisited
  1. John Geddes, MD,
  2. Peter Szatmari, MD,
  3. David Streiner, PhD
  1. Editors, EBMH

Statistics from Altmetric.com

Readers of Evidence-Based Mental Health will be familiar with the fundamental rationale for systematic reviews: reviews should have adequate and well described methods and, in particular, avoid the biased selection of primary studies dependent on their results. A comprehensive and reproducible literature search is now recognised to be a key feature of systematic reviews. We know a lot about the advantages and limitations of electronic bibliographic databases, such as Medline.1 There is now a substantial body of empirical evidence about the various kinds of bias that can plague the identification and selection of studies for reviews, including publication bias (studies with significant results are more likely to get published than studies without significant results), English language bias (studies with significant results more likely to be published in English language journals), citation bias (studies with significant results more likely to be cited) and so on.2 As long as studies get published somewhere in journals that are indexed in one database or another, then they are potentially retrievable with an adequate search. Publication bias, however, remains the most devastating potential bias. The reasons for the non-publication of studies vary. The authors of a small study with unexciting results may have little interest or motivation in getting the study published. More pernicious is the situation where a substantial study is not published—and even purposefully suppressed—because the authors or funders do not like the look of the results.

Publication bias has been a perennial concern of reviewers and evidence-based practitioners, but a recent series of events has provided a disturbing example of the potentially serious effects of the failure to publish trial data or to make them available. It has emerged that trials of paroxetine, a selective serotonin reuptake inhibitor, showing negative or neutral results in the treatment of depressive disorders in children and adolescents have been not been published by the trials’ sponsor, GlaxoSmithKline. The resulting distortions introduced into systematic reviews are illustrated in this issue of EBMH in the study by Whittington et al [see page 115]. This analysis shows clearly that the inclusion of unpublished data very much reduces the apparent efficacy and increases estimates of the harm of certain medications, in particular paroxetine.

The central problem of how to encourage pharmaceutical companies to become more open about their data has exercised the evidence-based community for a number of years. Companies are clearly concerned about the effects of negative trials on sales of a drug in their drive to recoup the development costs and generate a profit for their shareholders. For this reason, voluntary initiatives to get negative results into the public domain have largely been unsuccessful. In 1998, a clinical trial amnesty was announced.3 Despite some initial support from companies (including the then GlaxoWellcome), the amnesty has largely failed—as companies seem to have decided that openness about their products might damage their commercial prospects. It has long been acknowledged that the only satisfactory way of protecting against publication bias is by prospective registration of trials, but such registers remain unsatisfactory.4

It is hardly surprising, therefore, that the current SSRI case has been seized upon by the general media, medical commentators, and regulatory bodies alike. Here is a great chance to put real pressure on the pharmaceutical companies to adopt a new policy of openness. On 2 June 2004, GlaxoSmithKline was sued by the Attorney General of New York for fraud over the suppression of trial data of importance in the estimation of the efficacy and the potential increase in suicidal risk of paroxetine in children and adolescents. The action was settled and, belatedly, the company has made full trial reports of the unpublished paroxetine trials available on its website (http://www.gsk.com/media/paroxetine.htm). The International Committee of Medical Journal Editors has recently announced that it will no longer accept trials that have not been entered onto a clinical trial database at inception.5 Sir Iain Chalmers, meanwhile, has argued that only legislation can force drug companies to make all trial results available.6

From the point of view of the evidence-based practitioner, we are usually realistic enough in our expectations of our treatments to know that no drug provides benefits without costs. What we need, though, is reliable information about both benefits and costs to give our patients proper information about what they can expect from our recommended therapies. Evidence-Based Mental Health has abstracted two published trials that compared SSRIs with placebo in the treatment of childhood depression7,8 and a summary of the section on the treatment of childhood depression from Clinical Evidence.9 Although we believe that the coverage we have given the issue has been true to the status of the published evidence, we are of course unable to report evidence that is not published or is deliberately suppressed. In our view, publication bias subverts evidence-based practice and can harm patients.

We at Evidence-Based Mental Health therefore join the calls on our partners in the pharmaceutical industry to be more transparent and open about their trial data. Failing to do so means, at best, that ineffective treatments are widely used in patients and, at worse, can lead to unnecessary illness and even death if the reported risks of harms are underestimated. We believe that the industry will want to avoid the possibility of increased legislation and cannot afford repeated public relations disasters of this kind.

Lastly, while it is clearly important to make the most of the SSRI case to push for a general improvement in the accessibility of trial results, it is a tragedy that this has involved a treatment for a mental illness. We and our patients are continually fighting stigma, and public perceptions of antidepressants are likely to have become even more negative as a result of this episode.

REFERENCES

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles