Statistics from Altmetric.com
Returning to Italy from Dublin, where the XIV Cochrane Colloquium took place, we had the opportunity to reflect on the main purpose of the colloquium, which was “… to plan for the future and to continue to make progress” (Mike Clarke, Director of the UK Cochrane Centre, http://colloquium.info/welcome.htm (accessed 5 November 2006)).
The Cochrane Collaboration, an international non-profit and independent organisation dedicated to providing up-to-date, accurate information about the effects of healthcare interventions, has been successful in producing and disseminating systematic reviews. Further progress was made during the Colloquium with the development of strategies aimed at increasing the quality of systematic reviews through the improvement of search strategies, critical appraisal of primary studies, data extraction, statistical analysis and reporting. In addition, new initiatives, such as the development of “umbrella” reviews and the use of Cochrane reviews in the production of treatment guidelines, were discussed.
However, there are other key issues, particularly related to the conduct and reporting of drug trials that deserve careful consideration by people involved in evidence-based health care. In this article we discuss these issues and suggest strategies that we believe would improve the clinical implementation of treatments based on the best available evidence.
(1) IMPROVING THE QUALITY OF PHASE III STUDIES
Strategy: changing current regulatory systems for drug approval
For the last 15–20 years systematic reviews have been successful in highlighting the methodological pitfalls that may confound the results and implications of phase III clinical trials (drug trials conducted for regulatory purposes). These pitfalls include a lack of statistical power, the use of placebo arms in disorders where active treatments were available, the use of inappropriate dose regimens, the employment of multiple comparison strategies to identify differences, the use of complex outcome measures that are almost never employed in practice and, lastly, the very short lengths of follow up. Despite recognition of these problems, there has been no significant change in the design and reporting of phase III trials. We argue that such a change cannot be implemented without a radical review of the processes of drug approval. The Food and Drug Administration (FDA) and the European Medicines Agency (EMEA) are the regulatory bodies for drug approval and marketing in the US and European Union respectively. The standard of phase III studies, therefore, reflects the present rules and requirements issued by regulatory agencies, and if a change in phase III studies is warranted, a change in the regulatory systems has first to be implemented.1
For example, in Europe new drugs can be evaluated with no comparison with active alternative treatments. As a consequence, phase III studies compare new drugs with placebo to make them eligible for registration. If comparisons are made with active-controls, phase III studies usually rely on demonstrating therapeutic “non-inferiority”, because this is in agreement with current EMEA requirements. However, if there is broad scientific consensus that active-control superiority clinical trials should be designed and powered to generate evidence of superiority between competing treatments, then this should be reflected in pharmaceutical legislation. Similarly, other positive changes in terms of trial design, analysis and reporting will probably be achieved only if they are a requirement of new legislation.
(2) SUPPORTING PHASE IV RANDOMISED STUDIES
Strategy: governmental support for the conduct of phase IV randomised studies
In recent years there has been a renewal of interest in pragmatic trials (also called practical, effectiveness or management trials), that is for studies that randomly assign real-world patients to licensed drugs with the aim of assessing their effectiveness.2,3 While explanatory or phase III trials answer questions about whether an intervention can work under ideal conditions (efficacy), pragmatic or phase IV trials attempt to answer questions about whether an intervention will work in the real world (effectiveness). Explanatory or phase III trials are usually carried out by the pharmaceutical industry, while pragmatic or phase IV trials are more often undertaken by groups of clinical researchers. Recent examples of pragmatic trials include the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE)4 and the Cost Utility of the Latest Antipsychotic Drugs in Schizophrenia Study (CUtLASS).5
Although the public health importance of these studies has long been recognised, to date only a limited number of such studies have been undertaken. We argue that physicians should be encouraged to develop skills in the design and conduct of pragmatic trials, and that government support should facilitate their development.
As part of the Pragmatic RAndomized Controlled Trials in Health Care (PRACTIHC) project, Treweek and colleagues developed the Trial Protocol Tool (TPT), a centralised resource intended to help researchers, particularly in low and middle income countries, design effective trials and to write high quality protocols (http://www.practihc.org).6 Although the TPT is a considerable achievement, it will have little impact if the trials themselves are not supported. In Italy a key pragmatic study was an unblinded trial of intravenous streptokinase in early acute myocardial infarction that enrolled 11 806 patients in 176 coronary care units.7 The first report of this influential study was published in 1986 and in subsequent years there was an ongoing debate about the need to support such research. In 2004 a Ministerial Decree was issued recognising the public health importance of that study and the need for establishing rules to help implement pragmatic independent phase IV clinical trials. In essence, the Decree states that if a set of conditions are met: the study coordinating centre is independent of drug company support; study results can be disseminated autonomously; there is no personal financial interest in studying the drugs included in the trial; the study drugs are licensed for the indication to be investigated, then the National Health Service (NHS) supports the conduct of the trial in three ways—drug costs are paid by the NHS; there are no fees for submitting the study protocol to the local ethics committees; continuing medical education credits are provided to local investigators. It is too early to assess the impact of this Decree on the conduct of clinical research, but it represents a significant change from a situation where research governance processes were hindering pragmatic trials, leading to a loss of important evidence-based medical information.8
(3) DEVELOPING INNOVATIVE RESEARCH HYPOTHESES
Strategy: recognising the value of everyday clinical practice, not ignoring it
In all fields of medicine, there are discrepancies between treatment recommendations and everyday clinical practice. These differences are exemplified by the prescription of psychotropic drugs where usage is only partially consistent with indications derived from clinical trials and systematic reviews. Furthermore, the effect of psychotropic medication is often markedly different from that measured in experimental conditions. One critical view of drug trials is that they encourage the prescription of drugs rather than addressing clinically relevant questions. This raises the question of how such questions can be identified and addressed.
Pharmacoepidemiology is the study of the use and the effects of drugs in large numbers of individuals.9 It describes how drugs are prescribed and used, investigates reasons underlying prescriptions and monitors outcomes and the variables that may affect these outcomes. We argue that, since pharmacoepidemiology produces “real-world” evidence—or “medicine-based” evidence—it should increasingly be employed to recognise areas of uncertainty and define clinically meaningful research questions.10 In this model evidence, generated by means of experimental studies, is implemented in clinical practice, and questions generated in clinical practice through pharmacoepidemiological study are used to develop innovative research hypotheses. These hypotheses are then evaluated in experimental studies with the aim of addressing questions that are clinically relevant. It would seem essential that people who promote evidence-based health care recognise the value of everyday clinical practice. Epidemiological studies nested in routine clinical activity provide a powerful tool for discovering areas of clinical uncertainty and offer an infrastructure for experimental studies in typical patients and settings.
(4) INCLUDING UNPUBLISHED DATA IN SYSTEMATIC REVIEWS
Strategy: improving the interaction between organisations involved in research synthesis and the pharmaceutical industry
Authors of systematic reviews frequently report the frustrating experience of contacting authors of primary studies to obtain unpublished data. From unpublished trials to individual patient data, the lack of access to primary data prevents the integration of study results to produce meaningful findings. The majority of drug trials are conducted or supported by the pharmaceutical industry, and in a competitive market companies will be reluctant to share negative findings. We would argue however that given the importance of these data, there should be a global initiative that aims to make all trial data accessible to organisations involved in research synthesis.
A first step in achieving this would involve the medicines regulatory authorities. Regulatory authorities should require, before reviewing applications for the licensing of new drugs, a statement in which the applicant agrees that all data must be accessible to specific organisations involved in research synthesis, and that accessibility will be preserved for a minimum number of years. Scientific journals could adopt a similar stance, requiring that agreement before accepting reports of clinical trials. The Journal of the American Medical Association has already successfully implemented a policy requiring an independent statistical analysis for industry-sponsored studies.11 There is no reason why a similar policy should not be implemented on such an important issue as data accessibility.
The availability of independent and reliable information on the effectiveness of treatments is essential for evidence-based practice. We believe that the changes we propose would encourage researchers to address clinically relevant questions and foster true innovation in psychopharmacology. Although current clinical practice is often evidence based, there is still much to be gained.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.