Statistics from Altmetric.com
“Efficiency is concerned with doing things right. Effectiveness is doing the right things.” Drucker, 1993
Typically, evidence-based practice (EBP) refers to health practitioners applying the best currently available research evidence in the provision of health services. In other words, EBP challenges practitioners to “do things right” and to “do the right things”. EBP originated in medicine, where an estimated 10 000 new randomised controlled trials (RCTs) are published every year but where an estimated 20%–40% of services still do not reflect the best research evidence.1 Related disciplines such as psychology have also embraced the EBP movement to bridge research and practice in order to improve outcomes for people with mental disorders.2 In children’s mental health, high levels of unmet service need suggest a strong role for EBP. At any given time 14% of children experience mental disorders that cause significant distress and impair their functioning, yet only 25% of these children receive specialised mental health treatment services.3 It is also clear that children’s mental health services often fail to reflect the best available research evidence, leading researchers to argue that EBP is an ethical imperative if we are to improve children’s mental health.4,5
Despite being widely advocated, EBP has nevertheless proved difficult to implement. To some extent implementation barriers are a result of a restricted focus on interventions designed to change simple behaviours performed by individual practitioners, such as prescribing by physicians. These interventions have had only modest effects and need to be integrated with larger organisational and system changes that support EBP.1 However, a greater challenge may be posed by controversies about EBP’s narrow definitions of “evidence,” particularly when applied in mental health.6 Here, we discuss the controversies with regard to implementing EBP in children’s mental health. We illustrate the issues based on our experience as researchers engaged in an ongoing partnership with children’s mental health practitioners and with the policy makers who manage services in British Columbia, Canada. We conclude with suggestions for rethinking EBP in order to improve children’s mental health.
CONTROVERSIES ABOUT EBP
Although EBP is widely promoted by researchers and others, controversies have emerged about defining “evidence” too narrowly, particularly in diverse fields like mental health. With its origins in the modernist scientific paradigm and its focus on rationalism and objectivity, EBP has privileged some kinds of knowledge over others.6 The hierarchies of evidence used in EBP relegate individual experience to the lowest rung and neglect qualitative research in favour of quantitative.7 The modernist paradigm is conveyed by language such as “knowledge transfer” which implies unidirectional transmission from researchers to practitioners, rather than reciprocity in the creation of knowledge.
Three particular controversies have emerged about EBP in mental health. First, EBP privileges RCTs that address efficacy in controlled research settings, but practitioners require research evidence on effectiveness in typical practice settings. What works in research settings may not be the same as what works in practice. Research that addresses the complexities of typical practice settings is lacking.2 Further, the reliance on RCTs leaves many relevant questions unanswered. For instance, therapeutic relationships are central to mental health practice but efficacy research typically focuses on techniques such as cognitive behavioural therapy (CBT) that can be investigated quantitatively.6 Finally, efficacy research has been used to limit the types and duration of services that are funded by managed healthcare organisations in the United States, fuelling concerns that EBP will be used too narrowly by policy makers in the service of efficiency.6
There are additional controversies about implementing EBP in children’s mental health. An understanding of healthy child development is central to children’s mental health. Development involves continuous and dynamic interactions between children and their environments over time, and is inextricably linked to the ecological context that children develop within; families, schools, and communities.2 The dynamic nature of development means that what is efficacious for children at one stage may not be at another. For example, it is not known whether efficacious treatments such as CBT for anxiety apply across all age groups or apply to children with concurrent disorders, meaning that practitioners must constantly extrapolate from the existing research evidence. Also, practitioners must always consider a child’s family, school, and community in the provision of services, but most RCTs do not address these dimensions.2 Compounding the situation, children’s mental health services typically entail a diverse array of both practitioners (such as psychologists, social workers, nurses, and psychiatrists) and sectors (such as health, social services, education, child protection, and justice). Fragmentation is endemic across these disciplines and sectors, making it difficult to even achieve agreement on the role of EBP.8 These controversies have prompted calls for new approaches to implementing EBP in children’s mental health that are rigorous but that also take developmental and practice contexts into account.2
IMPLEMENTING EBP IN BRITISH COLUMBIA
As researchers, we have been engaged in an ongoing partnership with practitioners and policy makers in British Columbia with the goal of improving children’s mental health. Our partnership illustrates many of the controversies involved in implementing EBP. Policy makers initially asked us to summarise the relevant research evidence on efficacious interventions for both prevention and treatment for a range of children’s mental disorders. Policy makers then used this research summary to inform a new Child and Youth Mental Health Plan for British Columbia.9 This plan incorporates prevention, treatment, and outcome monitoring, with an explicit focus on EBP. We now provide ongoing research support and education as the plan is implemented.
In our experience, many practitioners have been sceptical about EBP. The lack of effectiveness research has been a stumbling block, as has the lack of research incorporating developmental and practice contexts. Thus, practitioners questioned whether the research evidence was applicable to the situations they encountered on a daily basis. Also, when new techniques such as CBT were suggested based on the efficacy research, many practitioners responded that they were not trained in those techniques and that having policy makers sanction particular approaches could come to limit the service choices available in British Columbia’s public health system. There were also challenges related to the interdisciplinary nature of children’s mental health. We found that some disciplines embraced EBP’s quantitative perspective but others focused on its limitations, creating differences of opinion among practitioners. Further, intersectoral fragmentation meant that research minded policy makers leading the plan’s implementation had to market EBP to multiple diverse groups.
As researchers engaged in a partnership with practitioners and policy makers, we have worked with both groups to respond to the scepticism about EBP. For example, to address concerns about effectiveness, we have encouraged practitioners and policy makers to rigorously evaluate child outcomes at the local, regional, and provincial levels. By evaluating outcomes, we postulate that effectiveness information can be generated on new and existing services. To address the lack of training, active educational strategies have been used to provide education on CBT and other efficacious techniques. This training has made many practitioners more receptive to adding new techniques to their repertoire. We have also worked closely with policy makers to promote EBP with multiple diverse groups and to create an organisational environment that encourages EBP. However, most importantly for us as researchers, our partnership has educated us about the concerns of practitioners and policy makers which echo those expressed in the literature.10 EBP is too narrowly defined if it simply means the application of RCT evidence to practice. Therefore, we concur that new forms of research are urgently needed and we suggest that EBP must be redefined in a rigorous yet more inclusive way for children’s mental health.
RETHINKING EBP FOR CHILDREN’S MENTAL HEALTH
Controversies about EBP can be unhelpful when they polarise debates, pitting practitioners against researchers. Researchers and practitioners are not at odds but rather share a common goal––to improve children’s mental health. This makes research and practice interdependent, a case of “both/and” rather than “either/or”. Moreover, researchers and practitioners share a common epistemology. Thoughtful work in children’s mental health research and practice always comprises a synthesis of multiple forms of evidence: individual experience (of the one researcher or practitioner, or the one child or family), aggregate experience (of the many children and families), qualitative evidence (about values, meaning, and preferences) and quantitative evidence (about issues such as causation, prevention, and treatment). Each of these kinds of knowledge is required to make good decisions.
As a step towards recognising their interdependence with practitioners, researchers could acknowledge the limitations of EBP’s current hierarchy of evidence and could be more inclusive of complementary forms of knowledge. The existing research evidence in children’s mental health does not address the developmental and practice contexts or the interdisciplinary and intersectoral challenges that are faced when implementing EBP. These issues need to be included in new research on children’s mental health and on implementing EBP.2 Rigorous qualitative methodologies could be promoted to conduct research that incorporates practitioners’ experiences and that investigates interventions in developmental and practice contexts (see Gabbay & May, 2004).11 Quantitative research could also be better positioned as one kind of evidence, but not the only kind.7 Given the unmet children’s service needs, it is also imperative that practitioners be open to EBP. In this respect, practitioners could be more rigorous in systematically seeking and applying multiple kinds of evidence, including quantitative. Best available RCT evidence should be used as a guide where it is available.
For researchers and practitioners in children’s mental health, both the lack of appropriate research and the controversies about EBP are ultimately instructive. As well as sharing the common goal of improving children’s mental health, researchers and practitioners also share accountability to children, families, and communities for how well they are meeting this goal. There is always ambiguity when dealing with multiple perspectives and multiple kinds of evidence.8 However, vigilance can be maintained amid ambiguity by asking, as Drucker suggests, how do we know that we are “doing things right” and that we are “doing the right things”?12 Thoughtful researchers and practitioners ask these questions continuously. Applying RCT evidence in practice is one essential component of “doing things right” and “doing the right things”, but it is not the only component. We suggest rethinking EBP to better acknowledge the lack of research on developmental and practice contexts and to better reflect the multiple sources of evidence that are needed in children’s mental health. There are high levels of unmet service need in children’s mental health. This problem makes it imperative for researchers and practitioners to conduct new research and resolve the controversies together in order to meet their common goal.
We thank George McLauchlin and Cody Shepherd for their contributions to the ideas shared in this paper. We are also grateful to Child and Youth Mental Health Services with the British Columbia Ministry of Children and Family Development who funded this work. Our ongoing partnership with them has educated us about evidence-based practice in children’s mental health. Charlotte Waddell also holds a Scholar Award from the Michael Smith Foundation for Health Research.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.