Article Text

Download PDFPDF

1. Enabling evidence-based change in health care
  1. André Tomlin, MSc1,
  2. Karin L Dearness, Mlis2,
  3. Douglas S Badenoch, MSc2
  1. 1 andre.tomlin{at}
  2. 2Minervation Ltd
    Oxford, UK

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


Evidence-based health care (EBHC) has come a long way from its introduction in the early 1990s. The scepticism that greeted its early proponents (Sackett et al1) has been replaced with pragmatism; instead of asking whether we should be practising EBHC, we are now asking how we can practise it. This represents an important change in attitude, but also presents new challenges to the advocates of EBHC.

Nowadays, a plethora of guidelines and service frameworks set an environment for health care that is, at least in principle, founded on the best evidence from systematic research. But these “top-down” initiatives still need to be interpreted in the local and individual context.

The ever expanding array of secondary sources offers a richer than ever source of evidence with which to inform decisions, from quality and relevance filtered summaries (eg, the Evidence-Based journals and Clinical Evidence) to comprehensive systematic reviews (eg, those by the Cochrane Collaboration). Again, however, this evidence needs to be found, appraised, and applied to a particular situation and, equally importantly, must be accessible in a form that promotes ease of use.

In summary, there is no magic bullet for making healthcare organisations evidence-based. Simply providing online access to The Cochrane Library or Clinical Evidence, although undoubtedly improving awareness of and access to good quality evidence, will not solve this problem.


To help plot a way forward, it is helpful to describe the objective clearly; here we present a definition of what an EB organisation might look like in how it manages information:

  • Routinely deploys evidence from the best available systematic research in making decisions about health care

  • Integrates EB information into the information resources

  • Embeds the skills of EBHC into their organisation

  • Establishes reliable, secure, and quality-assured processes for communicating knowledge and information

  • Ensures that information systems are relevant and timely to their users′ needs

  • Practises and promotes high standards of usability in information systems

Our definition of knowledge systems has crystallised around these 6 prerogatives. We recognise that substantial barriers remain, at both individual and organisational levels, to their effective implementation, principally:

  • Time: busy clinicians and patients do not have enough time to seek evidence to answer all the questions they have about all healthcare interventions

  • Lack of integration: evidence delivery systems are not integrated into clinical information systems

  • Access: sources of knowledge are not available to the people who need them when they need them

  • Usability: poorly designed systems simply present more barriers to the evidence rather than removing them

  • Relevance: what knowledge is available is often irrelevant to the problem at hand

  • Awareness: new publications and resources are publicised unequally

  • Skills: many professionals and most patients are not skilled in finding and appraising evidence

  • Beliefs: often, we separate practice from learning and regard EB resources in the latter category of knowledge: something to be indulged when we have the luxury of time off from the real world

  • Communication: the right people must share expertise and collaborate on similar goals. This means identifying where the expertise resides in the organisation and establishing channels of communication, collaborative goal setting, and progress management

Clearly, there is no simple solution to this question, but consideration of the barriers has given rise to a new generation of information systems that will change our conception of healthcare information in years to come.


In response to these barriers, we can identify a trend in EB information systems towards integrating the knowledge they provide into the problem solving process of the intended user. This can mean providing links from clinical guidelines to EB summaries; equally it can mean creating information systems that specifically answer the questions posed by patients and professionals (such as in NHS Direct Online2). Some examples of these types of systems are beginning to emerge (Evidence-Based On Call,3 Prestige guidelines standards,4 and the pilot site for the National Electronic Library for Health.5).

In this sense, we can refer to them as “knowledge systems”, rather than information systems, since they are customised to fit into their users′ knowledge gaps. In contrast, “information systems” provide access to resources but rely on the user to learn how they are structured and piece together the bits they need to answer their question.

Effective knowledge systems have 6 distinguishing features:

  • Excellent usability

  • Relevance to the specific questions of their user population

  • Flexibility to allow ad hoc queries

  • Content that is based on the best available evidence

  • Expert authors with the clinical expertise needed to put the evidence in context

  • Integration with existing systems, without substantial costs

By integrating usability and relevance into the rubric of EBHC, such knowledge systems offer the potential to revolutionise the provision of healthcare information.

Evidence-based systems development

An essential element in the overall drive towards EB organisational change is the process of EB systems development. Although audiences and system requirements differ enormously from organisation to organisation, it is possible to present a methodological process that can be adapted to the majority of situations.


The development of any system built to meet the knowledge needs of an audience must begin by addressing the information needs of that audience. It is often necessary to divide this process into a number of strands, recognising the range of potential users of the final system (eg, patients, caregivers, clinicians, and social workers).

A great deal of research is published that explores the information seeking behaviour of these groups within the healthcare sector.6–,8 However, we must not simply concern ourselves with the ways in which individuals traditionally find answers to their questions. It is also necessary to focus on the ways in which interfaces can be built to maximise usability9 and “stickiness” (the ability to retain end users during their visit to your website). This process will enable the system development team to ascertain a number of the key information requirements of the audience. These requirements will directly impact upon the way in which information is presented in the final system. An example of this is the list of audience requirements set out for the Evidence-Based Websites10–,13 developed for the UK NHS Direct Online project:

  • Fast access to relevant material (3 click goal)

  • Summarised and synthesised treatment information

  • Information presented to be understandable by individuals with a 14 year old reading age

  • Comprehensive subject coverage within each topic

  • Unbiased and reliable information produced by an independent group


The next stage in this methodology is the first step of the classic EBM model.1 If a system is to be comprehensive it must ensure that it addresses all of the questions that may be asked of it, irrespective of whether there is high quality research evidence to answer the question. Users will no doubt ask questions that can be answered by referring to The Cochrane Library14 or the latest issue of Evidence-Based Mental Health (EBMH). However, they will also ask about obscure therapies and rare disorders for which there is little high quality evidence.

The question formulation stage of the methodology ensures that it includes all of the relevant stakeholders, forging links with consumer health groups and charitable organisations, alongside professional societies and academic institutions. It should be primarily concerned with the types of questions that the end user is likely to ask of the final system, but should also consider the variety of ways in which individuals ask questions of online systems because this will have a direct impact on the way the systems media is created.15

Question formulation is an essential stage in our methodology, and as such should be given the time and resources it deserves. The outcomes of this process will be invaluable in creating the knowledge system, but should not be allowed to dictate the overall structure of the underlying system. This should instead be based on a controlled vocabulary and classification scheme, such as the internationally recognised Medical Subject Headings (MeSH) produced by the National Library of Medicine (USA).


Answering questions with the best available evidence is only possible when a systematic and reproducible literature search is carried out for each question. This process is simplified when the question has been formulated in a clear and well structured fashion.16

Considerable preparation is required before the search begins, identifying the type of question that has been asked (eg, therapy, diagnosis, and prognosis) and the study design that will best provide an applicable and reliable bottom line answer.17

Search results for some questions will identify a large amount of high quality evidence, whereas others will demand more indepth retrieval techniques such as searching less common databases electronically (or manually if necessary) to retrieve perhaps just one or two relevant papers (table 1). Tools can assist in developing a search strategy for locating high quality material, such as the “clinical queries” filters developed by Haynes et al which utilise the results of evidence-based question formulation.18

Table 1.

Literature searching for different types of questions


Assessing the validity and applicability of the literature identified by the search can be divided into 2 stages.

The first stage can be described as an evidence screen, which can be carried out by the same information specialist who performs the literature search. This process aims to streamline the next stage of the methodology (authoring) by providing the author with only the necessary evidence to produce the content in question. For example, if the “antidepressants for depression” search (table 1) has retrieved 11 EBMH abstracts, 6 Cochrane reviews, 1 chapter in Clinical Evidence, and a dozen randomised controlled trials, it would not benefit the author if (s)he was asked to read, review, and appraise all of this content. The evidence screen therefore identifies the key pieces of evidence that will enable the author to produce a comprehensive and unbiased piece of content to answer the question in hand.

The second stage is the critical appraisal performed by the author, utilising standard appraisal techniques,19 to reduce bias and put the research in the context of the content being produced.


Research has shown that knowledge system developers should “consider the full spectrum of the audiences who may access their site”. Furthermore, authors should be aware that “language, content intensity and presentation should have a different style and be identified by its intended audience to avoid a range of problems including misinterpretation, mistargetting of content, misrepresentation of source and quality”.20

Content authors must therefore be aware of the knowledge requirements of the audience, be clinically proficient in the field, and be able to apply EB methods to the authoring process.

A system that attempts to meet the knowledge requirements of a broad user base must reflect this breadth of experience and understanding in the team that is brought together to produce content. For example, a project aimed at developing a web-based resource for schizophrenia for UK patients, caregivers, and mental health professionals involved 3 levels of authors and reviewers in the production process (table 2).

Table 2.

Authors and reviewers involved in the production of a web-based schizophrenia resource for UK patients and professionals


The decision making process concerning media creation for an EB knowledge system is far too complex to summarise in this EBMH notebook. Issues to consider are many and varied, including:

  • Audience – access to technology

  • Audience – media preferences

  • Interoperability with other systems

  • Indexing system

  • Search functionality requirements

  • Personalisation


It is clear from sections a to f that the methodology suggested for the efficient production of an EB system is both complex and extremely challenging. Organisations that attempt to follow this methodology will require excellent project management support to drive and coordinate the production process because the project will inevitably involve a broad range of professional groups, often dispersed across a wide geographical area. In addition, many organisations will benefit from collaboration with EB centres who have proven expertise in this area, because this will enable them to draw upon the international network of experts in the field.

Knowledge management activities

Developing information systems is only one part of the knowledge systems approach, and must be seen within the context of a number of related activities. Indeed, the success of the whole undertaking will depend on an alignment of these activities.


Finding out what your users want is (or should be) central to any organisation′s remit. The process outlined in steps a and b above can provide a rich insight into broader questions of purpose and remit as well as specific aspects of relevance and usage. These include:

  • Expectations and experiences

  • What questions are being left unanswered

  • Opportunities for linking services

  • How existing services can be improved

  • Auditing usage patterns and rates

Web-based sources can provide valuable information, which can feed into (and from) existing research.


The requisite core knowledge and skills for successfully producing EB information systems are:

  • End user research

  • Search strategies

  • Critical appraisal

  • Clinical authoring

  • Interface design

  • Web design and coding

  • Editorial management

This blend of expertise is quite rare and expensive; often, it makes sense for organisations to outsource the production of such content, in whole or part. However, it is also likely that many of these skills already exist within an organisation of any substantial size; it may be worth considering whether an in house training programme would enable the organisation to produce and maintain its own systems independently from any outside group.

Assuming that specific clinical expertise (item 4 above) is covered in traditional educational activities; such a programme would focus on four main areas:

  • The basics of EB practice (1–3 above, to include those responsible for 4)

  • Systems development (1, 5–7)

  • Project management (7 & 8)

  • Training trainers (to ensure ongoing support and championing of the project)

The key to success will be to assemble the right team within the organisation and ensure that they are able to collaborate effectively towards the shared goals. A well designed training programme can foster this environment at the same time as addressing these specific learning needs.


As well as designing and creating knowledge systems, we also need to know whether they work. Broadly, we need to know whether systems

  • Provide users with the information they need

  • Reflect the current state of “best evidence-based practice”

  • Provide usable, accessible interfaces to the knowledge

It is likely that a wide variety of techniques will be utilised, depending on the kind of evaluation required. If the full impact of a system on the organisation′s performance is required, clear objective performance criteria must be set out prior to implementation of the system. At the other end of the scale, it is possible to address smaller scale studies at specific aspects of the system, such as the usability of its front end or its relevance to a particular user group. The scoping of evaluation is crucial, because this will determine the complexity of the exercise. In any case, the end users should be directly involved in establishing and evaluating success criteria.


We have presented an outline of how healthcare organisations can use knowledge systems as leverage towards promoting evidence-based change. The organic relationship between the organisation's human constituencies (both its workers and its users) and its information systems is central to the success of these systems. Education becomes stagnant without the support of the right systems; systems are redundant if they are not grounded in the knowledge needs of their users.

However, by combining practical relevance and technical usability with personal development and training, it is possible to push organisational culture towards more effective use of research evidence and improved healthcare provision. We are only beginning to understand how the “hard” and “soft” areas of information systems and organisational change interact with one another in this field, but many groups are now developing tools that will enable us to move to the next stage in the development of EB organisations.