Article Text

Download PDFPDF

Implementing tools to support evidence-based practice: a survey and brief intervention study of the National Elf Service across Oxford Health NHS Foundation Trust
  1. Lauren Z Atkinson1,
  2. Alexandra Forrest1,
  3. Leah Marriner1,
  4. John Geddes1,2,
  5. Andrea Cipriani1,2
  1. 1Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK;
  2. 2Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
  1. Correspondence to Professor Andrea Cipriani;{at}


Background Technology and the internet has enabled rapid access to research but most mental health professionals do not have time to keep up with the vast and growing scientific literature. Secondary information sources, such as the National Elf Service (NES), aim to summarise the most important and up-to-date research to improve mental health professionals' access to information to support evidence-based medicine (EBM).

Objective To explore mental health professionals' attitudes towards evidence-based practice and methods used to keep up-to-date with research. To promote use of a digital evidence-based platform (the National Elf Service), assess its use and explore its potential to impact clinical practice.

Methods Baseline and follow-up surveys were distributed among staff of 5 adult mental health community teams and 2 early intervention services (n=331) in Oxford Health Foundation Trust (OHFT) prior to and following an intervention raising awareness of the National Elf Service.

Findings Of 133 baseline survey responders, the majority of staff reported their clinical practice was informed by evidence, mostly using existing clinical guidelines and online resources. Few had used the National Elf Service. 122 staff members completed the follow-up survey. Postintervention, 42 staff members indicated they had used the National Elf Service (compared with 13 preintervention) and that it had improved access to research. Lack of time was most often the barrier restricting evidence-based practice.

Conclusions Mental health professionals are engaged with EBM and those that used the National Elf Service felt it did, or could have the potential to impact on their clinical practice.

Clinical implications Barriers and challenges to implement EBM more widely suggest targeted efforts should be made to embed evidence-based practice into the working culture.


Statistics from


Making informed decisions is a key component of everyday clinical practice. In order to do this, mental health professionals should combine their own clinical expertise and training with high-quality scientific evidence. This practice has been defined as evidence-based medicine (EBM).1 Mental health professionals are encouraged to practice EBM,2 therefore requiring them to update their knowledge continuously. While technology and the internet has enabled rapid access to the results of valuable research via online journals and databases to inform EBM,3 scientific literature is growing at an astonishing rate.4 Most mental health professionals simply do not have sufficient time to read important primary research reports. A number of ‘secondary journals' exist that aim to overcome this issue by summarising the most important and up-to-date research, potential clinical implications of research and methodological advances.5 Previous evidence suggests that evidence-based summaries are more likely to improve access to necessary information required by clinicians, can improve clinical knowledge and reduces errors in decision-making.6 ,7

The National Elf Service (NES) is an online platform that provides accessible, usable and reliable summaries of new research, policy and guidance using a modern approach; via blogs, Twitter and other social media ( The NES website was launched in April 2015 by Minervation, an evidence-based healthcare consultancy company based in Oxford. The website covers a number of areas of interest defined by various ‘Elves'. The Learning Disabilities Elf was launched first in early 2011 and was quickly followed by the Mental Elf, the Dental Elf and the Diabetes Elf later that year. There are now 11 different Elves (covering child health, commissioning, education, lifestyle, musculoskeletal, social care and stroke). The Mental Elf is the most established of the elves; it has the biggest user following on Twitter ( and the largest group of people contributing to blogs for the site ( The website currently has over 4000 published blogs. Each blog highlights a new piece of recent evidence relevant to mental health and social care practice; all are open access and free for anyone to read. Mental health blogs usually highlight systematic reviews, meta-analyses, randomised controlled trials and high-quality cohort studies, though qualitative research and other study designs are also included. The blogs all use a consistent structure, with a brief introduction to the topic, followed by a summary of the methods, results and conclusions of the new research. All blogs are written by individuals with a wide range of expertise and interests, who critically appraise the research as well as providing a summary and clinical commentary of the paper, explaining how they believe practice should change as a result, and what future research should be commissioned. There are over 400 individuals who contribute blogs to the NES with half of these people writing for the Mental Elf.

The overall objective of the NES is to make research more accessible to health and social care professionals, to help break down the barriers to engaging with research that exist for many frontline practitioners and to implement evidence in practice.2 In addition to a standard service offered by the NES, it is possible to buy a subscription. Subscribers have access to a range of additional membership features including: automatic tracking of learning, downloadable Continuing Professional Development certificates, gamification features, personalised email alerts, personal notes to reflect on learning, as well as various ways to connect with colleagues and experts in their field of interest. Oxford Health National Health Service (NHS) Foundation Trust (OHFT) had an institutional subscription to the NES for 1 year from April 2015 and the resource was made available to clinicians Trust wide. For clinicians and mental health professionals to have access to this resource was seen as an important step for the drive towards EBM locally. However, initially, use of the NES was not actively promoted to staff members.


This pilot study aimed to (1) explore mental health professionals' attitudes towards evidence-based practice and identify methods they use to keep up-to-date with research; (2) promote the use of the NES across selected teams of OHFT; (3) determine whether mental health professionals used the NES, its features and the mental health blogs over time, and if so, how and (4) explore whether the NES had the potential to impact on clinical practice. It was predicted that the majority of staff will not have heard of, or used the NES. Carrying out this pilot study would provide evidence to support whether the institutional subscription to the NES was a reasonable and sustainable investment for OHFT (particularly during times of austerity). Additionally, it would inform whether a randomised controlled trial should be implemented to assess the impact of the use of the Mental Elf on evidence-based practice and its potential to affect clinical outcomes.

For the purpose of this study, focus was given to the mental health content within the NES (a topic covered primarily, but not exclusively, by the Mental Elf). Moreover, mental health is a topic that is relevant to the vast majority of staff working in OHFT, so it is likely that the mental health blogs will highlight new research that a significant proportion of staff can apply to their own practice.



Five adult mental health community teams (AMHTs) and two early intervention services (EIS) that exist across Oxford and Buckinghamshire were approached to take part. These teams and services were selected for two key reasons. First, staff are part of multidisciplinary teams consisting of psychiatrists, psychologists, community psychiatric nurses, approved social workers, occupational therapists and support workers. The teams make up a total of 331 professionals providing mental healthcare and are a good representation of OHFT employees. Second, support is embedded within these teams to facilitate the implementation of research, and therefore this could be used for the current study. In October 2014, OHFT introduced a dedicated research recruitment team; seven research assistants (RAs) were embedded within each AMHT and EIS service to work daily alongside those delivering clinical care. The RAs roles are focused on increasing staff awareness and engagement in research, thus promotion of the NES was seen as part of this role. Furthermore, in terms of feasibility, the RAs could support practical elements of the intervention as well as data collection. All AMHTs and EIS staff were invited to take part in the study (331 staff in total); participation was voluntary.


Two surveys, a baseline and follow-up survey, were developed for the purpose of this study (see box 1). Surveys were developed through informal discussions between the seven RAs embedded within each AMHT and EIS service. The baseline survey was designed with a view to capture information that would evaluate individuals' current practices of EBM, whether individuals already knew about or used the NES to support their own evidence-based clinical practice and how the use of technology assists in their access to research. The follow-up survey was designed to evaluate whether the NES was a resource that could support and influence the use of EBM in OHFT, after it had been formally introduced to each team. Questions either followed a ‘yes’ or ‘no’ response format, were rated on a five-point Likert scale (very easy, somewhat easy, undecided, not very easy or not easy at all), or were free-text, open-ended questions.

Box 1

List of questions from the baseline survey

  1. What is your age? (<25, 25–35, 36–45, 46–55, >55)

  2. What is your gender?

  3. What is your job role?

  4. How do you keep up-to-date with research?

  5. When did you last read a research paper or an article? What was it about?

  6. Is your clinical practice currently informed by evidence?

    • If yes, how?

    • If no, what do you feel the barriers or challenges are?

  7. Have you ever used the National Elf Service to answer a clinical question?

    • If yes, in what way?

    • If no, why?

  8. Have electronic devices and social media improved your access to/use of research?

    • If yes, how?

    • If no, why not?

List of questions from the follow-up survey

  1. Did you attend a presentation about the National Elf Service in November/December 2015?

  2. Did you receive an email including information about the National Elf Service and details of how to register to the website?

  3. Have you used the Mental Elf Service?

    • If no, why?

  4. Has the Mental Elf Service influenced your practice?

    • If yes, how?

    • If no, why?

  5. Please share any clinical questions relevant to your work that have been answered by research covered by the National Elf Service?

  6. Has the National Elf Service prompted you to explore other sources of evidence?

    • If yes, please list them.

    • If no, why?

  7. How easy did you find the National Elf Service to use?

    • Very easy,

    • Somewhat easy,

    • Undecided,

    • Not very easy,

    • Not at all easy.

  8. Would you like to continue to access the National Elf Service through the Trust subscription?

    • If yes, why?

    • If no, why not?


As the OHFT NES subscription had not previously been promoted, an intervention was delivered within the chosen sample in order to increase awareness of its existence. The intervention consisted of a presentation to each AMHT and EIS team which lasted ∼20 min. Each presentation followed the same format and introduced the NES, explained how to use the service and illustrated its potential benefits. To ensure consistency, presentations were delivered by the same person for each team, this being the director of the NES, André Tomlin.


Prior to the intervention, mental health professionals from the AMHTs and EIS were asked to complete the baseline survey. The baseline survey was distributed in team business meetings, which all members of staff are required to attend and therefore it was hoped a large number of staff would be reached.

The NES intervention was systematically rolled out across the selected teams. Presentations were delivered in team business meetings during the months of November and December 2015, and the intervention was delivered to all teams within 1 month (between 25 November and 23 December 2015; see table 2). Immediately following the delivery of the intervention to each team, an email including a copy of the presentation slides, details of how to register to the NES and a point of contact for further support was circulated to all members of each team, regardless of whether they had attended the presentation or not in order to ensure everybody received the information. Staff who were not present in the team business meetings were sent the baseline survey at this stage and invited to complete it. Once the intervention had been delivered to all seven teams, an additional email was circulated encouraging all AMHTs and EIS staff to use the NES to keep up-to-date with the latest mental health research. A number of articles were also highlighted that came from the NES and were of potential interest. The RAs were well informed of the service and available to support staff with any queries or ongoing difficulties as these arose. Eight weeks after the intervention, the follow-up survey was emailed to the same 331 OHFT staff members in order to collect information about individuals' usage of the NES, accessibility of the service and the potential influence of the NES on clinical practice.


Data analysis was conducted in Microsoft Office Excel 2010. Frequencies and percentages of ‘yes’ and ‘no’ survey answers were calculated. Responses to free-text survey questions were coded into categories to allow for analysis. Categories were generated through discussions between the RAs and based on answers that emerged from multiple respondents. Frequencies and percentages of category answers were calculated for all free-text responses. Percentages expressed in the text are proportional to the number of responders to each survey question. Tests for differences in the uptake of the NES preintervention and postintervention were not undertaken as survey responses were anonymous and therefore baseline and follow-up surveys could not be linked. It was not known whether those that did or did not use the NES postintervention, had already used the NES at baseline.


Of the 331 staff in OHFT, 133 completed the initial baseline survey (40.2%). Baseline survey responders were made up of all staff members who attended the NES intervention presentation (n=102) and only 14% of the people who had received the NES information via email (n=31). Table 1 describes the demographics of all baseline survey respondents. It was revealed that 110 out of 123 question responders (90%) felt their current clinical practice was informed by evidence. When asked how their clinical practice was informed by evidence, 41 out of 110 responders (37%) reported using already established evidence-based guidelines such as National Institute for Health and Care Excellence (NICE) and Maudsley Prescribing guidelines and 29 (26%) used evidence-based interventions and assessment tools in their practice. More than a third of respondents reported finding research evidence through primary research articles, systematic reviews or meta-analyses to incorporate into their practise. Few individuals used evidence gained at conferences or training, and only 8% reported that evidence shared through colleagues or supervisors informed their own clinical practise.

Table 1

Demographic characteristics of baseline sample

Thirteen members of staff (11%) did not believe their clinical practice was informed by evidence, with ‘lack of time’ being the barrier reported most often, closely followed by workload pressures such as “size of caseload, pressure to document and expectations on us to manage individuals in crisis.” Some staff also reported lacking the skills to find EBM: “Finding time to keep up to date is very difficult. Not using databases for a long time means skills are lost.” When the staff were asked if they previously used the NES to answer a clinical question only 5 of 127 responders to the question (4%) answered yes (examples of clinical questions included treatment options in post-traumatic stress disorder and the effectiveness of cognitive–behavioural therapy (CBT)). When the remaining responders were asked why they had not used the NES, almost 80% (69 of 87 question respondents) stated that they had never heard of it. Eight staff had used it but had not applied it to clinical practice (2%), and of these only 2 (2%) did not find it useful but failed to provide reasons why.

Despite not always having the chance to integrate evidence into practice, 121 staff members of the 133 survey responders (91%) reported that they kept up-to-date with current research. The most popular ways of keeping up-to-date were via journal articles and electronic database searches (57%), meetings and training (49%) and general online browsing (40%). The time since having read a research paper or article varied across all staff members, but reassuringly, the majority of staff had very recently kept up-to-date with research (figure 1). One hundred and twelve out of 127 responders (84%) expressed that electronic devices and social media had improved their access to, and therefore use of research, making it easier, more convenient and almost instantaneous. Others admitted that they had poor IT skills to access this research or they were “not using technology to [its] full advantage.”

Figure 1

Bar graph of responses to question 5 of the baseline survey.

The NES intervention was successfully delivered face-to-face to 102 staff members (out of a possible 331 staff members; 31%). Table 2 outlines attendance rates for each of the OHFT AMHT and EIS teams. After 8 weeks of NES use, 122 staff members completed the follow-up survey, representing 37% of staff in OHFT.

Table 2

Attendance rates at each AMHT and EIS service in OHFT out of total number of possible staff

Postintervention, 42 staff members (34.4%) indicated that they had used the NES, more than three times as many compared with baseline. Of these individuals, 23 (55%) had attended the intervention presentation and 19 (45%) had only received information about signing up to the NES via email. Proportionally, uptake to the NES from face-to-face intervention was higher than uptake from emails (23% vs 8%). Twenty-nine staff members of 42 using it (62%) indicated that they felt the NES service had influenced their practice. Staff members' comments about this included that they had “access to updated resources for clinical practice,” the NES had “backed-up previous knowledge,” and provided “easier access to info.” Additionally, others felt it had “given [me] bitesize updates,” and kept them “up to date with studies and current findings.” Six respondents were able to identify specific clinical questions, which the NES had helped them to answer. These included evidence for effectiveness of mindfulness in depression, lack of strong evidence for CBT in bipolar disorder and antipsychotic drug choice.

Eighty (66%) respondents to the survey had not used the NES since the intervention. Of them, 26 (33%) had attended the intervention presentation. When asked reasons for not using the NES, only 60 staff members responded (49%). Reasons for not using the NES varied, but included being unaware of the system (in some cases being new starters; 47%), time restraints (42%), preferences to acquire knowledge via other methods (3%), not having registered to the system (3%), needing reminders to use the NES (3%) and poor motivation (2%). When asked if the NES had prompted exploration of other sources to obtain evidence, of 46 responders, 19 staff members (41%) stated it had, but 27 stated that it had not (59%). Among the 27 staff who felt that the NES had not prompted them to explore other sources of evidence, half of these respondents again highlighted a lack of time as the reason for this (48%). It was found that the NES was an accessible tool for most clinicians, with more than 80% of responders rating the system either ‘very easy’ or ‘somewhat easy’ to use.

Finally, staff members were asked if they would like to be able to continue using the service. Of the 67 responders, 62 (93%) indicated that they would. The majority of responders reported that the NES was a useful resource to keep up-to-date with research, providing easier access to the literature and that this was helpful for clinical practice. Comments included that it was a “much more diverse platform than Medline or PSYCHINFO, better for all mental health professionals and not just doctors” and others reported that “from talking to colleagues this [the NES] has been a benefit.” In comparison, five staff members stated they would not use the NES again because of time restraints and preferences to use other sources.


To the best of our knowledge, this is the first formal roll out of an evidence-based digital platform across a clinical service, and the first evaluation of its use within an NHS trust. Despite OHFT purchasing a subscription to the NES, the service was not robustly promoted prior to this intervention. All the staff who attended the NES intervention completed the baseline survey; however, it is perhaps disappointing that very few staff (14%) responded to the baseline survey sent out by email. An overall response rate of 40% for this survey is reasonable and comparable to response rates from surveys previously undertaken within this sample8 and email surveys more generally.9 Responders represented a variety of job roles within OHFT and therefore results are thought to be generalisable across the service.

As expected, the baseline survey confirmed that only a minority of staff were aware of the NES and actively used it to inform their clinical practice before the delivery of the intervention. The OHFT subscription to the NES was therefore hugely underused at this time raising questions about its value for money. Most importantly, although use of the NES was low, the large majority of staff felt their current clinical practice was informed by evidence, demonstrating that staff value the importance of evidence-based practice and see this as an essential part of their role. Mental health professionals were aware of where to find EBM literature, primarily accessing research online via databases and websites. Given that the NES is a web-based platform which uses social media interactions, it is promising that this is in line with current practices.10

Postintervention, there was an increase in NES users, with one-third of survey responders using the service. Although a reasonable uptake, this equates to just 13% of total OHFT staff. Of those that used the NES, 69% had used the service to improve knowledge and access to resources. In some cases, this knowledge had influenced practice by providing answers to specific clinical questions, which were varied in their scope. These findings support the use of the NES in aiding EBM across areas of psychiatry.

Attendance rate at presentations was low and it is important to consider reasons for this in order to improve roll out of similar services in the future, by OHFT or other NHS trusts. The NES intervention was delivered to ∼30% of staff. The OHFT provides 24-hour care and therefore not all staff were scheduled to work during the times or weekdays that the intervention was delivered, the priority of clinical duties may also have prevented some staff from attending. In addition, for some of the teams, it was only possible to secure a time for the intervention to be delivered at one of their two sites, which limited the number of attendees. A more intense intervention strategy would likely increase the number of staff reached, with the intervention offered over multiple dates and times for a more extended period than 1 month. In this case, it was unfortunate that time was limited by the few months remaining on the OHFT NES subscription. It is acknowledged that low attendance rates limit the impact of the intervention to increase awareness and usage of the NES, and thereby encouraging evidence-based practice. Despite low attendance, the use of face-to-face intervention was more effective than emails at increasing uptake to the NES (23% vs 8%).

There were a number of barriers and challenges reported as to why staff members had not used the NES, primarily focused around being too busy and a lack of time. Large caseloads, with priority for crisis management, and documentation pressures were also reported to effect staff members' ability to deliver EBM. Time restraints have long been identified as a barrier to the prioritisation and implementation of EBM.11 It seems that encouraging use of the NES, or other accessible, evidence-synthesising sources, would benefit from more continuous promotion within the NHS and also the consideration of protected time for clinicians to keep up-to-date with relevant research via such resources. In future, it would be interesting to explore if and how team managers view whether EBM is practiced as part of the working culture. We know that EBM is currently the best method available to treat patients and improve outcomes.2 Encouraging it can only be a good thing if clinicians are equipped with the resources and skills to apply it.12

This study has some limitations. First of all, reminder emails were not sent out at baseline to encourage individuals to sign up and use the service. A large majority of OHFT staff members were still unaware of the NES resource at follow-up, suggesting that emails to those staff who had not attended the intervention were not wholly effective in advertising the service. This is also likely to account for low survey responses at baseline. Reminders would have been an easy and possibly effective way to promote use. Other future possibilities to improve uptake of the service include advertisements within team bases and having team managers on board with promoting the resource to staff. Despite uptake being low, over half of survey respondents indicated that they would still like the NES resource to be available to them via a Trust subscription. Interestingly, this included individuals who have not used the NES, suggesting some staff just did not have time to use the service during the evaluation period. In the future, prolonging the evaluation period may also allow more staff to apply evidence-based knowledge they have acquired, in practice. Addressing all of the aforementioned limitations would make the OHFT NES subscription more sustainable. Finally, this study focused on only one NHS Trust, and only AMHT and EIS teams. Though the samples were representative of Trust employees as a whole, OHFT is a relatively innovative and research active trust.13 Issues highlighted here may be more prominent in less research active trusts.

Clinical implications

The current study demonstrates that OHFT staff are engaged with EBM and those that used the NES felt it did, or could have the potential to impact on their clinical practice. However, barriers and challenges to engagement suggest more targeted efforts should be made to embed evidence-based practice into the working culture. Learning from the limitations of the current study, we aim to undertake a larger scale study to measure and assess how evidence-based research, provided by tools such as the NES, influence practice and evaluate its impact on clinical outcomes.


The authors acknowledge the help of the following research assistants who helped to design the study questionnaires and deliver the intervention across the AMHTs and EIS: Thomas Misselbrook, Hayley Sapsford, Rachael Davies, Maninder Kaur, Joanna Ciapala and Bradley Mullins.



  • Funding AC is supported by the NIHR Oxford Cognitive Health Clinical Research Facility. JG is an NIHR Senior Investigator.

  • Disclaimer The views expressed here are those of the authors and not necessarily those of the funders, the National Health Service, the NIHR or the Department of Health.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.