Article Text


Proportionate methods for evaluating a simple digital mental health tool
  1. E Bethan Davies1,2,3,
  2. Michael P Craven1,3,4,
  3. Jennifer L Martin1,2,3,
  4. Lucy Simons1,2,3
  1. 1 NIHR MindTech Healthcare Technology Co-operative, Institute of Mental Health, University of Nottingham, Nottingham, UK
  2. 2 Division of Psychiatry and Applied Psychology, Institute of Mental Health, University of Nottingham, Nottingham, UK
  3. 3 NIHR Nottingham Biomedical Research Centre, University of Nottingham, Nottingham, UK
  4. 4 Bioengineering Research Group, Faculty of Engineering, University of Nottingham, Nottingham, UK
  1. Correspondence to Dr E Bethan Davies, NIHR MindTech Healthcare Technology Co-operative, Institute of Mental Health, University of Nottingham, Triumph Road, Nottingham NG7 2TU, UK; bethan.davies{at}


Background Traditional evaluation methods are not keeping pace with rapid developments in mobile health. More flexible methodologies are needed to evaluate mHealth technologies, particularly simple, self-help tools. One approach is to combine a variety of methods and data to build a comprehensive picture of how a technology is used and its impact on users.

Objective This paper aims to demonstrate how analytical data and user feedback can be triangulated to provide a proportionate and practical approach to the evaluation of a mental well-being smartphone app (In Hand).

Methods A three-part process was used to collect data: (1) app analytics; (2) an online user survey and (3) interviews with users.

Findings Analytics showed that >50% of user sessions counted as ‘meaningful engagement’. User survey findings (n=108) revealed that In Hand was perceived to be helpful on several dimensions of mental well-being. Interviews (n=8) provided insight into how these self-reported positive effects were understood by users.

Conclusions This evaluation demonstrates how different methods can be combined to complete a real world, naturalistic evaluation of a self-help digital tool and provide insights into how and why an app is used and its impact on users’ well-being.

Clinical implications This triangulation approach to evaluation provides insight into how well-being apps are used and their perceived impact on users’ mental well-being. This approach is useful for mental healthcare professionals and commissioners who wish to recommend simple digital tools to their patients and evaluate their uptake, use and benefits.

  • mental health

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

View Full Text

Statistics from


  • Contributors MPC, JLM and LS conceived the study, collected and analysed the data and interpreted the results. All authors drafted and revised the manuscript and gave final approval of the version submitted for publication.

  • Funding This study was funded by Comic Relief as part of Innovation Labs (a consortium of Comic Relief, Mental Health Foundation, Paul Hamlyn Trust and Nominet Trust).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.