There are more than 10,000 mental or behavioral health apps publicly available; however, there is little available information on the quality and effectiveness of the apps, “leaving consumers with very little to go on when trying to determine which apps might be helpful and worthwhile.” (1) Several recent studies highlight the need for better information on the effectiveness and quality of mental health apps.

One study, published in the journal Nature Digital Medicine, looks at claims made by top-ranked consumer-focused mental health apps regarding their acceptability and effectiveness and the support they provide for their claims. Most of the 73 mental health apps reviewed focus on improving symptoms, mood or self-management or diagnosing a mental health condition.

The study identified five basic functions among the apps (many with multiple functions):

• Information or psychoeducation 36%
• Therapy or treatment 32%
• Symptom or mood monitoring 25%
• Self-assessment 12%
• Peer-support or community support 11%

About a third (64%) of the apps provided claims about their effectiveness, and nearly half (45%) provided claims about their acceptability. Yet there was little direct evidence supporting the claims, and many techniques they mentioned are not supported by research. Only two of the 73 apps described direct evidence associated with the effectiveness of the app.

Another study, published in Psychiatric Services, looked at how people actual use the apps. Despite claims of highly rated user engagement, their sustained use is low, according to study authors led by Michelle M. Ng, with Harvard Medical School. The researchers reviewed 40 studies of user engagement – the ability of an app to engage and sustain people in using the app – including measures of usability, user satisfaction, acceptability and feasibility.

All 40 studies claimed high t user engagement for the app. However, they used a wide variety of methodologies (including subjective ratings and objective data) to make the assessments. The inconsistencies in the evaluations processes, the researchers conclude, “cast doubt on the studies’ ability to claim that their app was usable, satisfactory, acceptable or feasible.” The authors noted that 15 of the 40 studies used no objective data in evaluating the app.

The authors note that because the experience of mental illness is personal, no single scale or measurement will perfectly capture engagement for all people. However, they argue, the lack of clear definitions and standards leaves open the possibility of inappropriate measures, bias or misinterpretation and is harmful to patients who may not know which apps to trust.

A third study compared how popular mental health apps are rated and ranked by three available online app evaluation systems with expert reviews (ORCHA, Psyberguide and MindTools.io). Researchers Andrew D. Carlo, M.D., and colleagues at the University of Washington in Seattle, identified the 25 most popular behavioral health apps based on Apple App Store and Google Play Store downloads and number of daily and monthly active users.

Overall, apps offering brain training, meditation, mindfulness and relaxation skills were most common. Others offered peer support, mood or anxiety self-monitoring and therapy.

The researchers looked at ratings from each of the three online evaluation systems for each of the apps in three areas: user experience, credibility and evidence, and data use and security. They found wide variation in the ratings among the three evaluation models and found the expert ratings to be “broadly inconsistent and often contradictory, even with the most popular apps.” None of the apps received top scores across all categories in all frameworks. They concluded that there is a growing need for standardized quality and effectiveness measures and up-to-date app guidance for consumers and clinicians.

Collectively these studies show that for now, consumers looking for mental health apps are faced with many choices and many positive claims, but little in the way of independent or objective evaluation.

The American Psychiatric Association has developed an app evaluation tool that can help. While it does not provide assessment of specific apps, it does provide guidance to help people make better and more informed choices when considering mental health apps. Learn more.