Our approach to rating mental health apps is grounded in the belief that any decision between you and a patient is a personal decision based on many factors, for which there is rarely a binary 'yes' or 'no' answer. For example, cognitive behavioral therapy is often appropriate for many patients, but certainly not all. This is because it requires getting to know the patient's specific clinical case in order to make the best decision. However, the process behind selecting an app to use with a patient is slightly different from selecting a form of psychotherapy or a medication. This is because making this type of decision is not what psychiatrists and mental health clinicians are classically trained to do. It is still a fairly novel process with which many clinicians may be unfamiliar.
The goal of the APA App Advisor’s evaluation process is to provide a hierarchical rating system with content-specific questions that guide a thoughtful approach to app selection. This process is designed to help APA members, patients, and other providers understand the key information that should be considered when choosing an app, and how this differs from selecting traditional therapeutic interventions. By highlighting these considerations, the evaluation process supports more informed app selection, enhances clinical decision-making, and promotes improved patient outcomes.
The App Evaluation Model
There is no minimum or maximum number of levels of the model the app needs to meet to be considered "good" or "useful." There may be instances where an app meets only a few levels of the evaluation framework but is still chosen by you and a patient. In such cases, the hierarchical model is valuable because it highlights both the presence and absence of features the APA considers important, helping to support an informed choice. Likewise, an app may satisfy all criteria but still not be one you or your patients decide to use. The purpose of the evaluation is to give the psychiatric and mental health providers and the patient sufficient information from which to make an informed decision that they deem correct for their situation.
Step 1: Background
The first step of the model is to help ensure that as much useful background information about the app is known before you evaluate it. This information helps create a useful context in which you can consider using the app and provides a framework for your decision making as you progress through the model. Thus, the questions below will help you decide whether to proceed with the app evaluation. You do not need to have an answer for each question in order to proceed with evaluating an app.
- Does the app identify ownership?
- Does the app come from a trusted source?
- Does the app identify funding sources and conflicts of interest?
- Is the purpose of the app clearly stated?
- Does the app claim to be medical?
- Are there costs to using the app?
- Has the app been updated in the last 180 days?
- Does the app provide the opportunity for users to provide feedback?
Step 2: Access
This level provides a structured guide for evaluating how well an application can be used in real-world settings. The questions are designed to assess the app’s technical functionality, inclusivity, and appropriateness for its intended audience. By examining factors such as connectivity requirements, device and platform compatibility, accessibility for individuals with disabilities, language and literacy considerations, support needs, and the identification of intended users, evaluators can determine whether the app is practical, equitable, and relevant for the population it aims to serve. This ensures that recommendations are not only clinically sound but also feasible and responsive to diverse user needs.
- Does the app function in low bandwidth or offline environments ?
- On which mobile platforms/operating systems does it work? (iOS, Android)
- Does the app work on a computer or web browser (Windows, MacOS, Safari, Chrome)?
- Does that app offer accessibility features for those with visual, hearing, or other disabilities?
- What languages does the app support?
- Does the app identify the reading level?
- What type of technical support and non-technical support will be needed for the intended user to begin using the app?
- Does the app identify intended users of the app (e.g. age, literacy level, cultural context, health condition)?
Step 3: Privacy and Security
While all interventions involve some level of risk (physical, psychological, social, or economic), digital tools present unique risks that are often underestimated. Apps may expose users to data costs, social profiling, loss of insurance benefits, or breaches of sensitive health information. Unlike medications or in-person therapy, privacy and security are among the most critical considerations when selecting an app.
The following questions are meant to guide consideration of app privacy and security. They are not exhaustive, and there is currently no gold standard for rating these features. Most answers should be available in the app’s privacy policy or on the developer’s website. If no clear policy exists, the app may not be appropriate for handling sensitive health information.
Because some details, such as security measures, may only be described by the developer, you may need to take these at face value. There is no scoring system for this level; instead, you and the patient should decide whether the app meets your standards. If key issues are missing or raise concerns, carefully consider whether the app is suitable for use. The ultimate aim is to ensure the app does not compromise patient safety, privacy, or security, and that both you and the patient understand its risks and limitations.
- Is the privacy policy easy to understand?
- What reading level are the policies written in?
- Does the app declare data use and purpose?
- Does the app collect, use, and/or transmit sensitive data, such as protected health information (PHI) ? If yes, does it do so securely?
- Does the app de-identify/anonymize sensitive data such as PHI?
- Does the app offer secure methods of user authentication (ie strong passwords, two-factor authentication or biometric identification)
- Can users opt out of data collection or delete data?
- Does the app state the process of deleting data or opting out of collection?
- Does the app say where it stores its data? (i.e. on the device, in the cloud, physical location of data warehouse)
- What third parties does the app share data with?
- Is the app equipped to respond to potential clinical safety concerns ?
- Does the app disclose how it responds to these safety concerns?
- Does the developer disclose a plan for addressing breaches of data security?
- How have they handled past incidents?
- Does the app use AI? If so, does it describe how AI is used?
- If the app uses AI, are there mechanisms in place for monitoring, disclosing, and mitigating bias and potential concerns?
Step 4: Clinical Foundation
App developers often make many claims on their apps' clinical effectiveness or background—though there is often little data to support the claims. This does not mean that apps do not work, but rather that there is much we still do not know. If you decide that an app has sufficient privacy and security, then your task at Step 4 is to evaluate any evidence for potential benefits.
While some apps' benefits have been documented in clinical studies, many—if not most—have not. Because of this, we recommend that you download and try the app to see what it is actually doing and if the content and information it offers appear at least reasonable and not harmful (i.e., evidence of "face validity"). Again, few apps will have a gold standard, randomized double blinded placebo controlled study to suggest they are effective, so the questions presented below are designed to help you think of other ways you can make an informed decision about an app's evidence base.
- Is the app content clinically accurate, up to date and relevant to the intended user?
- Does the app have a clinical/recovery foundation relevant to its intended use?
- Are there relevant sources of evidence supporting effectiveness of the app?
- Evidence sources could include publications, user feedback, etc.
- Was the app studied among intended users of various demographics?
- Does the app disclose clinical retention rates ?
- Did app developers involve their intended users in the design of the app?
Step 5: Usability
To recap, if an app has satisfied criteria in Steps One - Four within this Model, then you may assume that:
- It offers minimal risk in terms of digital safety and privacy.
- It appears to have some benefit.
Thus, Step 5 helps you to evaluate usability, because an app is only as useful as determined by you and your patients, after taking it for a spin. Because of this, usability is a more subjective category and so different people will have very different ideas about what this means to them. The questions below are, again, designed to help you think about the app's interface and overall functionality and then make an informed decision about how usable an app will be for the case and patient at hand.
- What are the modes of engagement of the app?
- Do the features align with the intended user and priorities?
- Can the user customize the app content?
- Is the app's design intuitive and easy to navigate (ie clearly marked buttons and navigational aids)?
- Is the content of the app arranged in a way that is easy to understand?
- Does the app offer technical support to users (i.e. tutorial, help section, live support)?
- Does the app adapt to user preferences and personalize content?
Step 6: Integration toward Patient-Centered Goals
The final step of the model focuses on how well an app integrates data into care. Data sharing only becomes meaningful once the app has been deemed usable (Steps 1–2), safe and secure (Step 3), evidence-informed (Step 4), and user-friendly (Step 5). At this topmost level, the goal is to ensure that apps support, rather than fragment, patient care by enabling patients and clinicians to access, share, and use data in ways that inform treatment and advance therapeutic goals.
For some apps, data integration may be irrelevant. For others—such as mood trackers or medication management tools—ensuring that data can be accessed and securely shared with the patient, care team, or medical record systems is a critical consideration. The following questions are intended to help evaluate whether an app’s data can be used in a clinically meaningful, patient-centered way:
- Is the app intended for individual use or in collaboration with the care team?
- Can the user access their data?
- Can the user export/download their data for their own use?
- Can data be shared securely with other members of the care team?
- Can the app share data with Electronic Medical Records, data tools , or wearable devices?
- Is the data provided in a way that is useful and meaningful toward patient-centered goals?