Actionable companion · Motivation & engagement

Learning with agency: Combining Artificial Intelligence and Human Support in Mental Health: Digital Intervention With Comparable Effectiveness to Human-Delivered Care

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Background Escalating mental health demand exceeds existing clinical capacity, necessitating scalable digital solutions. However, engagement remains challenging. Conversational agents can enhance engagement by making digital programs more interactive and personalized, but they have not been widely adopted. This study evaluated a digital program for anxiety in comparison to external comparators. The program used an artificial intelligence (AI)–driven conversational agent to deliver clinician-written content via machine learning, with clinician oversight and user support. Objective This study aims to evaluate the engagement, effectiveness, and safety of this structured, evidence-based digital program with human support for mild, moderate, and severe generalized anxiety. Statistical analyses sought to determine whether the program reduced anxiety more than a propensity-matched waiting control and was statistically noninferior to real-world, propensity-matched face-to-face and typed cognitive behavioral therapy (CBT). Methods Prospective participants (N=299) were recruited from the National Health Service (NHS) or social media in the United Kingdom and given access to the digital program for up to 9 weeks (study conducted from October 2023 to May 2024). End points were collected before, during, and after the digital program, as well as at a 1-month follow-up. External comparator groups were created through propensity matching of the digital program sample with NHS Talking Therapies (NHS TT) data from ieso Digital Health (typed CBT) and Dorset HealthCare (DHC) University NHS Foundation Trust (face-to-face CBT). Superiority and noninferiority analyses were conducted to compare anxiety symptom reduction (change on the 7-item Generalized Anxiety Disorder Scale [GAD-7]) between the digital program group and the external comparator groups. The program included human support, and clinician time spent per participant was calculated. Results Participants used the program for a median of 6 hours over 53 days, with 232 of the 299 (77.6%) engaged (ie, completing a median of 2 hours over 14 days). There was a large, clinically meaningful reduction in anxiety symptoms for the digital program group (per-protocol [PP; n=169]: mean GAD-7 change –7.4, d=1.6; intention-to-treat [ITT; n= 99]: mean GAD-7 change –5.4, d=1.1). The PP effect was statistically superior to the waiting control (d=1.3) and noninferior to the face-to-face CBT group (P<.001) and the typed CBT group (P<.001). Similarly, for the ITT sample, the digital program showed superiority to waiting control (d=0.8) and noninferiority to face-to-face CBT (P=.002), with noninferiority to typed CBT approaching significance (P=.06). Effects were sustained at the 1-month follow-up. Clinicians overseeing the digital program spent a mean of 1.6 hours (range 31-200 minutes) of clinician time in sessions per participant. Conclusions By combining AI and human support, the digital program achieved clinical outcomes comparable to human-delivered care, while significantly reducing the required clinician time by up to 8 times compared with global care estimates. These findings highlight the potential of technology to scale evidence-based mental health care, address unmet needs, and ultimately impact quality of life and reduce the economic burden globally. Trial Registration ISRCTN Registry ISRCTN52546704; http://www.isrctn.com/ISRCTN52546704

What kind of evidence is this?

Conversational & pedagogical agents · Design requires full-text appraisal

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

This was a pragmatic, single-intervention arm study with multiple external control groups to measure the engagement, clinical effectiveness, acceptability, and safety of a digital program to alleviate symptoms of generalized anxiety in a sample of 300 UK participants. This study was conducted by ieso Digital Health (“ieso” [29]), an outpatient service provider within NHS TT delivering one-to-one human-delivered CBT via a typed modality to treat patients with common mental health disorders. The digital program evaluated here (ie, IDH-DP2-001) was developed by ieso as part of a clinical innovation program creating new scalable digital solutions for mental health support. This was an externally controlled trial, meaning comparator arms (sometimes referred to as synthetic control arms) were generated through one-to-one propensity matching of participants with real-world patients. External propensity-matched control groups were generated to evaluate the digital program in comparison to no intervention (ie, waiting control), face-to-face CBT (gold-standard benchmark), and typed CBT. This latter group provides an important comparator as it is an example of human-delivered care that closely mirrors the written content delivery within the digital program.

Limitations and uncertainty · source passage

First, compensation for time may have encouraged greater adherence to the program. Second, the prospective sample had limited low-mood symptoms. In line with the study’s exclusion criteria—based on the program’s specificity for anxiety—individuals with severe depressive symptoms were not included. Nevertheless, the propensity matching across groups accounted for this, as all groups included patients with similar baseline anxiety and depression symptoms. Third, there were differences in PP sample sizes across the control groups. These differences were likely driven by the definition of PP in each context rather than actual engagement, given the similar adherence rates across groups. Defining a comparable PP sample across external control groups is challenging due to differences in dose intensity, delivery mechanisms, data collected, and response to treatment. The PP samples for the therapy control groups were based on completed episodes of care, and thus were agnostic to therapy dose—potentially including individuals who received only a few sessions but recovered quickly. Such individuals would not have been included in the digital program PP sample, which was conservatively defined based on minimum program interaction over a 9-week period. Fourth, outcomes for the propensity-matched controls were not collected concurrently with those of the prospective digital program sample, potentially introducing temporal biases. For example, the face-to-face CBT data available for analysis spanned the COVID-19 pandemic; however, only 5 matched patients in the face-to-face control group received care during 2020-2021, and their data were not outliers—making it unlikely that this impacted the findings. Propensity matching within a pragmatic design makes these findings highly relevant for real-world implementation. However, this study was not randomized and relied on patient-reported outcomes, thereby introducing risks of unmeasured confounding factors. A prospective randomized clinical trial with additional clinician-reported outcomes will be important to confirm the clinical efficacy of the digital program.

Design the system before the pilot

Define whether the agent asks questions, gives hints, explains or assesses. Test hallucinated feedback, excessive praise, misleading confidence and responses when the learner is confused. A conversational interface alone does not establish autonomous agency.

Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Clare E Palmer; Emily Marshall; Edward Millgate; Graham Warren; Michael Ewbank; Elisa Cooper; Samantha Lawes; Alastair Smith; Chris Hutchins-Joss; Jessica Young; Malika Bouazzaoui; Morad Margoum; Sandra Healey; Louise Marshall; Shaun Mehew; Ronan Cummins; Valentin Tablan; Ana Catarino; Andrew E Welchman; Andrew D Blackwell. Combining Artificial Intelligence and Human Support in Mental Health: Digital Intervention With Comparable Effectiveness to Human-Delivered Care. Journal of Medical Internet Research (2025-05-13). https://doi.org/10.2196/69351

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.