Agentic education · Learning science

Actionable companion · Motivation & engagement

Learning with agency: AI Tool Use Among Osteopathic Medical Students: Pilot Digital Diary Study

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Health-professions learning must be distinguished from K–12 instruction.

Original authors’ abstract, reproduced under the linked source license:

Abstract Background AI is increasingly integrated into medical education, offering new ways for students to acquire knowledge and support clinical reasoning. However, the extent, patterns, and implications of AI use among medical students remain incompletely understood. Prior studies have relied on retrospective surveys that are susceptible to recall bias and have not quantified AI use as a proportion of total study time. Objective This pilot study aimed to quantify real-time AI use among medical students, including the proportion of study time devoted to AI, and how use varies by training stage and engagement style (active vs passive). Active use was defined as iterative, bidirectional engagement; passive use was defined as unidirectional consultation with limited interrogation. Methods This longitudinal observational cohort study recruited medical students from 2 osteopathic medical schools (April-May 2025) to complete a baseline survey and 7 digital diary entries over a 21-day period, delivered via automated SMS every 3 days. Students reported total study time, AI use time, tools used, and purposes of use. The data were analyzed using Stata 19. Multiple linear regression models examined associations between AI use (total minutes and percentage of study time) and key variables, and a mixed-effects model using diary-level data with a random intercept per student addressed within-person variability across entries. Results A total of 71 of 1332 (response rate: 5.3%) eligible students completed the baseline survey (mean age 26.6, SD 2.8 y; n=39, 55% identified as men; n=32, 45% identified as women; n=55, 77% preclinical). On average, students reported using AI tools during 19% of their total study time (mean 35.8 of 185.6 min per diary, SD 35.8 min). The most used tool was ChatGPT (n=63, 89%), followed by Google Gemini (n=22, 31%). Clinical-phase students (MS3-MS4) used AI significantly more than preclinical students (MS1-MS2), with an adjusted increase of 19% (P=.003). Students classified as active users spent significantly more total time using AI than passive users (P=.002). Across groups, AI use was primarily passive, including simplifying complex concepts, answering practice questions, and generating summaries. In the multilevel models, preclinical students reported significantly lower AI use than clinical-phase students (P=.02). The intraclass correlation coefficient was 0.47 (95% CI 0.34‐0.60). Conclusions Although preliminary, these findings suggest that medical students are incorporating AI into a substantial proportion of their study time, with greater use among clinical trainees and active users. Despite this, most use remains passive. Given mixed evidence on AI’s impact on deep learning, further research on learning outcomes is needed. Institutions may consider providing guidance on responsible AI use, including critical evaluation and verification of outputs. The digital diary methodology offers a practical approach for capturing real-time AI use and may inform future educational research and intervention design.

What kind of evidence is this?

Supporting AI learning science · Survey / observational study

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Participant ages · source passage

Results A total of 71 of 1332 (response rate: 5.3%) eligible students completed the baseline survey (mean age 26.6, SD 2.8 y; n=39, 55% identified as men; n=32, 45% identified as women; n=55, 77% preclinical).

Study methods · source passage

This was a longitudinal observational cohort study using a digital diary methodology. The study was conducted at Touro University California (TUC) and Touro University Nevada (TUN), both accredited colleges of osteopathic medicine in the United States. This study is reported in accordance with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement for cohort studies; the completed checklist is provided in Checklist 1. Recruitment spanned from April 10 to 28, 2025. Each participant began a 21-day digital diary upon completing the baseline survey; the final diary entry was collected on May 19, 2025. The study consisted of a one-time intake survey followed by a 21-day digital diary phase (Figure 1). Digital diaries have been shown to be a valid method of collecting responses while minimizing recall bias [16]. Participants received survey prompts 3 times per week for 3 weeks—an approach that balanced reducing recall bias and maximizing participation [17].

Limitations and uncertainty · source passage

This study has several limitations. Self-reported data, even with a diary methodology designed to reduce recall bias, remain subject to social desirability bias and time estimation error; self-reported minutes may not precisely reflect actual usage, and triangulation with device-level data was not possible.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.
  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Carinne Brody; Seth Schwindt; Achint Thakur; Pieter von Steinbergs. AI Tool Use Among Osteopathic Medical Students: Pilot Digital Diary Study. JMIR Formative Research (2026-09-01). https://doi.org/10.2196/96895

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.