Actionable companion · Learning design & outcomes
From AI evidence to responsible practice: Can educators distinguish between medical student and generative AI‐authored reflections?
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Health-professions education findings do not establish effectiveness in K–12 classrooms.
Original authors’ abstract, reproduced under the linked source license:
Abstract Introduction Reflection is integral to the modern doctor's practice and, whilst it can take many forms, written reflection is commonly found on medical school curricula. Generative artificial intelligence (GenAI) is increasingly being used, including in the completion of written assignments in medical curricula. We sought to explore if educators can distinguish between GenAI‐ and student‐authored reflections and what features they use to do so. Methods This was a mixed‐methods study. Twenty‐eight educators attended a ‘think aloud’ interview and were presented with a set of four reflections, either all authored by students, all by GenAI or a mixture. They were asked to identify who they thought had written the reflection, speaking aloud whilst they did so. Sensitivity (AI reflections correctly identified) and specificity (student reflections correctly identified) were then calculated, and the interview transcripts were analysed using thematic analysis. Results Educators were unable to reliably distinguish between student and GenAI‐authored reflections. Sensitivity across the four reflections ranged from 0.36 (95% CI: 0.16–0.61) to 0.64 (95% CI: 0.39–0.84). Specificity ranged from 0.64 (95% CI: 0.39–0.84) to 0.86 (95% CI: 0.60–0.96). Thematic analysis revealed three main themes when considering what features of the reflection educators used to make judgements about authorship: features of writing, features of reflection and educators' preconceptions and experiences. Discussion This study demonstrates the challenges in differentiating between student‐ and GenAI‐authored reflections, as well as highlighting the range of factors that influence this decision. Rather than developing ways to more accurately make this distinction or trying to stop students using GenAI, we suggest it could instead be harnessed to teach students reflective practice skills, and help students for whom written reflection in particular may be challenging. Wraith et al. offer an empirical test, using mixed methods, of whether educators can distinguish between reflections offered by medical students and those created by generative AI.
What kind of evidence is this?
Supporting AI learning science · Qualitative / mixed methods
Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Study methods · source passage
An exploratory study was designed with firstly, a quantitative aspect which analysed participants' answers to who they thought authored each reflection. Secondly, a qualitative study was conducted following the think‐aloud interviews, 16 , 17 the transcripts of which were analysed using reflexive thematic analysis. 18
Limitations and uncertainty · source passage
Our study suggests that educators cannot reliably distinguish between written reflections completed by students and those authored by GenAI. Therefore, if a programme only requires students to submit written reflections, it is possible that students could use GenAI without genuinely engaging in reflective practice. We report sensitivity and specificity, describing the educators' ability to detect and exclude GenAI‐ authored reflections in our sample. In terms of correctly predicting the author, sensitivity ranged from 0.36 to 0.64, whilst specificity ranged from 0.64 to 0.86. The higher specificity (student reflections correctly identified) in contrast to sensitivity (GenAI‐authorship correctly identified), may reflect the higher familiarity of assessors with student‐authored reflections. The generalisability of these metrics to real‐world scenarios may vary depending on the actual prevalence of AI‐generated reflective pieces in undergraduate medical education.
Design the system before the pilot
Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.
- Specify a measurable learning objective, a comparator and a task without tool assistance before adopting a system.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Constance Wraith; Alasdair Carnegy; Celia Brown; Ana Baptista; Amir H. Sam. Can educators distinguish between medical student and generative AI‐authored reflections?. Medical Education (2025-07-02). https://doi.org/10.1111/medu.15750
Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.