Agentic education · Learning science

Actionable companion · Self-regulation & metacognition

From AI evidence to responsible practice: Effectiveness and students’ perception of tutor-guided AI navigation in undergraduate medical education in Sri Lanka: a quasi-experimental study

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Health-professions education findings do not establish effectiveness in K–12 classrooms.

Original authors’ abstract, reproduced under the linked source license:

Background The integration of Artificial Intelligence (AI), particularly Large Language Models (LLMs) such as ChatGPT, into undergraduate medical education offers new opportunities for personalized learning. However, concerns remain regarding content accuracy, possible over-reliance and limited critical engagement when AI tools are used without guidance. Tutor-guided AI navigation has been proposed as a structured approach combining AI accessibility with educator oversight. This study aimed to evaluate the effectiveness and perception of tutor-guided AI navigation among medical undergraduates. Methods The study was conducted among 87 final-year medical students at the Faculty of Medicine, University of Kelaniya, Sri Lanka. A quasi-experimental pre-test/post-test design compared knowledge gains following tutor-guided versus self-directed ChatGPT-assisted learning. A cross-sectional survey assessing three domains-Perception of Learning Experience, AI Usability and Confidence, Satisfaction and Overall Impression- was administered immediately after the session. Responses were measured using a five-point Likert scale. Internal consistency was evaluated with Cronbach’s alpha and descriptive statistics were analysed using SPSS. Results Baseline pre-test scores were comparable between groups (W = 670.5, p = 0.153). Both groups improved significantly from pre- to post-test (p 0.86). Students rated ease of use (4.19 ± 1.07), tutor-guided AI experience (4.03 ± 1.13), satisfaction (3.91 ± 0.94) and willingness to recommend (4.01 ± 0.97) favourably, while trust in AI accuracy was moderate (3.27 ± 0.91). Conclusion Tutor-guided AI-assisted learning was well accepted by students and provided a positive, structured learning experience. Although it did not produce a statistically significant advantage in immediate knowledge gain over self-directed AI use, it was associated with favourable perceptions of usability, satisfaction and recommendation. These findings support the structured integration of tutor-guided AI into undergraduate medical education while maintaining educator oversight and critical appraisal of AI-generated content. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-026-09467-2.

What kind of evidence is this?

Supporting AI learning science · Comparative / pre-post study reported

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

This research was conducted in two parts: Quasi-experimental study: A pre-test/post-test design compared knowledge acquisition in students who underwent the tutor-guided AI session versus those who used ChatGPT independently (intervention: n = 59; control: n = 28). Cross-sectional survey: Assessed students’ perception after the tutor-guided learning session.

Limitations and uncertainty · source passage

The following limitations have been identified. The sample was restricted to final-year medical students at a single institution and a single clinical topic within Obstetrics and Gynaecology, which limits the generalisability of findings to broader medical education contexts. This study employed convenience sampling with non-randomised group allocation, which may introduce selection bias. The study measured only immediate knowledge gain; long-term retention and application in clinical contexts were not assessed. Both the pre-test and post-test comprised identical questions; therefore, score improvement may partly reflect familiarity with test items rather than solely representing knowledge acquisition. Future studies should consider using parallel forms to mitigate this confounder.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Use plan–attempt–check–reflect prompts and compare confidence with actual performance. Keep learner choice and reflection visible.
  • Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.
  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.
  • Co-design accommodations with learners and specialists. Check access and error burden across groups without inferring subgroup effectiveness from small or unreported samples.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

P. J. S. Randombage; M. Y. S. Ahamed; T. D. Kulathunga; H. Wickramasekara. Effectiveness and students’ perception of tutor-guided AI navigation in undergraduate medical education in Sri Lanka: a quasi-experimental study. BMC Medical Education (2026-05-18). https://doi.org/10.1186/s12909-026-09467-2

Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.