Generative AI · Education leadership

Actionable companion · Staff capability and AI literacy

From evidence to district action: Use and Evaluation of Generative Artificial Intelligence by Medical Students in Japan

AI-assisted editorial implementation guide. The district actions are proposed applications, not findings or recommendations attributed to the original authors. No additional peer review is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Health-professions education context: test transfer to K–12 rather than assuming it.

Original authors’ abstract, reproduced under the linked source license:

Introduction: Generative artificial intelligence (AI) has become more accessible due to technological advancements. While it can support more efficient learning, improper use may lead to legal issues or hinder self-directed learning. Medical education is no exception, as generative AI has the potential to become a powerful tool. However, its practicality remains uncertain. Therefore, we investigated how generative AI is perceived among medical students and utilized within the realm of medical education. Methods: In January 2024, we conducted a study with 123 second-year medical students who had completed a physiology course and laboratory training at Gunma University, Japan. Students used ChatGPT (Chat Generative Pre-trained Transformer) 3.5 (OpenAI) for four tasks and evaluated its responses. A survey on the use of generative AI was also conducted. Responses from 117 participants were analyzed, excluding six non-participants. Results: Among the students, 41.9% had used ChatGPT. The average scores for tasks 1-4 were 6.5, 4.6, 7.4, and 6.2 out of 10, respectively. Although 13% had a negative impression, 54 students found it challenging to apply for medical purposes. However, 64.1% expressed a willingness to continue using generative AI, provided its use extended beyond medical contexts. Conclusions: Nearly 60% of students had never used generative AI before, which is consistent with general usage trends. Although they were impressed by the speed of generative AI responses, many students found that it lacked precision for medical studies and required additional verification. Limitations of generative AI, such as “hallucinations,” were evident in medical education. It remains important to educate students on AI literacy and their understanding of the potential issues that generative AI could bring about.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Run a staff workshop using three authentic lesson-planning tasks. Ask participants to identify hallucinations, revise outputs and document when they reject the tool. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Measure error-detection accuracy before and after training, preparation time and the quality of the final lesson plans. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Izuki Amano; Kisho Obi-Nagata; Ayane Ninomiya; Yuki Fujiwara; Noriyuki Koibuchi. Use and Evaluation of Generative Artificial Intelligence by Medical Students in Japan. JMA Journal (2025-07-02). https://doi.org/10.31662/jmaj.2024-0375

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.