Agentic education · Learning science

Actionable companion · Learning design & outcomes

From AI evidence to responsible practice: Enhancing AI competence in health management: students’ experiences with ChatGPT as a learning Tool

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participants, intervention and comparator before applying this study in a new setting.

Original authors’ abstract, reproduced under the linked source license:

Background The healthcare industry has had to adapt to significant shifts caused by technological advancements, demographic changes, economic pressures, and political dynamics. These factors are reshaping the complex ecosystem in which healthcare organizations operate and have forced them to modify their operations in response to the rapidly evolving landscape. The increase in automation and the growing importance of digital and virtual environments are the key drivers necessitating this change. In the healthcare sector in particular, processes of change, including the incorporation of artificial intelligent language models like ChatGPT into daily life, necessitate a reevaluation of digital literacy skills. Methods This study proposes a novel pedagogical framework that integrates problem-based learning with the use of ChatGPT for undergraduate healthcare management students, while qualitatively exploring the students’ experiences with this technology through a thematic analysis of the reflective journals of 65 students. Results Through the data analysis, the researcher identified five main categories: (1) Use of Literacy Skills; (2) User Experiences with ChatGPT; (3) ChatGPT Information Credibility; (4) Challenges and Barriers when Working with ChatGPT; (5) Mastering ChatGPT-Prompting Competencies. The findings show that incorporating digital tools, and particularly ChatGPT, in medical education has a positive impact on students’ digital literacy and on AI Literacy skills. Conclusions The results underscore the evolving nature of these skills in an AI-integrated educational environment and offer valuable insights into students’ perceptions and experiences. The study contributes to the broader discourse about the need for updated AI literacy skills in medical education from the early stages of education.

What kind of evidence is this?

Supporting AI learning science · Qualitative / mixed methods

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

The study involved 89 third-year undergraduate students enrolled in a Health System Management degree program, specifically participating in a course on Service Quality in the Healthcare System during the 2023 academic year. The researcher, serving as the lecturer for this course, integrated writing reflective journals into the curriculum as part of the learning process. Following the course’s conclusion and after grades were distributed, the researcher asked students, in adherence to ethical guidelines, if they consented to have their reflective journals analyzed for research purposes, as outlined in the data collection section. Only students who completed all components of the intervention plan outlined for the class were considered potential participants in the research population.

Limitations and uncertainty · source passage

This study utilized a pedagogical framework that integrates problem-based learning with the use of ChatGPT. While the researcher focused on the pedagogical aspect, future research is warranted to compare this digitally supported activity to a non-digital equivalent and examine the impact on students’ literacy and skills. Such a comparison would make it possible to assess what the digital instrument contributes to skill development and to identify any challenges encountered.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Specify a measurable learning objective, a comparator and a task without tool assistance before adopting a system.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Lior Naamati-Schneider. Enhancing AI competence in health management: students’ experiences with ChatGPT as a learning Tool. BMC Medical Education (2024-05-30). https://doi.org/10.1186/s12909-024-05595-9

Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.