Actionable companion · Feedback & scaffolding
From AI evidence to responsible practice: Development of a Universal Prompt as a Scalable Generative AI-Assisted Tool for USMLE Step 1 Style Multiple-Choice Question Refinement in Medical Education
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Health-professions education findings do not establish effectiveness in K–12 classrooms.
Original authors’ abstract, reproduced under the linked source license:
Abstract We developed a generative artificial intelligence (genAI)-assisted tool enabling learners to receive feedback on, revise, and clone multiple-choice questions aligned with learning objectives. Initially designed as a custom GPT, we adapted it to a universal prompt for platform-agnostic, equitable access. This innovation exemplifies readily adaptable genAI-enhanced learning driven by pedagogy. Supplementary Information The online version contains supplementary material available at 10.1007/s40670-025-02334-7. Keywords: Multiple-choice questions, Generative AI, Medical students, Prompts for medical education, Assessment writing
What kind of evidence is this?
Supporting AI learning science · Design requires full-text appraisal
Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Limitations and uncertainty · source passage
To overcome these limitations, we developed a more scalable and accessible solution: a universal prompt (Table 1) that can be used with any chatbot, by transferring essential components of the custom GPT into a single prompt (SI 2). All instructions needed to critique questions are contained within the prompt without requiring a separate knowledge file or memory, thereby lowering the threshold for use for a genAI beginner. This prompt was evaluated by four faculty from two medical schools, with varying MCQ revision experiences, and a student with no chatbot use history, using free versions of three chatbots (ChatGPT 4o, Claude Sonnet 3.5, and CoPilot). They confirmed the consistent functionality of the prompt while recognizing variations in tones and styles of each chatbot. This eliminates the financial and access barriers to using the genAI tool while maintaining the following features to enhance students’ MCQ writing learning experience.
Design the system before the pilot
Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.
- Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Youngjin Cho; Grace L Park; Gabi N Waite; Abhijay Mudigonda; John L Szarek. Development of a Universal Prompt as a Scalable Generative AI-Assisted Tool for USMLE Step 1 Style Multiple-Choice Question Refinement in Medical Education. Medical Science Educator (2025-02-25). https://doi.org/10.1007/s40670-025-02334-7
Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.