Actionable companion · Feedback & scaffolding
Learning with agency: Lessons Learned and Future Directions of MetaTutor: Leveraging Multichannel Data to Scaffold Self-Regulated Learning With an Intelligent Tutoring System
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.
Original authors’ abstract, reproduced under the linked source license:
Self-regulated learning (SRL) is critical for learning across tasks, domains, and contexts. Despite its importance, research shows that not all learners are equally skilled at accurately and dynamically monitoring and regulating their self-regulatory processes. Therefore, learning technologies, such as intelligent tutoring systems (ITSs), have been designed to measure and foster SRL. This paper presents an overview of over 10 years of research on SRL with MetaTutor, a hypermedia-based ITS designed to scaffold college students’ SRL while they learn about the human circulatory system. MetaTutor’s architecture and instructional features are designed based on models of SRL, empirical evidence on human and computerized tutoring principles of multimedia learning, Artificial Intelligence (AI) in educational systems for metacognition and SRL, and research on SRL from our team and that of other researchers. We present MetaTutor followed by a synthesis of key research findings on the effectiveness of various versions of the system (e.g., adaptive scaffolding vs. no scaffolding of self-regulatory behavior) on learning outcomes. First, we focus on findings from self-reports, learning outcomes, and multimodal data (e.g., log files, eye tracking, facial expressions of emotion, screen recordings) and their contributions to our understanding of SRL with an ITS. Second, we elaborate on the role of embedded pedagogical agents (PAs) as external regulators designed to scaffold learners’ cognitive and metacognitive SRL strategy use. Third, we highlight and elaborate on the contributions of multimodal data in measuring and understanding the role of cognitive, affective, metacognitive, and motivational (CAMM) processes. Additionally, we unpack some of the challenges these data pose for designing real-time instructional interventions that scaffold SRL. Fourth, we present existing theoretical, methodological, and analytical challenges and briefly discuss lessons learned and open challenges.
What kind of evidence is this?
Conversational & pedagogical agents · Design requires full-text appraisal
Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Limitations and uncertainty · source passage
Results demonstrate that in different situations or given different contextual factors, learners might benefit from using metacognitive processes in different ways. By knowing this, do we want to continue randomly assigning them to conditions we know will not be useful to them? For example, students with low prior knowledge demonstrated a lack of use of cognitive strategies in Taub and Azevedo (2019), but do we want to inundate them with prompts when they need to allocate a substantial number of resources to learn the material? If students (regardless of prior knowledge) are not being provided with any feedback from the PA from being in the control condition, are they at a disadvantage? How does this impact how we design experimental conditions to ensure sufficient randomization? Thus, implications for future research that examines learning with advanced learning technologies should encourage researchers to employ a within-subjects design to ensure students are exposed to all possible learning contexts so they can benefit from learning with these systems. Additionally, we should consider how a learner’s needs may change over the learning session (i.e., they may require prompt and feedback support early, but not later on) and over multiple learning sessions as they learn more strategies.
Design the system before the pilot
Define whether the agent asks questions, gives hints, explains or assesses. Test hallucinated feedback, excessive praise, misleading confidence and responses when the learner is confused. A conversational interface alone does not establish autonomous agency.
- Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
- Use plan–attempt–check–reflect prompts and compare confidence with actual performance. Keep learner choice and reflection visible.
- Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Roger Azevedo; François Bouchet; Melissa Duffy; Jason Harley; Michelle Taub; Gregory Trevors; Elizabeth Cloude; Daryn Dever; Megan Wiedbusch; Franz Wortha; Rebeca Cerezo. Lessons Learned and Future Directions of MetaTutor: Leveraging Multichannel Data to Scaffold Self-Regulated Learning With an Intelligent Tutoring System. Frontiers in Psychology (2022-06-14). https://doi.org/10.3389/fpsyg.2022.813632
Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.