Actionable companion · Retention & transfer
Learning with agency: Emerging Technologies in Higher Education Course Development: A Systematic Review of Design Frameworks, Learning Outcomes, and Pedagogical Integration
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.
Original authors’ abstract, reproduced under the linked source license:
This review explored the impact of technology on course design, student learning outcomes, and engagement in higher education. The major enablers and barriers to implementation are also discussed. Following the PRISMA 2020 procedure, the systematic review integrated findings from 76 peer-reviewed empirical studies from 1996 to 2025. A mixed-method narrative and quantitative synthesis supported by statistical conversions mean standardized effect estimate (Cohen’s d) to enable cross-study comparison was used. Results showed that technology-involved instruction could be most effective with integration of an existing instructional design model. Technological Pedagogical Content Knowledge (TPACK); Utilization of Constructive Alignment (CA); Analyze, Design, Develop, Implement, and Evaluate (ADDIE), Design-Based Research (DBR) revealed moderate-to-high effects on learning outcome (d = 0.65-0.74), while deep learning model with embedded artificial intelligence (AI) and mixed/extended reality (MR) reported the highest effect (d = 1.30). For technology categories, the strongest effects on learning and engagement observed at immersive technologies (AR/VR/MR) (g = 0.98), followed by AI, and learning analytics (g = 0.62), and game-based learning (g = 0.71). Improvement in engagement was common across studies, while long-term behavioral and transfer effects were little observed. Success indicators identified were pedagogical alignment, institutional readiness, faculty competence, learner preparedness, whereas the challenges were infrastructure constraints, digital divides, and inadequate longitudinal evaluation. Synthesizing these through the lens of socio-technical systems perspective, results indicate that educational effectiveness is outcome of the relationship between technological affordances, pedagogical design, and organizational capacity.
What kind of evidence is this?
Supporting AI learning science · Evidence synthesis
Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Study methods · source passage
This review adheres to the PRISMA 2020 ( Page et al., 2021) procedure and statement. Due to variability of interventions and study designs within the emergent educational technologies domain, a narrative synthesis and determination of mean effect sizes were used. This enabled the integration of quantitative, qualitative, and mixed-method findings to capture the multifaceted nature of course development in higher education. Two key questions guided the focus of the review: (a) what empirical research is available on the design and delivery of degree-level courses incorporating emergent technologies; and (b) to consider the educational outcomes and contextual factors influencing success or failure. The review is, therefore, a blend of description mapping (technologies types, disciplines, geographical spread) with thematic aggregation (engagement, learning gains, and institutional readiness).
Limitations and uncertainty · source passage
At the same time, the data indicate that engagement was often assessed qualitatively or via self-report rather than via validated behavioral indicators of long-term engagement. This limitation is particularly pronounced in the generative AI meta-analysis which reveals strong cognitive and affective effects, but surprisingly small behavior change ( Liu & Zhong, 2025; Li et al., 2025). A homologous problem arises in serious game and VR research, where short-term motivation and attention are common measures, but enduring behavioral change and long-term transfer are infrequently addressed ( Lee et al., 2024; Radianti et al., 2020). Thus, the review shows that technology consistently leads to higher engagement, yet there is a need in the field for more robust measures of sustained participation, transfer, and longer term behavioral outcomes.
Design the system before the pilot
Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.
- Include a delayed assessment and an unfamiliar transfer problem without AI. Immediate supported performance cannot establish durable learning.
- Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Lemecha Geleto Wariyo; Addise Abame; Abebe Lolamo; Tola Bekene; Jabe Bekele. Emerging Technologies in Higher Education Course Development: A Systematic Review of Design Frameworks, Learning Outcomes, and Pedagogical Integration. F1000Research (2026-06-01). https://doi.org/10.12688/f1000research.179684.2
Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.