Actionable companion · Assessment & reasoning
Learning with agency: Development of a Computerized Adaptive Assessment and Learning System for Mathematical Ability Based on Cognitive Diagnosis
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participant population and task before transferring the findings.
Original authors’ abstract, reproduced under the linked source license:
Abstract With the rapid evolution of technology and the continuous deepening of digital transformation in education, personalized and adaptive learning have emerged as inevitable trends in the educational landscape. This study focuses on a Computerized Adaptive Learning System Based on Cognitive Diagnosis (CAL-CDS)—an integrated platform that incorporates multiple technologies for assessment and learning. The study is organized around two dimensions: (1) constructing a foundational cognitive diagnostic assessment framework, and (2) investigating the operational mechanisms of the cognitive diagnosis-based computerized adaptive system. It comprehensively incorporates core components including cognitive modeling, Q-matrix generation, and diagnostic test development. On this basis, this study dissects the system’s operational logic from four aspects: the adaptive testing system, diagnostic system, recommendation system, and empirical case studies. This study effectively addresses two core questions: how to construct a cognitive diagnostic assessment framework that alignes with China’s mathematics knowledge structure, and how to facilitate personalized student learning via cognitive diagnosis. Overall, this study offers a systematic solution for developing mathematics-specific cognitive diagnosis-driven adaptive learning systems. Keywords: adaptive learning system, cognitive diagnosis assessment, online learning
What kind of evidence is this?
Intelligent tutoring & adaptive learning · Design requires full-text appraisal
Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Study methods · source passage
Based on the adjacency matrix obtained, a Q matrix was developed to clarify the exam pattern. Considering the requirement of the number of tasks that students should complete in the exam, this study designed 2–3 items for the measurement of each attribute (Wu et al. 2022), Each item is marked in the Q-matrix with “1” or “0” in the corresponding knowledge attribute column (where “1” indicates that the item measures the attribute, and “0” indicates irrelevance). Furthermore, three independent teams were formed to conduct Q-matrix calibration tasks, with each team consisting of one educational measurement expert and one creativity researcher. To quantitatively validate the Q-matrix effectiveness, internal consistency analysis was performed: the Fleiss’ Kappa (Landis and Koch 1977) coefficient among six experts reached 0.89, which falls into the “almost perfect agreement” category. This result confirmed the validity of the constructed Q-matrix. Consequently, the finalized Q-matrix was determined. Taking the module “Concepts and Properties of Functions” as an example, the final Q-matrix is shown in Table 4.
Limitations and uncertainty · source passage
This study introduces the development of an innovative adaptive learning system grounded in cognitive diagnostic assessment theory. Although this online learning system serves as a valuable tool for diagnosing student learning outcomes and providing feedback, creating accurate learning resources for senior high school mathematics remains challenging. Specifically, the development of the recommendation functionality relies on a structured resource repository; however, the current research has not yet systematically implemented metadata annotation (e.g., knowledge point relevance, difficulty level) for these learning resources. Consequently, our system currently lacks the capability to provide corresponding learning resources to students. Addressing this gap will constitute the paramount research direction for further enhancing the adaptive online learning system. In addition, the validity verification of the assessment tool of this system mainly relies on expert judgment, and variations in experts’ knowledge backgrounds and experience preferences may lead to significant differences in the structural model. Furthermore, this study did not conduct a long-term longitudinal study or randomized controlled trial, which to some extent limits the ability to assess its long-term impact on learning effectiveness. In conclusion, this study holds theoretical value in the construction of cognitive diagnostic models and the implementation of test diagnostic functions, but its limitations are mainly reflected in the lack of a closed-loop mechanism, insufficient dynamics, and limited empirical depth. Future work should focus on developing a resource recommendation module, optimizing the dynamic diagnosis algorithm, and verifying the system effectiveness through large-scale educational experiments. Furthermore, integrating generative AI and multimodal interaction technologies has the potential to further enhance the system’s intelligence and educational accessibility.
Design the system before the pilot
Document the learner model, mastery rule, hint sequence and adaptation signal. Compare adaptive routing with the same content in a non-adaptive sequence. Check whether early errors lead to persistent low-expectation pathways.
- Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Yi Zhang; Liping Zhang; Heyang Zhang; Xiaopeng Wu. Development of a Computerized Adaptive Assessment and Learning System for Mathematical Ability Based on Cognitive Diagnosis. Journal of Intelligence (2025-09-02). https://doi.org/10.3390/jintelligence13090114
Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.