Agentic education · Learning science

Actionable companion · Assessment & reasoning

Learning with agency: Applying GenAI to Optimize Q-Matrix Construction for Cognitive Diagnostic Assessment in EFL Reading

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Q-matrix construction is a foundational yet challenging step in cognitive diagnostic assessment (CDA), which is traditionally reliant on labor-intensive and subjective methods like expert judgment and verbal report analysis. This study explores the potential of generative artificial intelligence (GenAI) to optimize this critical process within the domain of EFL reading. By applying three GenAI models (DeepSeek-V3.2, Kimi 2.5, and Doubao 2.0), three purely GenAI-informed Q-matrices (Qmat-DS, Qmat-K, and Qmat-DB) were generated, and through expert revision, a human–AI collaborative Q-matrix (Qmat-DS-H) was obtained. These were compared with an expert-constructed Q-matrix (Qmat-E) and a student-derived Q-matrix (Qmat-S). Using a simulated dataset (N = 1000) and empirical response data from 1083 EFL learners on a diagnostic reading test, the psychometric performance of the six Q-matrices was estimated via the G-DINA model, ACDM model, and RRUM model. Results demonstrated that the human–AI collaborative Q-matrix consistently outperformed the other five Q-matrices, achieving the best absolute model-data fit, the highest classification accuracy, the most stable item parameters, and the most balanced attribute correlation structure. The purely GenAI-informed Q-matrices showed mixed results: there were some improvements in relative fit and slip stability compared to manually constructed Q-matrices, but variable absolute fit and attribute correlation patterns. The findings substantiate GenAI as a feasible pathway for enhancing the efficiency, consistency, and psychometric quality of Q-matrix construction. This study offers a preliminary framework for advancing CDA development, addressing a key methodological bottleneck in language assessment.

What kind of evidence is this?

Supporting AI learning science · Design requires full-text appraisal

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

This study adopted a comparative, cross-sectional design to evaluate the psychometric performance of six Q-matrices constructed through different methods: (1) experts’ Q-matrix (Qmat-E); (2) students’ Q-matrix (Qmat-S); (3) three GenAI-informed Q-matrices generated by DeepSeek (Qmat-DS), Kimi (Qmat-K), and Doubao (Qmat-DB); (4) one human–AI collaborative Q-matrix (Qmat-DS-H). All six Q-matrices were specified for the same diagnostic reading test and estimated using the same response dataset and the same cognitive diagnostic model. An additional simulated dataset was also generated using R program (version 4.5.1, R Core Team, Vienna, Austria) containing a similar sample size (N = 1000) and test length (20 items) to the empirical data for model robustness check. The guessing and slip parameters of the simulated 20 items were randomly generated within (0.05–0.15) as high quality (Ma & de la Torre, 2020). The diagnostic capacities were evaluated by model-data fit (both at the test level and item level) (RQ1), classification accuracy and attribute correlation (RQ2). Based on the reviewed literature, two hypotheses were formulated: (1) The three purely GenAI-generated Q-matrices (Qmat-DS, Qmat-K, Qmat-DB) will demonstrate comparable and improved psychometric performance compared to two manually constructed Q-matrices (Qmat-E and Qmat S). (2) The human–AI collaborative Q-matrix (Qmat-DS-H) will outperform all other Q-matrices across most psychometric indicators.

Limitations and uncertainty · source passage

In existing CDA studies, Q-matrices are mainly constructed via expert judgment (e.g., H. Chen et al., 2023; A. Kim, 2015; Ravand & Robitzsch, 2018; Toprak & Cakir, 2021) and verbal report analysis (e.g., Du & Ma, 2021; Jang, 2009; Javidanmehr & Anani Sarab, 2019). While both methods have contributed valuable insights, each is associated with specific methodological limitations regarding subjectivity, inter-expert variability, and scalability.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Wenbo Du; Jiayi Shen; Xiaomei Ma. Applying GenAI to Optimize Q-Matrix Construction for Cognitive Diagnostic Assessment in EFL Reading. Journal of Intelligence (2026-05-05). https://doi.org/10.3390/jintelligence14050079

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.