Agentic education · Learning science

Actionable companion · Assessment & reasoning

From AI evidence to responsible practice: Second Language Learners’ Cognitive Evaluation of GenAI Interactive Learning Experience: Scale Development and Network Analysis

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participants, intervention and comparator before applying this study in a new setting.

Original authors’ abstract, reproduced under the linked source license:

Abstract As generative artificial intelligence (GenAI) becomes increasingly integrated into second language (L2) learning, there is a growing need for reliable instruments to assess learners’ cognitive evaluation of GenAI-mediated interactive learning experiences. This study developed and validated the GenAI L2 Interactive Learning Experience Scale (AI-L2-ILES) among 1130 Chinese university students. Drawing on an interactive experience perspective and adapting it to L2 learning, the study conceptualized learners’ cognitive evaluation of GenAI interaction as comprising five dimensions: Effectiveness, Sociability, Novelty, Flow, and Seamlessness. Exploratory and confirmatory factor analyses supported this five-factor structure, with satisfactory evidence of reliability, validity, and measurement invariance across gender and academic disciplines. Network analysis further revealed that Flow was the most central dimension, Sociability showed the strongest association with Effectiveness, and Novelty occupied a relatively peripheral position. The network structure was stable across academic disciplines. Overall, the AI-L2-ILES provides a psychometrically sound measure of learners’ cognitive evaluation of GenAI-mediated interactive L2 learning experiences and captures five distinct but interconnected dimensions of this emerging construct. The scale offers a useful tool for examining how L2 learners cognitively evaluate their interactions with GenAI. Keywords: GenAI, L2 learning, cognitive evaluation, interactive learning experience, scale development, network analysis

What kind of evidence is this?

Supporting AI learning science · Design requires full-text appraisal

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

For sampling, convenience sampling was used. Recruitment announcements were posted on course-management platforms and student WeChat groups of five Chinese universities, inviting students who had used GenAI tools (e.g., ChatGPT, DeepSeek, Kimi) for L2 learning purposes. Inclusion criteria were: (a) current enrollment at one of the five participating universities, and (b) at least a month of experience using GenAI for L2 learning. Prior GenAI-assisted L2 learning experience was determined by two screening questions at the beginning of the questionnaire: one asking whether they had ever used GenAI tools for L2 learning (yes/no), and another asking about the frequency of such use (with options ranging from “rarely” to “daily”). Only those who answered “yes” and reported at least monthly use were included.

Limitations and uncertainty · source passage

This study makes three theoretical contributions. First, it proposes and validates the construct of AI-L2-ILE, conceptualized as a comprehensive experiential evaluation formed by L2 learners through their perceptions of the interaction process, interactional relationship, and learning value during sustained interactions with GenAI. This construct expands the conceptual boundaries of GenAI-assisted language learning research by shifting the focus from the functional attributes of GenAI to the structure of learners’ experiential evaluations, offering a learner-centered perspective for understanding how GenAI is experienced as an interactive learning partner in L2 learning. Second, this study identifies and validates the five-factor structure of the AI-L2-ILES (Effectiveness, Sociability, Novelty, Flow, and Seamlessness). The emergence of Effectiveness demonstrates how the goal-oriented nature of educational contexts can shape experiential dimensions: when learners evaluate GenAI-mediated interaction in relation to their learning goals, experiential quality may become closely linked to perceived learning value. This structural insight informs both L2 research and AI-in-education studies by specifying how learners distinguish different aspects of their interactions with GenAI in L2 learning. Third, this study introduces dimension-level network analysis into research on AI-L2-ILE and reveals differentiated association patterns among dimensions, including the strong connection between Sociability and Effectiveness, the relative independence of Novelty, and the central role of Flow. These findings provide a complementary perspective beyond traditional dimensional approaches for understanding the internal structure of this construct, with implications for understanding how learners experience GenAI through its interactive qualities in L2 learning.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Hanwei Wu; Gurpinder Singh Lalli; Yongliang Wang. Second Language Learners’ Cognitive Evaluation of GenAI Interactive Learning Experience: Scale Development and Network Analysis. Journal of Intelligence (2026-09-19). https://doi.org/10.3390/jintelligence14090226

Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.