Actionable companion · Assessment & reasoning
Learning with agency: Development and psychometric validation of the AI chatbots acceptance and perception scale for higher education students
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.
Original authors’ abstract, reproduced under the linked source license:
Abstract The rapid integration of artificial intelligence (AI) chatbots in higher education has transformed student learning experiences and institutional operations, yet comprehensive measurement tools for assessing student usage patterns remain limited. This study presents the development and psychometric validation of the AI Chatbots Usage Scale, a comprehensive instrument measuring perceptual and attitudinal dimensions of AI chatbot acceptance among higher education students. Using a two-phase methodology, the research employed separate samples for Exploratory Factor Analysis (EFA) (n = 374) and Confirmatory Factor Analysis (CFA) (n = 599) among university students aged 17–23 from the Faculty of Education in Egypt. The initial 25-item scale underwent rigorous expert validation and pilot testing before final administration. EFA revealed a four-factor structure comprising Ease of Use, Perceived Usefulness, Trust, and Accessibility, accounting for 42.48% of total variance. CFA demonstrated excellent model fit indices, with the multidimensional model significantly outperforming a unidimensional alternative. Internal consistency reliability was excellent across all factors, with coefficients ranging from 0.731 to 0.928. The validated scale demonstrates strong theoretical alignment with established technology acceptance frameworks while extending traditional models to accommodate AI-specific considerations. These findings provide researchers and educational institutions with a reliable, theoretically grounded instrument for assessing and optimizing AI chatbot implementation in higher education contexts. Keywords: Artificial Intelligence chatbots, Higher education, Scale validation, Psychometric assessment Subject terms: Mathematics and computing, Psychology, Psychology
What kind of evidence is this?
Conversational & pedagogical agents · Survey / observational study
Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Participant ages · source passage
Using a two-phase methodology, the research employed separate samples for Exploratory Factor Analysis (EFA) (n = 374) and Confirmatory Factor Analysis (CFA) (n = 599) among university students aged 17–23 from the Faculty of Education in Egypt.
Study methods · source passage
This study employed a cross-sectional survey design with a psychometric validation approach to develop and validate the AI Chatbots Usage Scale for university students. The research was conducted during the 2024–2025 academic year and utilized a two-phase methodology involving separate samples for exploratory and confirmatory analyses.
Limitations and uncertainty · source passage
Several measurement instruments have been developed to assess student interactions with AI chatbots; however, each presents notable limitations that the current study seeks to address. Köhler and Hartig47 developed instruments assessing student knowledge, usage, and attitude toward ChatGPT, yet their scale was platform-specific and not grounded in a comprehensive technology acceptance framework. Çobanoğullari and Özbek48 developed a ChatGPT usage scale for foreign language learners, limiting its applicability to a single disciplinary context and a specific student population. Nemt-allah et al.49 validated a ChatGPT Usage Scale among postgraduate students, again restricting generalizability across broader undergraduate populations and AI chatbot platforms beyond ChatGPT. In contrast, the present study addresses these gaps by developing a platform-agnostic, theoretically grounded instrument applicable across disciplines and undergraduate populations, explicitly integrating established technology acceptance frameworks within a single psychometrically validated multidimensional structure.
Design the system before the pilot
Define whether the agent asks questions, gives hints, explains or assesses. Test hallucinated feedback, excessive praise, misleading confidence and responses when the learner is confused. A conversational interface alone does not establish autonomous agency.
- Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.
- Co-design accommodations with learners and specialists. Check access and error burden across groups without inferring subgroup effectiveness from small or unreported samples.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Ashraf Ragab Ibrahim; Mohamed Ali Nemt-allah. Development and psychometric validation of the AI chatbots acceptance and perception scale for higher education students. Scientific Reports (2026-07-22). https://doi.org/10.1038/s41598-026-62798-4
Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.