Development and Psychometric Validation of the Artificial Intelligence Literacy Scale for University Students: Evidence from Exploratory and Confirmatory Factor Analysis

Abstract

Background of study: Artificial intelligence (AI) is now embedded in university learning, assessment, research, and academic decision-making, increasing the need for psychometrically sound measures of students’ AI literacy. Existing instruments differ substantially in scope, dimensionality, and evidence of validity. Aims and scope of paper: This study develops the Artificial Intelligence Literacy Scale for University Students (AILS-U), a concise multidimensional self-report instrument designed for cross-disciplinary higher-education contexts. Methods: Following best-practice scale-development procedures, an initial item pool was subjected to expert content validation, cognitive review, pilot testing, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). Because no empirical dataset was supplied for this manuscript draft, numerical results are transparently demonstrated using a synthetic sample (N = 742; EFA n = 371; CFA n = 371) and must be replaced before submission. Result: The illustrative EFA supports a 24-item, five-factor structure covering conceptual understanding, applied use and human-AI collaboration, critical evaluation and verification, ethical and responsible use, and human agency and self-regulation. The illustrative CFA shows good fit (CFI = .958, TLI = .952, RMSEA = .050, SRMR = .045), with strong reliability and convergent/discriminant validity. Conclusion: The proposed AILS-U offers a theoretically integrated framework for assessing university students’ AI literacy, but publication requires validation with verified participant data and cross-context replication.