Verification and Validation of Adaptive Instructional Systems: A Text Mining Review

The current paper aims at qualifying the distribution of academic papers related to verification and validation of Adaptive Instructional Systems (AIS) and Adaptive software (ASOFT). For the purpose of the literature review, the theme of software verification and validation is divided into three sub-themes: 1) software verification, 2) empirical validation, and 3) model and simulation validation. In order to maintain the paper broad objectives, the approach will apply text mining techniques to analyze the literature. The corpus contained 33 546 documents extracted from Scopus (January 2024) in 5 document sets that were analyzed using TF-IDF vectors to measure cosine similarity between documents. The similarity distributions as well as the statistical tests indicate a difference in attention given to verification and validation issues between publications in AIS and ASOFT. The more important difference between the two sets of documents is the role given to software verification. The stronger emphasis on empirical, and model and simulation validation in the AIS literature points to the high dependency of considering human factors in the success of adaptive instructional systems. In regard to software verification in AIS, the few publications addressing this issue in comparison to publications looking at tutoring software engineering techniques indicate a possible research and practical gap to explore. The brief overview of the extended number of publications in software verification of ASOFT did not provide an immediate and clear set of methods and techniques that could be applied in the context of AIS. A further analysis could look in more depth in the ASOFT literature or at the AI development engineering literature.