The effect of the frequency of use of an intelligent tutoring system on learning gains in mathematics secondary education

Intelligent tutoring systems (ITS) are characterized by their direct and adaptive feedback as well as their capability of assessing the knowledge of students and administering exercises based on those assessments. The state of research regarding the effectiveness of ITS in mathematics is inconclusive. Hence, in this study, we examined the impact of utilizing an ITS on the learning gains in mathematics for students in grades 7 and 8. This longitudinal investigation was conducted with students from 55 classes (940 students) in northern Germany. Mathematics performance as well as relevant covariates were measured at the beginning and the end of the school year, and interactions with the ITS were recorded throughout the school year, providing a comprehensive dataset for analysis. Teachers were free in choosing the extent, subjects and methods of ITS usage. In addition, students could also use the ITS on their own. A multilevel analysis revealed that the frequency of ITS usage had no significant effect, neither at the class level nor at the individual level. Our results show that using ITS does not automatically lead to better learning gains. Therefore, future studies need to identify the conditions and practices that contribute to effective ITS use.