Practicing with Educational Software

by Anna Hilz

Adaptive digital math educational software programs provide a promising way to support students with varying levels of achievement in math class. But how do students’ individual characteristics influence their use of these programs? To what extent is regular use linked to improved learning outcomes? An IPN study has examined these questions and provides initial insights into how teachers can ensure the successful integration of math learning programs in the classroom.

Figure 1: User interface of the Prowise Learn© learning programme. The exercises included in the programme are primarily designed to improve mental arithmetic skills. The programme’s adaptive algorithm presents learners with exercises in such a way that the probability of getting the answer correct is 75%.

Mathematical skills are essential for successful participation in modern society. Declining skill levels recently observed in major international student assessment studies, as well as the increasing heterogeneity in student performance, are therefore cause for concern and present teachers with the challenge of effectively supporting students with varying levels of prior knowledge.

Digital, adaptive math learning programs offer a promising solution here. They automatically adapt the tasks to the learners’ respective performance levels, thereby enabling targeted differentiation. Although research suggests that such programs can have a positive effect on math performance, the effects are often smaller than hoped for. Therefore, a central question is which factors influence learning success with such programs.

The IPN study presented here therefore examines the impact of the adaptive mathematics learning program Prowise Learn© on the usage patterns and learning outcomes of students at the start of secondary school (5th grade, comprehensive schools). Particular attention was paid to how individual student characteristics influence usage behavior and to what extent usage behavior is related to mathematics performance and motivational factors, such as self-concept in mathematics.

The students practiced with Prowise Learn© in mathematics class and at home for up to one school year to explore these questions.

Individual Characteristics and Their Impact on Usage Behavior

A key finding of the study is that not all students used the provided math learning program to the same extent. Students with higher levels of math anxiety or a migrant background, especially, showed lower usage rates. These students completed fewer assignments compared to their peers, suggesting that additional support could help encourage regular practice. Usage also declined over time, particularly during school breaks. These findings indicate that it is not only crucial to provide the program but also to continuously encourage and support its use.

Other individual characteristics, such as prior mathematical knowledge, conscientiousness, self-concept regarding mathematics, or socioeconomic status, had no significant influence on usage behavior. In other words, at least in this study, these characteristics did not appear to play a role in how intensively the students used the program. These results are encouraging in that lower-performing and thus potentially disadvantaged students—characterized by less prior knowledge, a lower self-concept regarding mathematics, or a lower socioeconomic status—did not seem to generally avoid the program.

Figure 2: User interface of the Prowise Learn© learning programme. Learners receive feedback after each task, with incorrect answers being corrected immediately. However, given the target response rate of 75%, learners should, on average, receive more positive feedback than corrective feedback.

Effects of the Learning Program on Performance and Self-Concept

Furthermore, the IPN study shows that usage patterns play a key role in learning success. Students who used the math learning program more frequently tended to perform better, particularly in more challenging areas such as subtraction. These findings underscore the importance of regular use of such programs to achieve learning success, as simply providing access to the program does not appear to be sufficient.

In addition to math performance, the study also examined the effects on math self-concept. Improvements in math self-concept were observed that appeared related to the provision of the program but seemed to be independent of the frequency of use. Given that adaptive learning programs incorporate motivation-enhancing mechanisms such as increased frequency of positive feedback, this result could be interpreted to mean that these mechanisms contribute to enhancing self-concept, whereby the principle “more is better” does not necessarily seem to apply. However, further research is needed to substantiate this assumption.

Practical Implications for Teaching

The study’s findings provide valuable insights into day-to-day school life. Firstly, the study shows that the effectiveness of math learning programs in terms of student performance depends on how frequently they are used. To maximize the effectiveness of these programs, it is therefore crucial that students practice with them regularly. Teachers should therefore continuously integrate the program into their lessons to ensure regular use, especially after school breaks, and make sure that students resume their practice.

Secondly, the results show that certain students, particularly those with high math anxiety or a migrant background, require additional support. Without this support, these students may not use the program as frequently and thus missing out on its potential advantages. Measures such as fostering a positive culture of error tolerance or ensuring access to suitable digital devices (e.g., tablets instead of smartphones) could help engage these students in the practice process.

Conclusion

In conclusion, digital math learning programs appear to be a promising tool for fostering mathematical skills, but their effectiveness seems to depend on how students actually use them. With this in mind, groups such as students with math anxiety or those from immigrant backgrounds require additional support to ensure they practice with the program regularly and effectively. Teachers must play an active role in implementing such programs. They should ensure not only that the program is made available but also it is used regularly and in a targeted manner. Which measures are most suitable in this context still needs to be determined in future research.

About the Author:

Dr. Anna Hilz studied Psychology at Kiel University and works as a researcher in the Department of Educational Science and Educational Psychology at the IPN. Her research focuses on exploring the potential of digital learning programs, particularly for students who struggle academically. Her work involves analyzing process data generated when students use these programs, in order to map their behavior while using them. The results presented here form part of her doctoral thesis. ahilz@leibniz-ipn.de

Further Reading:

Hilz, A., Guill, K., Roloff, J., Aldrup, K., & Köller, O. (2023). The relationship between individual characteristics and practice behaviour within an adaptive arithmetic learning program. Journal of Computer Assisted Learning, 39(3), 970–983. https://doi.org/10.1111/jcal.12780

Hilz, A., Guill, K., Roloff, J., Sommerhoff, D., & Aldrup, K. (2023). How to continue? New approaches to investigating the effects of adaptive math learning programs on students’ performance, self-concept, and anxiety. Journal of Intelligence, 11(6), Article 108. https://doi.org/10.3390/jintelligence11060108

Hilz, A., Hofman, A., Jansen, B., & Aldrup, K. (2025). Tracing students’ practice behavior in an adaptive math learning program: Does it mediate the math anxiety–performance link?. Learning and Instruction, 98, Article 102113. https://doi.org/10.1016/j.learninstruc.2025.102113