Multimedia effects in testing: A meta-analysis on cognitive, metacognitive and affective effects of pictures in test items

Illustrations are increasingly used in large-scale assessments like the Programme for International Student Assessment, highlighting the emerging field of multimedia testing, which examines the effects of pictures in test contexts. These effects, often referred to as multimedia effects in testing, have been linked to performance, time on task, metacognition, and test motivation. The present meta-analysis investigated the effects of three types of pictures on these four educational outcomes in testing and problem solving. It is the first meta-analysis to include decorative pictures and affective-motivational outcomes, offering novel insights for the field. Based on n = 42 studies (k = 158 effect sizes), our findings provide empirical evidence that the picture type is a critical moderator for multimedia effects in testing. Decorative pictures had no significant impact on performance (g = −0.02; p = .669), time on task (g = −0.10; p = .118), metacognition (g = 0.22; p = .382), or affective-motivational outcomes (g = 0.03; p = .302). In contrast, representational pictures showed significant positive effects on performance (g = 0.35; p < .001), metacognition (g = 0.32; p < .001), and affective-motivational outcomes (g = 0.21; p < .001), though not on time on task (g = −0.01; p = .938). The effects of informative pictures were highly heterogeneous, suggesting the presence of additional moderating variables. Our work provides, based on the conducted moderator analyses, important starting points for a more differentiated examination of moderating factors in future primary studies, which may provide clearer insights into the processes that drive multimedia effects in testing.