Context counts: Unveiling the impact of achievement level on teachers’ text assessment

Background: Teachers' assessments of student writing proficiency are essential for adaptive instruction and providing feedback. However, studies have revealed a lack of objectivity in text quality ratings of the same text in different contexts, although texts of identical quality should receive identical teacher judgments. Aims: Our two studies – one experimental and one more ecologically valid – address whether the achievement level impacts teachers’ individual text assessments even when relevant explicit criteria are used. Samples: In Study 1, participants were 102 student teachers from a university in Germany. In Study 2, participants were 136 student teachers and 91 experienced teachers from Germany. Methods: In Study 1, participants were randomly assigned to two groups. Student teachers in the high-achievement level group assessed three high-quality texts and one standard text with average quality. Student teachers in the low-achievement level group assessed three low-quality texts and the same standard text with average quality. Participants in Study 2 assessed ten texts randomly drawn from a large corpus. Results: The comparison of the standard text in Study 1 showed that student teachers in the high-level achievement group assessed it more negatively on a holistic scale and three analytic scales compared to the low-level achievement group. Multi-level data analysis in Study 2 revealed a negative influence of the quality of the achievement level on the text assessment for experienced and for student teachers. Conclusions: We discuss how the achievement level can bias teachers’ judgments of specific student performances when concrete assessment criteria should be used.