Assessing socioscientific argumentation: Exploration of potential threats to validity using the assessment triangle

This paper explores the challenges associated with assessing socioscientific argumentation (SSA) using the assessment triangle as a framework. Our considerations point to the difficulty of validly capturing SSA as it depends on how it is measured (observation) and how the construct is inferred from the observation (interpretation). Therefore, we systematically varied observation and interpretation by implementing four dilemmas and two coding schemes. In total, 64 preservice teachers for biology processed two to four dilemmas with different contexts, and their argumentation was analysed using two similar coding schemes for structural complexity to investigate whether there are systematic differences in the inferred level of SSA, even if the coding schemes are similar. Our findings show that, on average, the participants achieved an approximately medium level of SSA in both schemes. A significant difference in participants’ SSA levels when coded with the same scheme was only found between four of the 12 possible pairwise dilemma combinations. With regard to the relevance of interpretation, the participants achieved significantly higher levels in one coding scheme than in the other, regardless of the dilemma. These results indicate that the specific dilemma can but does not necessarily influence the level of SSA. Conversely, the choice of coding scheme has a major influence on the interpretation, which stresses the importance of carefully choosing an appropriate coding scheme.