Using ordered multiple-choice items to assess students’ understanding of the structure and composition of matter

Helping students develop a sound understanding of scientific concepts can be a major chal-lenge. Lately, learning progressions have received increasing attention as means to support students in developing understanding of core scientific concepts. At the center of a learning progression is a sequence of developmental levels reflecting an idealized progression towards understanding a particular core concept. This sequence is supposed to serve as a basis for de-signing instruction that can foster and assessments that can monitor students’ progression. So called ordered multiple-choice (OMC) items have recently been suggested as a simple and effective way of assessing students’ level of understanding of a core concept. This article de-tails our efforts in developing an instrument for assessing students’ understanding of the structure and composition of matter based on OMC items. Ten OMC items were developed and administered to a sample of N = 294 students from grade 6 to 12. Rasch analysis was used to investigate instrument functioning and to determine linear measures of person abilities and item difficulties. In addition to the OMC items students were administered corresponding open ended items in order to investigate the validity of the results obtained through the OMC items. Our findings suggest assessing students’ understanding of scientific concepts through OMC items is indeed very worthwhile and should be subject to further research.