NEPS technical report for scientific literacy: Scaling results of starting cohort 1 for five-year-old children

The National Educational Panel Study (NEPS) examines the development of competencies across the life span and develops tests for the assessment of different competence domains. In order to evaluate the quality of these competence tests, various analyses based on item response theory (IRT) were performed. This paper describes the data and scaling procedures for the scientific literacy test that was administered to five-year-olds in starting cohort 1. The scientific literacy test contained 20 items with different response formats representing different contexts as well as different areas of knowledge. The test was administered to 2,080 children. Their responses were scaled using a partial credit model. Item fit statistics, differential item functioning (DIF), Rasch-homogeneity, the test’s dimensionality, and local item independence were evaluated to ensure the quality of the test. These analyses showed that the test exhibited an acceptable reliability and that all items fitted the model well. Furthermore, test fairness could be confirmed for different subgroups. Only one item showed a substantial gender DIF. The analysis of the test’s dimensionality supported a one-dimensional model. A limitation of the test was the lack of very difficult items. Overall, the scientific literacy test showed good psychometric properties that allowed for the estimation of reliable scientific literacy scores. Besides the scaling results, this paper also describes the data available in the scientific use file and provides the ConQuest syntax for scaling the data.