Reliability of individual teacher administrator practices during national monitoring assessment interviews in social studies

Since 1995, New Zealand’s National Education Monitoring Project (NEMP) has been responsible for the national assessment of students’ achievement in each of the learning areas in the curriculum. One of the assessment approaches used by NEMP is the one-to-one student interview. This paper addresses variation within individual teacher administrators’ practice as they conducted interviews during the 2005 round of monitoring in social studies. An observation schedule was used to gather data from 12 randomly selected teacher administrators, across 10 categories, as they carried out three administrations for each of three selected social studies tasks. It is argued that the variations observed within individual teacher administrators’ practice were related to elements of the assessment tasks and teacher administrators’ interpretations of these tasks. In addition, teacher administrators’ subject knowledge, their understanding of the teacher administrator role and their understanding of the “standardised” one-to-one interview process also contributed to variations within individuals. Overall, the nature and levels of variation observed have the potential to pose threats to reliability and have an effect on the consequential validity of information.