The Progressive Achievement Tests (PATs) from NZCER have been undergoing a refresh in the last two years, including significant changes to visuals and content. This is to ensure all ākonga are reflected in the assessments, and includes some updates to our robust design and reporting.
Our refreshed maths assessment is now known as PAT Pāngarau | PAT Mathematics, and has replaced all previous versions of the tests in both digital and physical formats. Schools can administer the refreshed assessment either online or with pen and paper. In both cases our NZCER Assist platform is used to provide comprehensive reporting for schools.
PAT Pāngarau | PAT Mathematics includes nine new tests, suitable for ākonga from Year 3 to Year 10. It also remains available as a computer adaptive test that actively adjusts its difficulty based on ākonga responses.
From the beginning of Term 1 2024, we are rolling out the latest changes to PAT Pāngarau | PAT Mathematics – recalibration of the question bank and an update to the assessment norms.
Minimal scale and score changes for 2024
The 2024 recalibration of the PAT Pāngarau measurement scale has resulted in a slight change, with generally little impact on scores, but some changes (largely within the margin of measurement error) that are explained below.
NZCER has analysed the question bank for the maths assessment, and recalibrated the bank of questions that is used across the tests. This is an important part of our refresh, ensuring the reported scores reflect the most recent information about how students respond to questions.
Recalibrating simply means locating the relative difficulty of each question on the Rasch measurement scale which underpins all the tests. These locations are used to calculate the scale scores that you see when a student completes their assessment. To carry out the recalibration we have analysed data from tens of thousands of mathematics assessments on our NZCER Assist platform, and used this to make adjustments.
In this 2024 recalibration, NZCER found that changes to the location of items on the scale generally had little impact on students’ scale scores. However, students at the top of the scale may see a small decrease in scale score, and students at the bottom may see a small increase, as a result of the recalibrated scale. For all but a handful of students, this change will be very small (within measurement error). This means that a student’s scale scores are still comparable over time, including between the previous PAT: Mathematics tests and the refreshed PAT Pāngarau | PAT Mathematics tests.
Small changes to national reference group information
NZCER has also updated the national reference information for PAT Mathematics. This is the first update since 2009.
The reference information indicates how well a student’s scale score compares with the scores for nationally representative groups of students at different year levels – we report this using stanines.
The updated reference information for students in Years 4 to 10 will be available in 2024, with Years 3 and 11 being added in 2025, once enough data has been gathered.
As could be expected over a 15-year period, there are some differences between the 2024 and 2009 reference information. The 2024 data shows higher year levels recorded slightly lower average scores than in 2009, and the lower year levels recorded slightly higher average scores.
At least some of the difference in the pattern of scores reflects the results of recalibration, where changes to the scale were more likely to affect the highest and lowest scores on the scale.
What you’ll notice in your PAT Pāngarau | PAT Mathematics reporting
For tests administered from the start of 2024, the NZCER Assist platform will base the reporting of scale scores and stanines on the new question calibrations and reference information.
For the reasons discussed above, teachers working with students in Years 8 to 10 might notice that in terms of stanines students appear to be doing slightly better than in previous years, and teachers working with students in Years 4 and 5 may notice that students appear to be doing slightly less well.
This should not necessarily be taken to reflect an actual change in achievement. As indicated above, it could reflect the updates to the reference information and the scale that underpins it.
Changes in the scale scores are a more accurate reflection of change over time. Again, as noted, for most students, scale scores will be directly comparable across the old and new PAT: Mathematics scales, taking into account measurement error.
If you wish to generate reports for tests taken prior to 2024, these will be reported using the older calibrations and reference information – which remains broadly comparable to the new information.
If you have any questions about the changes to PAT Pāngarau | PAT Mathematics or this information more generally, you can contact our Assessment Services team at assessmentservices@nzcer.org.nz or request PLD support from our Kaitohu Mātauranga (Education Advisors) here.