Addressing issues of equity
Progressive Achievement Tests (PAT) mathematics
BRONWYN GIBBS
Rangahau Mātauranga o Aotearoa / New Zealand Council for Educational Research (NZCER) is refreshing the PAT mathematics assessments. As the owners and administrators of PATs and strong proponents of Te Tiriti o Waitangi, NZCER has a responsibility to ensure our assessments evolve and reflect the unique context of Aotearoa New Zealand. The following article outlines the PAT mathematics voyage as we refresh our assessments to be more equitable, culturally relevant, authentic, and accessible for diverse learners in Aotearoa New Zealand. Concepts of whakatere tōmua—wayfinding—anchor this refresh mahi and our aspirations for this complex journey of change.
Rangahau Mātauranga o Aotearoa / New Zealand Council for Educational Research (NZCER) is committed to improving equity in education. This commitment is about finding ways to realise the strengths, hopes, and dreams of every ākonga, and supporting and catering for underserved groups in the education system. These groups include ākonga Māori, Pacific learners, those from low socioeconomic backgrounds, and learners with additional learning needs.
In 2022 NZCER started a refresh of PAT mathematics, with the intention of providing a lever for equity through more culturally relevant, authentic, and accessible assessments. Concepts of whakatere tōmua—wayfinding—anchor this mahi and the aspirations of the assessment refresh team. This article utilises concepts of wayfinding to outline the PAT mathematics journey as we refresh our assessments to be more equitable, and culturally relevant for diverse learners in Aotearoa New Zealand.
Accepting the challenge to embark on new journeys
PAT mathematics assessments are part of a suite of assessments that were developed to support teachers and school leaders to understand what level learners are at, what progress they are making, and how teachers can meet the learning needs of their students. These assessments are standardised, meaning they are administered, scored, and interpreted in a consistent, or standard, way. While standardised assessments are a straightforward way to gather reliable and useful information, they also treat students uniformly with little recognition of the inherent differences and diversities that exist among learners (Mahuika et al., 2011).
NZCER recognises that mathematics and mathematics assessment practices are culturally located, and that ākonga bring their own backgrounds, experiences, cultural perspectives, traditions, and knowledge to assessment. NZCER has therefore embarked on the journey of improving the cultural relevance of PAT mathematics in order to create opportunities for diverse learners to show their mathematical capabilities with assessments that are more responsive to their cultural identities.
Stepping off solid land
The first step for the NZCER PAT mathematics refresh team was to review each of the current questions and make decisions as to whether the item was mathematically relevant, culturally authentic, and accessible. To ensure continuity the PAT mathematics refresh team could make changes to up to one third of the items.
The PAT mathematics refresh team were guided by an internally developed NZCER cultural perspectives tool (Hunia et al., 2020), current national and international research in equity and assessment in mathematics education (Kerr and Averill, 2021), and consultation with a teacher advisory panel. Most of the teacher advisory panel are Māori and Pacific teachers. They all work in low-decile schools with high Māori and Pacific rolls, and have extensive experience as teachers of mathematics.
After the initial review, in fitting with the scope of the project, approximately one third of items within each PAT mathematics assessment were significantly changed in one or more of the following ways:
•Items were contextualised to provide opportunities for diverse learners to see themselves, and life experiences that relate to their cultural and social worlds, reflected in the assessments. For example, updated contexts include waka hourua, kī-o-rahi, siapo patterns, solar energy in Tokelau, and Cook Islands drums.
•Principles and values such as mahi tahi (working together) and manaakitanga (kindness, generosity) were represented within some items. Examples included hāngi fundraisers, community volunteering, and caring for the environment.
•Home, community, and settings outside of school were prioritised. Although school is a shared context for learners, it is not a neutral space. Like assessment, schools themselves traditionally normalise New Zealand European/Pākehā culture, and reinforce New Zealand European/Pākehā cultural values and beliefs (Milne, 2017).
•New graphics were created to depict objects and people that are realistic and recognisable. There are a range of cultures, body sizes, genders, and several people with visible disabilities represented. Many of the people have Māori or Pacific names, and they become familiar as they appear across the assessments.
•Each PAT mathematics assessment begins with a kīwaha—Karawhiua!—encouraging students to give the assessment their best shot.
Additionally, we endeavoured to ensure the language of the assessments wasn’t a barrier to responding to the questions. We aimed for clear, concise wording of all the assessment questions, and avoided unnecessarily complex grammar and unfamiliar vocabulary.
The items then underwent multiple review cycles, a pilot in schools, and a larger scale trial.
Feedback from the pilot shows us we are heading in the right direction:
They were simple, quick, and not boring. I also liked how Māori terms were used ... The illustrations were fresh and excellent. One of the best things about these tests is that they heavily focus on real-world questions, which is excellent, as it makes it easier, and more relatable. (Student)
Alongside the assessment questions, the teacher guide is being rewritten with a corresponding focus on equity. We are aware that there are times when NZCER (and other) assessment tools are used in ways that do not necessarily benefit the learner, or uphold their mana. The teacher guide will communicate the significant message that PAT achievement data should be considered in the light of barriers in and outside of school which may have contributed to differential patterns of mathematics success. Teachers must explicitly reject deficit theories and consider the role of classroom teaching and learning, schools, and wider systemic factors, when analysing student data.
Travelling across uncharted waters
We are journeying over the horizon, towards opportunities to build an equitable assessment from the ground up. Solano-Flores and Nelson-Barber (2001) argue that “correcting for cultural bias, promoting the participation of ethnic minorities in pilot student samples, and providing accommodations for linguistic minorities are in the end simply remedial strategies that address cultural differences not considered in an assessment’s original plan” (p. 556). This leads us to ask, can we address cultural validity at the inception of a transformational mathematics assessment as a means of supporting equity for diverse learners?
Transforming assessment is complex, and requires a journey of change. While the current equity review has started the journey to make improvements to the cultural relevance of a number of PAT mathematics items, it is important for us to continue to explore ways to transform standardised mathematics assessments beyond the status quo.
For further information on upcoming changes to NZCER assessments, visit our website: www.nzcer.org.nz or email: assessmentservices@nzcer.org.nz
References
Hunia, M., Huria, J., & Spiller, L. (2020). Ready for partnership? A tool for creating written and visual texts in Aotearoa New Zealand. [Unpublished manuscript]. New Zealand Council for Educational Research.
Kerr, B., & Averill, R. (2021). Contextualising assessment within Aotearoa New Zealand: drawing from mātauranga Māori. AlterNative: An International Journal of Indigenous Peoples, 17(2), 236-245. https://doi.org/10.1177/11771801211016450
Mahuika, R., Berryman, M., & Bishop, R. (2011). Issues of culture and assessment in New Zealand education pertaining to Māori students. Assessment Matters, 3, 183-198. https://doi.org/10.18296/am.0093
Milne, A. (2017). Coloring in the white spaces: Reclaiming cultural identity in whitestream schools. Peter Lang. https://doi.org/10.3726/b10459
Montenegro, E., & Jankowski, N. A. (2017). Equity and assessment: Moving towards culturally responsive assessment. (Occasional Paper No. 29). University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).
Solano-Flores, G. & Nelson-Barber, S. (2001). On the cultural validity of science assessments. Journal of Research in Science Teaching, 38(5), 553–573. https://doi.org/10.1002/tea.1018
NEW ZEALAND COUNCIL FOR EDUCATIONAL RESEARCH
RANGAHAU MĀTAURANGA O AOTEAROA
Set: Research Information for Teachers
(incorporating set special)
GUIDELINES FOR WRITERS
Set: Research Information for Teachers is published by the New Zealand Council for Educational Research.
Available three times a year, each issue contains about nine articles based on quality educational research which reflect current practices and thinking. Set reaches most New Zealand schools and is used extensively for professional development over the full educational spectrum.
set also has a growing number of international subscribers.
GUIDELINES FOR CONTRIBUTORS TO SET
Referees. Set is a peer-reviewed journal. All articles are refereed by two academics or other recognised experts in the field. All refereeing is “blind”.
Editing. All articles are professionally edited.
Length. 4000 words maximum.
Content and style. Articles for Set should be based on quality educational research.
Our readers are primarily teachers and principals, but also include students, lecturers, parents and members of boards of trustees. We need articles that will attract and hold the attention of busy teachers. They may be based on one research project or on a review of several studies in one area. They should be easy to read, and relevant to the classroom. They should provide practical ideas and suggestions for further reading.
•Please keep the audience in mind—stress the implications of your research and its relevance for teachers. If possible, include suggestions about how the findings could be put to classroom use.
•The research procedures or methods must be visible, but not necessarily described in detail. Avoid complex statistics, but highlight the implications of them.
•Articles should be inclusive of Māori students wherever possible.
•If required, include one or two diagrams, or simple tables, charts or graphs, or a combination of all.
•Keep details for the Notes section.
•Articles may be submitted in English or te reo Māori.
Abstract. Include an abstract of about 100 words.
Key points. Include 4–6 brief bullet points outlining the key points of your article.
Acknowledgements. It's good practice to seek feedback on your manuscript from a colleague or academic supervisor before submission. Include the names of colleagues who have peer reviewed your article.
References. Use the APA 7th edition style for references. Please take care to ensure that all references are complete and accurate. For example:
Allington, R. L. (1980). Teacher interruption behaviors during primary-grade oral reading. Journal of Educational Psychology, 72(3), 71–77.
Alton-Lee, A. (2004, April). Improving educational policy and practice through an iterative best evidence synthesis programme. Paper prepared for The Evidence-Based Policy Research OECD-US seminar, Washington DC. http://www.excelgov.org/usermedia/images/uploads/PDFs/OECD-Alton.pdf
Fraser, D. (2004). Creativity: Shaking hands with tomorrow. In D. McAlpine & R. Moltzen (Eds.), Gifted and talented: New Zealand perspectives (2nd ed., pp. 145–169). ERDC Press, Massey University.
Glynn, T., Bethune, N., Crooks, T., Ballard, K., & Smith, J. (1989). Reading recovery in context. [Report to Research Division.] Ministry of Education.
Iversen, S. J. (1991). Phonological processing skills and the reading recovery programme [Doctoral thesis, Massey University].
Biographical details. Include a sentence or two about yourself and indicate your current position.
Copyright. Copyright belongs to individual authors, but is assigned to NZCER. It is expected that articles will be original and not published elsewhere. Authors will be given an agreement to sign.
Artwork. Please supply artwork (graphics and photos) either as hard-copy originals or electronically as TIFF, EPS, or PDF files in a high resolution (minimum 300dpi). We can also use high-quality JPEG digital photos (2–5 MB file size please). Artwork should be legible in black and white. Each graphic or photo should be supplied as a separate file. Please do not embed artwork in your article or place it in a Word document.
Copy. Please send double spaced electronic copy as an email attachment (saved in Microsoft Word format) to:
Josie Roberts
Email: josie.roberts@nzcer.org.nz
Contact details. Please attach your contact details:
•postal address
•phone
•email