You are here

Kei tua o te pae: Assessing learning that reaches beyond the self and beyond the horizon

Margaret Carr
Abstract: 

This paper outlines the development and the conceptual design of an assessment for learning resource of 177 exemplars from early childhood contexts. Two metaphors provide the foundation for the resource: te whāriki and kei tua o te pae. These metaphors underpin a discussion about assessment of learner outcomes that reach “beyond the self” and“beyond the horizon”, using the resource as an example. The resource that is built on these two metaphors includes three themes: principles that frame up the assessment;  practices that are based on narrative; and a view of progression that aligns with the principles. The assessment practice exemplified is sited in stories and portfolios, and the paper concludes with an argument for the consequential validity of narrative assessments.

Kei tua o te pae: Assessing learning that reaches beyond the self and beyond the horizon

Margaret Carr

Abstract

This paper outlines the development and the conceptual design of an assessment for learning resource of 177 exemplars from early childhood contexts. Two metaphors provide the foundation for the resource: te whāriki and kei tua o te pae. These metaphors underpin a discussion about assessment of learner outcomes that reach “beyond the self” and “beyond the horizon”, using the resource as an example. The resource that is built on these two metaphors includes three themes: principles that frame up the assessment; practices that are based on narrative; and a view of progression that aligns with the principles. The assessment practice exemplified is sited in stories and portfolios, and the paper concludes with an argument for the consequential validity of narrative assessments.

Introduction

The year 2009 marks the publication of the final five of 20 booklets on assessment for learning in the early years, prepared for the Ministry of Education and sent out to all early childhood centres and primary schools. The title of the resource is Kei Tua o te Pae/Assessment for Learning: Early Childhood Exemplars.1 This paper outlines the development and conceptual design of this resource, and builds an argument for the consequential validity of the narrative assessments that are a feature of these exemplars. The first section lays the foundation of the learning outcomes that are the basis for the assessments in Kei Tua o te Pae and sets out the two metaphors that have contributed to these outcomes. The second section introduces the resource and its development. This is followed by a discussion of three themes in the resource: principles, narrative practices and the construction of continuities and progressions. Finally, building on this discussion, the paper sets out a logic model of consequential validity for narrative assessments and Kei Tua o te Pae.

Two metaphors: Te whāriki and kei tua o te pae

This exemplar resource was developed from a Ministry of Education initiative in 2001 at the same time as contracts for assessment exemplars to support the school curriculum were initiated. It was proposed that assessment exemplars for early childhood be developed to support the 1996 national early childhood curriculum, Te Whāriki (Ministry of Education, 1996). In 2002 the initiative became part of the 10-year 2002–2012 Early Childhood Strategic Plan. A whāriki is a woven floor mat, and in the curriculum document this was adopted as a metaphor for curriculum and learning environments:

The early childhood curriculum has been envisaged as a whāriki, or mat, woven from the principles, strands, and goals defined in this document. The whāriki concept recognises the diversity of early childhood education in New Zealand. Different programmes, philosophies, structures and environments will contribute to the distinctive patterns of the whāriki. (Ministry of Education, 1996, p. 11)

The early childhood curriculum framework of principles and strands was developed in partnership with Māori consultants, and the core principles and strands are written in both English and Māori.2 These principles and strands also form the foundation for Kei Tua o te Pae. Book 3 of the resource discusses pathways to bicultural assessment practice, and an additional resource for assessment from a kaupapa Māori perspective will be completed and published in 2009. The concept of the curriculum as a woven whāriki also refers to the collective: a mat on which the community stands together for celebrations and sits together for conversations. It acknowledges that the curriculum is more than the self: it also references the role of the collective, the wider world of community and family. This wider world as an integral part of the early childhood curriculum is one of the four curriculum principles in the curriculum document.

Kei tua o te pae is the second metaphor. It is translated from Māori as “beyond the horizon” and comes from an oriori, a Māori lullaby, composed by Hirini Melbourne. Three lines in this oriori invite the baby to sleep well until a new day dawns with the rising of the sun beyond the horizon:

Ka tō te marama e tiaho nei

Ka hī ake ko te ra

Kei tua o te pae.

In the Kei Tua o te Pae resource, the line in this lullaby is adopted as a metaphor to mean that assessment for learning refers to learning well beyond the here and now, in not-yet-familiar contexts and supported by as-yet-unknown technologies. The contexts and technologies that our children will meet as they grow up are beyond the horizon: “In an ever-changing world, we know that young children’s horizons will expand and change in ways that cannot be foreseen. Children will travel beyond the current horizon, and early childhood is part of that” (Kei Tua o te Pae, Book 1, p. 5).

Therefore, the learning will include outcomes that are dispositional: learners who are ready, willing and able to engage with—and to construct and transform—learning opportunities.

So we were working with two metaphors for learning: reaching beyond the self and reaching beyond the horizon. Referring to “self-making narratives”, Jerome Bruner has commented:

A self-making narrative is something of a balancing act. It must, on the one hand, create a conviction of autonomy, that one has a will of one’s own, a certain freedom of choice, a degree of possibility. But it must also relate the self to a world of others—to friends and family, to institutions, to the past, to reference groups. But the commitment to others that is implicit in relating oneself to others of course limits our autonomy. We seem unable to live without both autonomy and commitment, and our lives strive to balance the two. So do the self-narratives we tell ourselves. (Bruner, 2002, p. 78)

Assessment practices are required to take on a similar balance. One side of the balance is a concern for individual development and the journeys of children and students towards competence and the recognition of possible learner selves. The other side of the balance is a commitment to a wider society, its values and its possible futures, and to others as reference groups who have an interest in what is going on.

Assessment of what? Learning outcomes as learning disposition, working theories and key competencies

A discussion of assessment exemplars and practices must be prefaced by a statement about the learning outcomes to be assessed. The early childhood sector had already begun to grapple with what assessment for learning may look like in terms of these two metaphors. Te Whāriki had introduced not only the collective but also learning outcomes as learning dispositions and working theories, an acknowledgement of the uncertainty of the environments that children will encounter as they grow up. It required the early childhood sector to explore ways to achieve progress towards the following vision for learners: “To grow up as competent and confident learners and communicators, healthy in mind, body, and spirit, secure in their sense of belonging and in the knowledge that they make a valued contribution to society” (Ministry of Education, 1996, p. 9).

This was a vision that combined five strands of outcome: exploration; communication; wellbeing; belonging; and contribution. Once again, this was a home-grown framework, developed from domains of mana (loosely translated as empowerment—another of the curriculum principles). In a sense the strands can be described as referencing five sources of strength or power: knowledge of the world; language; spiritual, mental and physical wellbeing; place; and people. Book 1 of Kei Tua o te Pae notes that there are particular dimensions to this when considering Māori educational advancement and includes Mason Durie’s commentary on three goals for Māori educational advancement: to live as Māori; to participate as citizens of the world; and to enjoy good health and a high standard of living (Durie, 2001). In later (2003) comments, included in Book 3 (p. 3), and eloquently describing some requirements for an assessment that reaches beyond the individual, Durie has added his view that:

The essential difference [between Māori and other New Zealanders] is that Māori live at the interface between te ao Māori (the Māori world) and the wider global society (te ao whānui). … As a consequence, educational policy, or teaching practice, or assessment of students, or key performance indicators for staff must be able to demonstrate that the reality of the wider educational system is able to match the reality in which children and students live. (Durie, 2003 pp. 5–6)

Ten years later the school sector has begun to grapple with new discussions about outcomes, confronted by a new school curriculum which includes key competencies—outcomes that include knowledge, attitudes and values—that parallel the early childhood curriculum strands (Ministry of Education, 2007, p. 42) and will be woven into the learning areas:

More complex than skills, the competencies draw also on knowledge, attitudes and values in ways that lead to action. … The competencies continue to develop over time, shaped by interactions with people, places, ideas and things. … Opportunities to develop the key competencies occur in social contexts. People adopt and adapt practices that they see used and valued by those closest to them, and they make these practices part of their own identity and expertise. (Ministry of Education, 2007, p. 12)

Key competencies are also dispositional: ability is supplemented by motivation and attunement to circumstance, and the learner is described as being ready and willing as well as able.3 They reach beyond the horizon. They are also closely tethered to social context and learning opportunities: they reach beyond the self.

Learning dispositions as outcomes can be described as situated learning strategies plus motivation (Carr, 2001, p. 9), and a section on the ways in which assessment will protect and enhance the motivation to learn in Book 1 quotes from a keynote address that Terry Crooks presented at the New Zealand Association for Research in Education (NZARE) conference in Palmerston North in 2002. He spoke about some requirements for effective learning, and emphasised motivation:

First, people gain motivation and are most likely to be learning effectively when they experience success or progress on something they regard as worthwhile and significantly challenging. … My second point about motivation is that personal desire to learn something is an incredibly powerful force, often able to carry learners through repeated disappointments and difficulties. … My final point about motivation is the importance of how students interpret their success or failure. It matters whether they attribute successes to ability, effort, or good luck or attribute failures to lack of effort, lack of ability, or bad luck.

He also quoted words from Mihaly Csikszentmihalyi (1990, p. 115):

The chief impediments to learning are not cognitive. It is not that students cannot learn; it is that they do not wish to. If educators invested a fraction of the energy they now spend on trying to transmit information, in trying to stimulate the students’ enjoyment in learning, we could achieve much better results.

Both of our curriculum documents—early childhood and school—affirm that these outcomes matter. They are about orientations towards learning challenges, responsibilities and enthusiasms, and they contribute to growing learner identities. The literature that supports them is now vast, and although I hesitate to cite just a few sources, that literature forms the backbone of the arguments and exemplars in Kei Tua o te Pae.

Supporting the notion of outcomes that reach beyond the horizon, Margaret Donaldson argues that education “is about suggesting new directions in which lives may go” (1992, p. 259); Hazel Markus and Patricia Nurius (1986) have set out the case for possible selves, and Gunter Kress (2003) writes of the technological and communication revolution that makes reaching beyond the horizon imperative. Supporting the notion of outcomes that reach beyond the self, the sociocultural literature includes Barbara Rogoff (2003), who insists on the cultural nature of human development; Etienne Wenger (1998) analyses education in terms of communities of practice; James Gee (2003, 2008) integrates the notion of “opportunity to learn” with discussions of learning; James Greeno (1991) introduces us to the metaphor of a learning environment as a kitchen; Luis Moll and colleagues (2005) have researched “funds of knowledge” from home; Margie Hohepa and Stuart McNaughton (2007) elaborate on the viewpoint that literacy knowledge is culturally constructed; and Allan Luke and Peter Freebody (1999) site literacy in a map of practices. The work of Rose Pere (1997) and of Bishop, Berryman, Takiwai, and Richardson (2003) continues to remind us of the central role of relationships in education and assessment.

Other papers in this first edition of Assessment Matters consider some of the implications for assessment in schools of the new New Zealand curriculum. This paper outlines three aspects of the Kei Tua o te Pae conceptual story: the alignment with assessment principles in Te Whāriki; the use of narrative as an assessment tool; and some emerging ideas about continuity and progression. These ideas are then summarised as a logic model for the consequential validity of narrative assessment. But first, the resource is introduced.

Kei Tua o te Pae: The resource development

The booklets in Kei Tua o te Pae/Assessment for Learning: Early Childhood Exemplars were published in three series. Each series includes an introduction (Books 1, 10 and 16) that introduces the reader to key theoretical ideas and research related to the series focus. Series one, Books 1 to 9, focuses on issues of assessment and was published in 2004. The issues covered were: sociocultural assessment; bicultural assessment; children contributing to their own assessment; assessment and learning (community, competence and continuity); assessment for infants and toddlers; and inclusive assessment. Series two, Books 10 to 15, focuses on the five strands of curriculum outcome in Te Whāriki and was published in 2007. Series three, Books 16 to 20, focuses on symbol systems and technologies for making meaning (oral, visual and written literacy, mathematics, the arts and ICT) and was published in 2009. Apart from the introductions, the exemplar books average 28 pages each, with between seven and 15 exemplars in each book. The resource was distributed to all early childhood centres and services, all early childhood professional development providers and all primary schools.

Altogether the resource contains 177 exemplars. We invited teachers (through the Education Gazette) and project co-ordinators to submit actual assessment examples from children’s portfolios (with attached permissions), and we worked on the assumption that exemplars are not exemplary: they should invite and provoke reflection. In a paper written with Bronwen Cowie as a position paper for the first series of exemplars, and later published, we said that while assessments must have consequences for learners, they should have some of the same consequences for teachers:

It is our view that if governments develop assessment formats or exemplars, then the consequences of these policies or recommended practices should play out for the profession in the same three ways as outlined in this chapter [with consequences for community, competence and continuity]. First they must act as a conscription device for participants, establishing a social community of early childhood teachers who want to talk about learning and assessment. Second, they must be seen as permeable: providing social spaces for new ideas about assessment and its relationship with learning so that teachers develop their identities as learners and assessors. Third, they must invite teachers to think about their own learning pathways around assessment: to set up dialogue opportunities within their own settings as they adapt formats and try out new ideas, in order to develop local assessments that afford community, competence and continuity. (Cowie & Carr, 2004, p. 106)

These ideas about consequences were elaborated in three books of Kei Tua o te Pae: Community (Book 5), Competence (Book 6) and Continuity (Book 7). I return to them in the final section of this paper.

Professional development support for the resource

The Ministry of Education began funding professional development contracts to support Kei Tua o te Pae in 2005, in addition to continuing professional development contracts that support the implementation of Te Whāriki. Wrapping professional development around resources that introduce alternative ways of being a teacher and being a learner—as both of these resources do—recognises the powerful combination of resources plus professional support for setting up and strengthening sustainable structures, teacher beliefs and opportunities for reflection. The importance of this support was confirmed in a report evaluating progress from 2004 to 2006 on the Early Childhood Strategic Plan: Pathways to the Future: Ngā Huarahi Arataki (Ministry of Education, 2002). The authors surveyed a sample of 46 individual settings of different early childhood education service types and asked participants in 2006 about their use of Kei Tua o te Pae. During this period, of the 18 services that had improved their ratings for assessment practices, all but one had used Kei Tua o te Pae and “most had professional development associated with it” (Mitchell & Hodgen, 2008, p. 6).

In a 2007 New Zealand Council for Educational Research (NZCER) national survey of early childhood education services looking at “assessment practices and aspects of curriculum in early childhood”, the findings from 401 teachers/educators included the following:

Professional learning was regarded as beneficial, with most teachers/educators agreeing or strongly agreeing that it helped them try out new practices, offer new ways to engage children, or identify areas to develop. Somewhat fewer teacher/educators (but nevertheless over half) agreed that professional learning challenged their assumptions or beliefs, offered them new ways to use data to plan teaching, or offered new ways to use data to give feedback and elicit feedback from families. (Mitchell, 2008, p. 24)

The Ministry of Education commissioned an impact evaluation of the Kei Tua o te Pae professional development in 2006. This is a lengthy and detailed report from a survey of 19 services (settings or centres), case studies and interviews from 18 services, and telephone interviews with people at seven tertiary-level organisations (who also received Kei Tua o te Pae professional development). The executive summary reports that:

Services [settings or centres] reported substantial and sustained shifts in the quality of assessment practices over the time period of the professional development and beyond. … The reported influence of the professional development on these shifts was generally high and the professional development was positively regarded by a majority of services. … There is evidence from the evaluation that the 2006 professional development had strengthened sociocultural assessment practices in these services. Services had taken significant steps in building an assessment community of practice inclusive of educators, children and parents. (Stuart, Aitken, & Gould, 2008, pp. i–ii)

The authors concluded that, overall, “the Kei Tua o te Pae resource itself has become a touchstone reference for assessment practice and assessment policy development in services” (p. 52).

Four assessment principles

Three themes characterise Kei Tua o te Pae as an assessment resource. The first of these is the guiding principles. Book 2, entitled Sociocultural Assessment, responded to Te Whāriki’s invitation to teachers to use the four curriculum principles as principles of assessment as well (Ministry of Education, 1996, p. 30). There are four principles: empowerment; holistic development; family and community; and relationships. The following includes the suggestions in Te Whāriki, and the response made in Book 2.

Principle one

Empowerment (whakamana) for assessment practice includes feedback to children on their learning and development that enhances their sense of themselves as capable people and competent learners, and the development of children’s capacities to assess their own learning. Book 2 (p. 6) suggests what to look for:

Assessments that refer to children setting their own goals

Children developing their own criteria for assessing achievement

Teachers’ criteria for assessment that are transparent and accessible (and that may be negotiated by older children)

Children being consulted about what they will do next

Children being consulted about what will be recorded or collected.

Principle two

Holistic development (kotahitanga) for assessment practice requires assessment and observations to take place in the same contexts of meaningful activities and relationships that have provided the context for the curriculum. Book 2 (p. 15) suggests what to look for:

The integration of children’s physical, intellectual, emotional, social and spiritual development … [as] working theories and learning dispositions

Assessments that connect what is being learned to meaningful situations and purposes

Multiple perspectives that enhance the interpretation and analysis of the learning

Assessments that recognise that learning is multidimensional.

Principle three

Family and community (whānau tangata) for assessment practice insists that families should be part of the assessment and evaluation of the curriculum as well as of children’s learning and development. The comment is made that “parental understandings and expectations will alter children’s expectations of themselves”. Book 2 (p. 24) suggests what to look for:

Assessment practices that are accessible to families and whānau (extended family)

Assessments that invite families and whānau into the curriculum

Families and whānau participating in assessment

Assessments that encompass participation by the teachers and children in community activities outside the early childhood setting.

Principle four

Relationships (ngā hononga) for assessment practice emphasises the role of relationships in the assessment process, and comments on the influence of teachers’ expectations. Book 2 (p. 32) suggests what to look for:

Assessments that are conversations between learner, peers, teachers, and families about the learning in progress

Adult assistance and teaching as part of assessments

Peer assistance and teaching as part of assessments

Documented assessments that reflect opportunities to learn what is valued (the roles of people, places and things)

Documented assessments that build on and construct informal everyday interactions and vice versa.

These four principles set the scene for all 20 books in Kei Tua o te Pae.

Assessment practices

The second theme in Kei Tua o te Pae is the format of the assessment practice. The introductory books (Books 1, 10 and 16) elaborate on this approach to curriculum and assessment in various ways. Book 1 sets out some of the features of assessment for learning. It includes “noticing, recognising and responding”, documented and undocumented assessments, everyday contexts, listening to children, keeping a view of learning as complex, collective as well as personalised assessments and clear goals. It aims to protect and enhance the motivation to learn and to acknowledge uncertainty.

By Book 10 the definition of assessment for learning practices has extended from noticing, recognising and responding (p. 6, Book 1), to noticing, recognising, responding, recording and revisiting (p. 8, Book 10). Book 10 refers to the research findings on formative assessment summarised by Paul Black and Dylan Wiliam (Black, Harrison, Lee, Marshall, & Wiliam, 2002, 2003; Black & Wiliam, 1998a, 1998b). Five of the key features (assessment will be sited in meaningful and interesting tasks, learners will be actively involved, there will be a culture of success, there will be opportunities for all learners to express their ideas, and self-assessment) are aligned with the five strands of curriculum outcome in Te Whāriki.

The use of narrative as an assessment tool

In Kei Tua o te Pae all of these assessment discussions are accompanied by “real-life” examples from current practice in New Zealand early childhood settings. By 2005 there was widespread interest in the use of narrative assessment formats in the early childhood sector in this country. This interest followed a project to research assessment in five different service types (a kindergarten, an education and care centre, a kōhanga reo, a play centre and a home-based setting) consistent with the principles and strands of Te Whāriki, and a three-video resource with an accompanying booklet which followed that research (Carr, 1998a, 1998b). During the early 21st century there had been an explosion of interest in the use of digital technology to document children’s learning (Lee, Hatherly, & Ramsey 2002; Ramsey, Breen, Sturm, Lee, & Carr, 2007). In 1998 the research project on assessment documentation used Polaroid cameras, and we were excited by the instant quality of the photographs. From 2001, when we asked teachers to send in examples of their assessments, digital photography was frequently included and we began to loan digital cameras to co-ordinators and centres to increase the quality of the photographs for publication. By the time the third series was published, the exemplars included children putting an iSight© camera into the mouse house, preparing PowerPoint stories and making digital movies.

Learning stories and narratives became the mode of assessment in Kei Tua o te Pae, a mode that responded to the theoretical call for assessments that reach beyond the self and beyond the horizon of the here and now. Vivian Gussin Paley, who taught us the powerful role of story in teaching, is referenced throughout (Paley, 1988, 2001). The literature on narrative assessments includes teachers’ journeys of assessment exploration using learning stories (Carr, Hatherly, Lee, & Ramsey, 2003; Hatherly & Sands, 2002; Lee et al., 2002), and some of the emerging work in schools (Carr et al., 2008). In a 2007 paper written for the Ministry on “Assessing Key Competencies”, Rose Hipkins writes about the way in which key competencies have introduced new dimensions to learning outcomes and environments: “meta” knowing, fostering a disposition to learn, empowering students to become experts on their own learning and rich learning contexts. She adds: “Over time, students develop personal stories about themselves as learners. Assessment needs to help them build coherent narratives about their identities as people who can practise, persist, and overcome obstacles to immediate learning success” (p. 5). She suggests that as schools explore ways to teach dispositional outcomes, they will need to rethink familiar assessment strategies, and consider newer assessment strategies such as learning logs or journals, learning stories, portfolios and rich tasks (first designed as part of the New Basics curriculum initiative in Queensland, Australia).

Discussions about narrative assessment have some kinship with the growing literature on narrative inquiry as a research methodology. Chapter one in a 2007 Handbook of Narrative Inquiry sets out what the authors describe as four historical “turns to narrative”, each of which represents a philosophical turn away from four assumptions about knowledge and knowing. The first of these is an assumption about reliability: “When social facts, like rocks, can be treated as ‘thing-like’, then researchers can measure them and [reliably] number them” (Pinnegar & Daynes, 2007, p. 29). But early childhood centres and school classrooms can better be described as “wild”, complex and diverse environments that “present a wilderness of vaguely marked and ill-defined occasions for thoughtful engagement” (Perkins, Tishman, Ritchhart, Donis, & Andrade, 2000, p. 270). The second assumption is objectivity: “What fundamentally distinguishes the narrative turn from ‘scientific’ objectivity is understanding that knowing other people and their interactions is always a relational process that ultimately involves caring for, curiosity, interest, passion, and change” (Pinnegar & Daynes, 2007, p. 29). The third assumption is generalisability: a narrative values the local and the particular, although elsewhere I have argued for the value of combining both a local and particular lens with a wider, more generalised lens as a process of “zooming in and zooming out” (Carr, 2008b). The final assumption that narrative inquirers interrupt is a positivistic stance on the notion of validity: the notion that observations discover the “truth”. Narratives acknowledge the role of interpretation in all inquiry, especially when the topic of interest is education and learning.

Multiple voices

One consequence of acknowledging the interpreted nature of all assessments is an educational environment that provides opportunities for multiple perspectives, including self-assessment by learners. The executive summary of the impact evaluation of the Kei Tua o te Pae professional development commented: “Children’s active engagement in the assessment process was strongly evidenced in interviews, including child voice in assessment documentation, child reflection on and analysis of assessment narratives, and the co-construction of next steps with educators” (Stuart et al., 2008, p. ii). Book 4 provides theoretical and research-based rationale, and includes 13 exemplars, for discussion of the questions “How can children contribute to their own assessments?” and “Why should they?”

Research tells us that families’ engagement with their children’s education is a powerful mediator of interpretation and support (Brooker, 2002; González, Moll, & Amanti, 2005; Pollard & Filer, 1999; Siraj-Blatchford, 2004), and Carolyn Jones wrote a paper on this during the project (Jones, 2006). Many of the exemplars in Kei Tua o te Pae illustrate the ready accessibility of the assessment portfolios for children and families. A “parent’s voice” in Book 2 reads:

Damien loves to ‘read’ his portfolio. He is so enthusiastic in searching out the stories he loves the most that I have to hide away on my own to read the stories carefully and thoroughly. That way I can make sure that I am not constantly interrupted and asked to look at the next one. When we look at it together, he turns the pages over and over until he gets to his favourite story about the dinosaurs T Rex and Long Neck, and he says, ‘Those are the exact words I said, Mum! That’s exactly what I said!’ (Kei Tua o te Pae, Book 2, p. 7)

Five weeks after he left for school, Damien’s mother commented to his early childhood teacher that his portfolio is still one of his most loved books. The impact evaluation confirmed that early childhood teachers see family engagement as having a role in assessment practices: 40 of 47 interviewed teachers affirmed that parents contributed to assessments and 32 of them said that the parental contribution was used by the teachers (Stuart et al., 2008, p. 84). The evaluators analysed 967 assessment items and reported that 11 percent of them contained extensive evidence of the visibility and value of the child’s family context (and a further 16 percent contained “slight” evidence) (p. 86). Book 5 of Kei Tua o te Pae discusses ways in which the participation of the family and wider community can be represented and reflected in assessments.

The portfolio

Families add comments and stories to the portfolios, as Damien’s mother did, and sometimes these family contributions are in the home language, so the portfolio becomes a “boundary object” that plays “different roles in different situations” (Lemke, 2000, p. 281), connecting the early childhood centre or classroom with home and acting as a “conscription device” (Cowie & Carr, 2009, p. 106) that invites families to participate. Children sometimes dictate a learning story (or, at school, write their own), and, for Damien, the portfolio appeared to be a literacy artefact that he could “read”. Portfolios include individual stories, group or collective stories, personalised group stories and centre stories. But the core elements of a useful learning story are: an episode of learning described by an observer or participant who knows the child and recognises that this is a story worth telling; a clear analysis or commentary highlighting the learning; and, usually, some suggestions about the possibilities for further action or challenge.

As part of a Teaching and Learning Research Initiative project (Carr et al., 2008), portfolios or folders in Yvonne Smith’s school classroom included the children’s reading and writing and mathematics progress, along with learning stories that illustrate their progress with the key competencies. Yvonne began to combine the two, writing a group story about a lesson and personalising it for a child with comments about progress in the subject area, progress in key competencies and photographs. She had commented in a working paper she wrote for this project that she wanted to “explore how the draft Key Competencies could be integrated into the daily programme, and assessed, without creating extra workload for teachers already struggling with an overloaded curriculum”. She retained the same class from new entrants in 2006 to Year 1 in 2007, and for 18 children we analysed the learning stories from when they began (some time in 2006) to November 2007. The portfolios for these children included group stories, personalised group stories and individual stories. Yvonne’s individual stories, true to the dispositional nature of the key competencies, often recorded an event when the learner chose their activity. For instance, an individual story was written about Abby, who initiated a role-playing game in which she was a librarian, issuing books to a small group of willing participants. Yvonne writes: “This is the first time Abby has instigated an activity and taken a lead role.” A self-comment by Abby soon after this reads, “I was shy when I started school … [now] I put my hand up.” In a personalised group story in which the class were invited to see what they could find out about clouds (“Budding Researchers”), there is a photo of Abby presenting to the class, with the caption “I did a Google search”.

We were becoming aware of the social justice and identity implications of story. The analysis of portfolios, for instance, revealed that Lily and Amy were not high achievers at writing, and their portfolios include work samples and teacher commentary that indicated this. However, their learning stories also reveal other aspects of their learning: leadership activities and activities of “high focus” for Amy; Lily’s “empathy for others”, her “great memory” for dances and her disposition to teach by example.

Emerging ideas about continuity and progression

The third theme of Kei Tua o te Pae has been the emerging ideas about continuity and progression. Linda Mitchell’s survey of early childhood assessment practices in 2007 notes that 97 percent of the 401 teachers surveyed used assessment portfolios, and 84 percent indicated that children revisit them, with or without the teacher (Mitchell, 2008, p. 15). In this way, the portfolio discussions can construct continuity: looking back, and looking forward. In a current research project in which teachers are recording their revisiting, the following transcript was shared. The teacher and Isabella have been looking at Isabella’s portfolio, and discussing the mosaic tile she has made:



T:

[points to a photo] Did you make this for anyone special?

I:

[looks at teacher, smiling] Um, I made it for my sister! [pauses and then nods] I have to make one for my Mummy [counting on her hand] and my Daddy as well.

T:

Do you?

I:

I’ve got two things to make. [holds up both hands with two fingers pointing up on each hand]

T:

Two things. Is that your plan?

I:

Yep.

T:

That’s a good plan. [Isabella smiles and recording ends]

Our ideas about continuity and progression of dispositional learning shifted over the three series of Kei Tua o te Pae. Book 7, Continuity, was published in the 2004 series. This book points out that cultural views of continuity take it far back in time:

The child was, and still is, the incarnation of the ancestors: te tanohi ora, ‘the living face’. The child was, and still is, the living link with yesterday and the bridge to tomorrow: te taura here tangata, ‘the binding rope that ties people together over time’. The child is the kāwa tangata, the ‘genealogical link’ that strengthens whanaungatanga, ‘family relationships’, of that time and place. (Reedy, 1995/2003, p. 58)

The Continuity book (Book 7) is 40 pages long because the exemplars include case studies of assessments over time. For instance, 10 of Fe'ao’s learning stories are included in this book, a selection from portfolio entries over 15 months, annotated to describe his progress in terms of the increasing complexity of his relationships, literacy and interests. His family have added their comments too, making connections between home and the work that Fe‘ao is doing at the centre. In the first learning story that includes an interaction with a teacher, the teacher takes much of the initiative. The final story “covers his work over several days in which he makes most of the decisions and dictates the text to the teacher”. One of the parent comments begins: “When Fe‘ao arrived home with his school folder he wanted to show everyone what (was) inside the folder. He explain[ed] what he was doing in the photos.” The text in Book 7 analyses continuity in terms of:

• looking back and developing forward in learning stories and portfolios

• continuity and change in the learning community, documenting the increasing connections “beyond the walls” of the early childhood setting (Fe‘ao’s family’s participation in the portfolio was one example)

• continuity in terms of competence becoming more secure, more widely applicable and more complex.

These domains of continuity and progression—time, community and complexity—were elaborated in the introduction to the 2007 series, by introducing four “dimensions of strength”:

• mindfulness and flexible power balances

• connection to a diversity of social communities

• frequency and regular events

• distribution across helpful people and enabling resources.

A lengthy exemplar in Book 15—three children developing sewing projects together over a number of months—was used to illustrate each of these dimensions. The dimensions of strength are repeated as a reminder in Book 16, which is the introduction to the third series (p. 6).

As we considered the learning over time in the exemplars and in the research project where Yvonne’s work was analysed, the dimensions of strength came to be dubbed the ABCD of strength, their definitions were sharpened and their value for schools was explored:

agency is about authoring and responsibility, and includes taking the initiative and asking questions (see also Carr, 2008a, for a discussion of agency and resourcefulness as outcomes for education, and the role of assessment in this)

breadth refers to a widening of connections to social practices and communities outside the early childhood or school setting

continuity refers to the increasing recognition by the learner of connections between the past, the present and the possible future

distribution refers to the learning being “stretched over” a diversity of resources and the increasing capacity of learners to recognise and call on useful resources (including people) for particular goals.

These dimensions remain tethered to topic, domain or interest, but they acknowledge the multifaceted and complex nature of learning journeys when motivation and attunement are added to ability.4 Carr et al. (in press) elaborate further on what we have called transactional and progressive processes of “learning in the making”: authoring (agency); connected knowing (breadth and continuity); and recognition of opportunity (distribution).

Consequential validity

Being involved with the development of Kei Tua o te Pae has been a great privilege, and I have learnt much from my colleagues, and from the records of learning journeys that have been so professionally, carefully and often joyfully recorded in annotated photographs, learning stories and portfolios by teachers, children and families. This account of Kei Tua o te Pae’s conceptual journey from 2004 to 2009 has stimulated me to think more about the consequential validity of narrative assessment, and to extend the conversation about consequences as community continuity and competence that began with Bronwen Cowie in 2004. Inspired by a “logic model for a causal argument” in a discussion about the likely paths to pupils’ academic achievement in a Learning How to Learn project (James, Black, McCormick, & Pedder, 2007, p. 14), I suggest that the argument and evidence in the 20 books in Kei Tua o te Pae, together with the background analysis in this paper, can be described in a logic model for the consequences of narrative assessment, a statement of consequential validity. It is included here as Figure 1.

Image

A recent document prepared for the Ministry of Education entitled Directions for Assessment in New Zealand: Developing Students’ Assessment Capabilities (Absolum, Flockton, Hattie, Hipkins, & Reid, 2009) includes a section on “Valid Interpretations and Decisions” (p. 33). The authors say:

When we use the term valid, we mean that the descriptive (scores, levels, observations etc.) and prescriptive (what to do next) interpretations and inferences made are defensible in their consequences. It is not the test, test score, or observation that is validated so much as the decisions and actions that flow from the text, score, or observation. Validity is a function of both parts of the decision-making process: if the descriptive part is good but the prescriptive part is poor (in other words, if the student’s performance is correctly determined but consequential decisions are detrimental to learning) or vice versa, the assessment lacks validity. … Our concern is the extent to which the accumulated evidence supports a particular interpretation or decision. Ultimately, a lot depends on informed professional judgement, so the more effort we put into strengthening the teachers’ assessment capabilities, and the greater the effort that teachers put into strengthening students’ assessment capabilities, the more we can expect that interpretations and consequential actions will be valid.

Figure 1 sets out a logic model of consequential validity from the discussion in this paper, a later elaboration of the argument in the Cowie and Carr (2004) chapter. In this paper, the development of a learning community and the construction of continuity are on their way to competence and growing learner identities. They are, of course, also important consequences in their own right. The logic model sets out four consequences from narrative documentation:

• revisiting and reflecting in a wide learning community

• further action

• construction of continuity and progression

• competence and growing learner identities.

I provide examples of all of these. A wide learning community is authoring these consequences: the teachers, the learners and the families.

Revisiting and reflecting

Assessments are documented in narrative assessments or learning stories and are revisited in a wide learning community. Early childhood teachers have had professional development opportunities to revisit and reflect on their learning and teaching through discussions about documentation, and teachers have also reflected on their learning stories at staff meetings or in informal conversations. The narratives invited families to become engaged and interested, and to revisit the stories and photographs of learning episodes at home with their children and other siblings and relations.

Further action

The narrative assessments include suggestions about possible pathways: what children and peers and teachers and resources might do next to progress the learning. There is therefore further action from the previous learning episodes; teachers comment on this, and they may write further stories. Many families have added comments to the stories, and have sometimes contributed stories and photographs from home.

Construction of continuities and progressions

Portfolios—collections of assessments that will also include children’s work—construct continuities. The ideas about progression during the development of Kei Tua o te Pae are a work in progress, and professional development providers and teachers using the 2009 books are continuing these discussions. Families often interpret for and with the children, making connections with other events at home or in the community.

Competence and growing learner identities

The final consequence in the logic model is a strengthening of competence and growing learner identities. Interacting across early childhood education settings and homes, the learners are in the middle: collaboratively constructing and recognising narratives about their identities as learners, growing up as, in the words of Te Whāriki: “competent and confident learners and communicators, healthy in mind, body and spirit, secure in their sense of belonging and in the knowledge that they make a valued contribution to society” (Ministry of Education, 1996, p. 9).

References

Absolum, M., Flockton, L., Hattie, J., Hipkins, R., & Reid, I. (2009). Directions for assessment in New Zealand: Developing students’ assessment capabilities.

Bishop, R., Berryman, M., Takiwai, S., & Richardson, C. (2003). Te kotahitanga: The experiences of year 9 and 10 Māori students in mainstream classrooms. Wellington: Ministry of Education.

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2002). Assessment for learning: Putting it into practice. Buckinghamshire: Open University Press.

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. Buckinghamshire: Open University Press.

Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education, 5(1), 7–74.

Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. London: School of Education, Kings College.

Brooker, L. (2002). Starting school: Young children learning cultures. Buckinghamshire: Open University Press.

Bruner, J. (2002). Making stories: Law, literature and life. Cambridge, MA: Harvard University Press.

Carr, M. (1998a). Assessing children’s experiences and five case studies (Vols. 1 & 2). Final report to the Ministry of Education. Wellington: Ministry of Education.

Carr, M. (1998b). Assessing children’s learning in early childhood settings. A set of three videos and accompanying book. Wellington: New Zealand Council for Educational Research.

Carr, M. (2001) Assessment in early childhood settings: Learning stories. London: Sage.

Carr, M. (2008a). Can assessment unlock and open the doors to resourcefulness and agency? In S. Swaffield (Ed.), Unlocking assessment (pp. 36–54). London: Routledge.

Carr, M. (2008b). Zooming in and zooming out: Challenges and choices in discussions about making progress. In J. Morton (Ed.), Making progress—measuring progress. Conference proceedings (pp. 3–18). Wellington: NZCER Press.

Carr, M., Hatherly, A., Lee, W., & Ramsey, K. (2003). Te whāriki and assessment: A case study of teacher change. In J. Nuttall (Ed.), Weaving te whāriki (pp. 187–214). Wellington: New Zealand Council for Educational Research.

Carr, M., Lee, W., & Jones, C. (2004, 2007, & 2009). Kei tua o te pae/Assessment for learning: Early childhood exemplars (Books 1–20). A resource prepared for the Ministry of Education. Wellington: Learning Media.

Carr, M., Peters, S., Davis, K., Bartlett, C., Bashford, N., Berry, P., et al. (2008). Key learning competencies across place and time: Kimihia te ara tōtika, hei oranga mō to aō. Final report to the Teaching and Learning Research Initiative programme. Available at www.tlri.org.nz.

Carr, M., Smith, A. B., Duncan, J., Jones, C., Lee, W., & Marshall, K. (in press). Learning in the making: Disposition and design in early education. Rotterdam: Sense Publications.

Cowie, B., & Carr, M. (2004). The consequences of sociocultural assessment. In A. Anning, J. Cullen, & M. Fleer (Eds.), Early childhood education: Society and culture (pp. 95–106). London: Sage.

Cowie, B., & Carr, M. (2009). The consequences of sociocultural assessment. In A. Anning, J. Cullen, & M. Fleer (Eds.), Early childhood education: Society and culture (2nd ed. pp. 105–116). London: Sage.

Crooks, T. (2002, December). Assessment, accountability and achievement: Principals, possibilities and pitfalls. Keynote address presented at the 24th annual conference of the New Zealand Association of Research in Education, Palmerston North.

Csikszentmihalyi, M. (1990). Literacy and intrinsic motivation. Daedalus, Spring, 115–140.

Donaldson, M. (1992). Human minds: An exploration. London: Penguin.

Durie, M. (2001, February). A framework for considering Māori educational advancement. Opening address to the Hui Taumata Mātauranga, Tūrangi/Taupō.

Durie, M. (2003, March). Māori educational advancement at the interface between te ao Māori and te ao Whānui. Paper presented at the Hui Taumata Mātauranga, Tuatoru, Tūwharetoa.

Freebody, P., & Luke, A. (2003). Literacy as engaging with new forms of life: The “four roles” model. In G. Bull & M. Anstey (Eds.), The literacy lexicon (2nd ed., pp. 52–65). Frenchs Forest, NSW: Prentice Hall.

Gee, J. P. (2003). Opportunity to learn: A language-based perspective on assessment. Assessment in Education, 10(1), 27–46.

Gee, J. P. (2008). A sociocultural perspective on opportunity to learn. In P. A. Moss, D. C. Pullin, J. G. Gee, E. H. Haertel, & L. Jones Young (Eds.), Assessment, equity, and opportunity to learn (pp. 76–108). Cambridge: Cambridge University Press.

González, N., Moll, L., & Amanti, C. (2005). Funds of knowledge: Theorizing practices in households, communities, and classrooms. Mahwah, NJ: Lawrence Erlbaum.

Greeno, J. (1991). Number sense as situated knowing in a conceptual domain. Journal for Research in Mathematics Education, 22(1), 170–218.

Greeno, J. G., & the Middle School Mathematics Through Applications Project Group. (1998). The situativity of knowing, learning and research. American Psychologist, January, 5–26.

Hatherly, A., & Sands, L. (2002). So what is different about learning stories? The First Years: Ngā Tau Tuatahi, 4(1), 8–12.

Hipkins, R. (2007). Assessing key competencies: Why would we? How could we? Wellington: Ministry of Education.

Hohepa, M., & McNaughton, S. (2007). Doing it proper: The case of Māori literacy. In L. Makin, C. Jones Dias, & C. McLachlan (Eds.), Literacies in early childhood: Changing views, challenging practice (2nd ed., pp. 217–229). Marrickville, NSW: Elsevier, Australia.

James, M., Black, P., McCormick, R., & Pedder, D. (2007). Promoting learning how to learn through assessment for learning. In M. James, R. McCormick, P. Black, P. Carmichael, M-J. Drummnd, A. Fox et al. (Eds.), Improving learning how to learn: Classrooms, schools and networks (pp. 3–29). London: Routledge.

Jones, C. (2006). Continuity of learning: Adding funds of knowledge from the home environment. set: Research Information for Teachers, 2, 28–31.

Kress, G. (2003). Literacy in the new media age. London: Routledge.

Lee, W., Hatherly, A., & Ramsey, K. (2002). Using ICT to document children’s learning. Early Childhood Folio, 6, 10.

Lemke, J. (2000). Across the scales of time: Artifacts, activities, and meanings in ecosocial systems. Mind, Culture, and Activity, 7(4), 273–290.

Luke, A., & Freebody, P. (1999). A map of possible practices: Further notes on the four resources model. Practically Primary, 4(2), 5–8.

Markus, H., & Nurius, P. (1986). Possible selves. American Psychologist, 41(9), 954–969.

Ministry of Education. (1996). Te whāriki: He whāriki mātauranga mā ngā mokopuna o Aotearoa: Early childhood curriculum. Wellington: Learning Media.

Ministry of Education. (2002). Pathways to the future: Ngā huarahi arataki. Wellington: Learning Media.

Ministry of Education. (2007). The New Zealand curriculum: The English-medium teaching and learning in years 113. Wellington: Learning Media.

Mitchell, L. (2008). Assessment practices and aspects of curriculum in early childhood education. Wellington: New Zealand Council for Educational Research.

Mitchell, L., & Hodgen, E. (2008). Locality-based evaluation of pathways to the futurengā huarahi arataki (Stage 1 report). Wellington: Ministry of Education.

Paley, V. (1988). Bad guys don’t have birthdays: Fantasy play at four. Cambridge, MA: Harvard University Press.

Paley, V. (2001). In Mrs Tully’s room: A childcare portrait. Cambridge, MA: Harvard University Press.

Pere, R. (1997). Ako: Concepts and learning in Māori tradition. Wellington: National Te Kōhanga Reo Trust.

Perkins, D., Tishman, S., Ritchhart, R., Donis, K., & Andrade, A. (2000). Intelligence in the wild: A dispositional view of intellectual traits. Educational Psychology Review, 12(3), 269–293.

Perkins D. N., Jay, E., & Tishman, S. (1993). Beyond abilities: A dispositional theory of thinking. Merrill-Palmer Quarterly, 39(1), 1–21.

Pinnegar, S., & Daynes, J. (2007). Locating narrative inquiry historically. In J. Clandinin (Ed.), Handbook of narrative inquiry (pp. 3–34). Thousand Oaks, CA: Sage.

Pollard, A., & Filer, A. (1999). The social world of pupil career. London: Cassell.

Ramsey, K., Breen, J., Sturm, J., Lee, W., & Carr, M. (2007). Weaving ICTs into te whāriki at Roskill South kindergarten. In A. Meade (Ed.), Cresting the waves: Innovation in early childhood education (pp. 29–36). Wellington: NZCER Press.

Reedy, T. (1995/2003). Tōku rangatiratanga nā te mana-mātauranga: Knowledge and power set me free. In J. Nuttall (Ed.), Weaving te whāriki: Aotearoa New Zealand’s early childhood curriculum document in theory and practice (pp. 51–77). Wellington: New Zealand Council for Educational Research.

Ritchhart, R. (2002). Intellectual character: What it is, why it matters, and how to get it. San Francisco, CA: Jossey-Bass.

Rogoff, B. (2003). The cultural nature of human development. Oxford and New York: Oxford University Press.

Siraj-Blatchford, I. (2004). Educational disadvantage in the early years: How do we overcome it?: Some lessons from research. European Early Childhood Education Research Journal, 12(2), 5–20.

Stuart, D., Aitken, H., & Gould, K. (2008). Evaluation of the implementation of kei tua o te pae assessment for learning: Early childhood exemplars: Impact evaluation of the Kei Tua o te Pae 2006 professional development. Auckland: Cognition Consulting.

Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge: Cambridge University Press.

Notes

1 The resource was developed in the Early Childhood Learning and Assessment Exemplar project, co-directed by Margaret Carr and Wendy Lee, and based at the University of Waikato. Carolyn Jones managed the project and is a co-author and compiler of the books. Additional authors provided expertise for Book 8, Infants and Toddlers (Keryn Davis); Book 9, Inclusion (Lesley Dunn and Sally Barry); Book 17, Literacy (Ann Hatherly); and Book 18, Mathematics (Sally Peters). Project co-ordinators kept a close link with participating centres and settings. We were advised by a Project Advisory Committee and Te Rōpū Kaiwhakangungu.

2 Te reo Māori, the Māori language, is the language of the first peoples of New Zealand.

3 There is not room in this paper to elaborate on the definition of disposition that includes motivation (being ready), attunement to circumstance (being willing) and being able. This triad was introduced to the literature by David Perkins and colleagues in 1993, and elaborated by Ritchhart (2002) and Carr (2001). It was adopted in Te Whāriki, p. 44. Perkins and colleagues used the term “sensitivity to occasion”; the notion of “attunement” comes from James Greeno and the Middle School Mathematics Through Applications Project Group (1998), who emphasise the importance of learners recognising—and sometimes constructing— opportunities to learn.

4 Consistent with this seeking of alternatives to one-path linear approaches to progression, for reasons outlined in this paper, the “literacy”-based topic books in the final series of Kei Tua o te Pae (oral, visual and written literacy, mathematics, the arts and ICT) adapt Alan Luke and Peter Freebody’s map or repertoire of literacy practices to describe the outcomes (Freebody & Luke, 2003; Luke & Freebody, 1999).

Acknowledgements

The author gratefully acknowledges the Ministry of Education, who initiated and funded this project, and Learning Media, who published it. It was a long journey, with many constructive discussions and debates along the way. Also acknowledged with thanks are my colleagues in the Early Childhood Learning and Assessment Exemplar project: Wendy Lee, Carolyn Jones, the project co-ordinators, the Advisory Committee, Te Rōpū Kaiwhakangungu and the specialist co-authors. Thanks also to the teachers, families and children from early childhood settings who so willingly contributed their stories, and the participants in the professional development programme who took Kei Tua o te Pae off the shelves and changed assessment practices.

The author

Margaret Carr is a Professor of Education at the Wilf Malcolm Institute of Educational Research at the University of Waikato. Her areas of research and writing include working with teachers in action research projects that focus on curriculum and assessment topics in the early years. She was a co-director with Wendy Lee of the Early Childhood Learning and Assessment Exemplar project that developed Kei Tua o te Pae/Assessment for Learning: Early Childhood Exemplars, and she has written a book on learning stories as an assessment tool (Carr, 2001).

Email: margcarr@waikato.ac.nz