You are here

The complexities of moderating student writing in a community of practice

Rosemary Hipkins and Sally Robertson
Abstract: 

This article uses Wenger’s (1998) theory of communities of practice as a framework to describe the complexities of moderation processes when these processes are intended to support teacher learning in ways that subsequently help lift student achievement. We draw on a case study in which we observed teachers working together to moderate students’ writing in a school that had an established reputation for good assessment practice. We discuss these observations in relation to Wenger’s four attributes of a community of practice: practice, community, meaning and identity. Our purpose is not to prescribe “ideal” practice but rather to highlight the effort and commitment required to develop and maintain a community of practice around moderation processes that strengthen the professional knowledge teachers bring to their classroom practice. Developing and sustaining a robust community of practice for moderation purposes entails important leadership work and a strategic alignment between moderation practices and other systems and processes in the school.

The complexities of moderating student writing in a community of practice

Rosemary Hipkins and Sally Robertson

Abstract

This article uses Wenger’s (1998) theory of communities of practice as a framework to describe the complexities of moderation processes when these processes are intended to support teacher learning in ways that subsequently help lift student achievement. We draw on a case study in which we observed teachers working together to moderate students’ writing in a school that had an established reputation for good assessment practice. We discuss these observations in relation to Wenger’s four attributes of a community of practice: practice, community, meaning and identity. Our purpose is not to prescribe “ideal” practice but rather to highlight the effort and commitment required to develop and maintain a community of practice around moderation processes that strengthen the professional knowledge teachers bring to their classroom practice. Developing and sustaining a robust community of practice for moderation purposes entails important leadership work and a strategic alignment between moderation practices and other systems and processes in the school.

The context and the challenge

In New Zealand, recently introduced National Standards for reading, writing and numeracy in Years 1–8 have shone a spotlight on primary schools’ moderation practices. For some years now moderating student work has been integral to many—but by no means all—primary schools’ self-auditing of their effectiveness in meeting students’ learning needs. As part of an overall assessment for learning strategy, data generated via school-wide assessment and moderation can flow back into classrooms to inform ongoing learning (Absolum, Flockton, Hattie, Hipkins, & Reid, 2009). The National Standards policy introduces a new outward-looking accountability that provides a different reason for moderation and raises the stakes somewhat. The responsibility for assessment decision making remains with the schools, but when the intent is to make comparisons between schools, leaders and teachers must make the basis for their reported judgements as transparent as possible. In this context there is a risk that moderation could become more compliance-focused and in the process lose its professional learning edge. Our agenda is to keep the moderation focus on assessment for learning, without in the process compromising more outward-looking accountabilities.

We argue that any positive opportunity in the National Standards policy is more likely to be realised when teachers learn together in ways that support them to grow professionally and that enhance their ability to meet the specific needs of their learners with the aim of raising overall achievement levels. This process arguably begins with practising moderation and building a community of moderators who have careful focus on the meaning of the work that students generate. In this way, judgements being made for accountability reasons will also inform future learning challenges. Etienne Wenger’s theory of communities of practice is a good fit for the type of dynamic collegial learning we envisage, and we use his ideas as the framework on which we can build a snapshot of the complexity of actual moderation practice. As we will demonstrate, it is one of those types of activity that is easy to outline in principle but not so easy to do in practice.

During moderation, there are multiple possible points of decision making (Klenowski & Wyatt-Smith, 2010). Teachers must make important judgements about the meaning of individual pieces of work. Students’ outputs are likely to be multifaceted and hence could well display evidence at differing levels with respect to one or more standards, requiring that someone makes an overall judgement before assigning a single final mark or grade. Similarly, for a portfolio of work accumulated over time, different pieces of evidence will need to be weighed up to create an overall teacher judgement (OTJ) against the standard for reporting purposes. Again, someone will need to decide. When such decisions are made by a collective, moderation can support teachers to reach a shared understanding of the meaning of a standard, and to more reliably judge a range of evidence in relation to that standard. We acknowledge the importance of externally focused accountability reasons for moderating students’ work, but our interest lies in leveraging the potential of moderation conversations as opportunities to strengthen the professional knowledge that teachers bring to their classroom practice.

Building a community of practice around moderation activities

Moderation should ideally entail conversations that take place in conditions for productive collegial learning. It is very important that classroom teachers are active and valued learners in this process. The trick here is to put any imposed accountability agenda into perspective without in the process compromising its robustness, so that an assessment for learning purpose can come to the fore. Compared with “doing the paperwork”, helping their students will always be a more compelling reason for teachers to engage, particularly given that moderation conversations must often take place in out-of-class time. The idea of a community of practice provides a helpful theoretical framework for addressing practical issues and challenges in ways that make the establishment and maintenance of such learning-focused engagement more likely. Participation comes with the territory: within a community of practice, learning is defined as the production of practice with all its emergent insights and complexities (Wenger, 2010).

Teachers will often interact collegially during the course of their work but a culture of support and sharing does not necessarily imply a productive teacher learning focus (Wylie, 2010, 2011). Learning and changing via participation in a group that shares a common learning cause is a necessary condition for consideration as a community of practice. As Wenger (2006) describes:

Communities of practice are groups of people who share a concern or passion for something they do and learn how to do better as they interact regularly. (p. 1, emphasis added)

The focal point is the practice itself and the improvements to that practice that come from learning and acting together. As outlined above, this is precisely the opportunity and challenge presented by the need to moderate student work. For accountability reasons, students’ work must be moderated, and be seen to be moderated, via robust processes. If such moderation is to generate insights that inform future teaching and learning, every teacher who wishes to engage in robust assessment for learning conversations with their students should be a member of the group that engages in the relevant moderation practices. Notice also that the practice must engender a shared passion or concern: this provides the energy and sense of identification that motivates sustained engagement. Teachers feeling coerced into participation are unlikely to be open to learning and growing together. Indeed, addressing the question of what he sees as the three key enablers of the successful establishment and maintenance of a community of practice, Wenger lists: passion/social energy, effective leadership and the importance of ensuring high value for the time expended (Wenger, n.d.). It is important to establish all three of these conditions when setting up moderation processes.

In his earlier work, Wenger (1998, p. 5) listed four key attributes of a community of practice:

Practice—a way of talking about the shared historical and social resources, frameworks, and perspectives that can sustain mutual engagement in action.

Meaning—a way of talking about our (changing) ability—individually and collectively—to experience our life and the world as meaningful.

Community—a way of talking about the social configurations in which our enterprises are defined as worth pursuing and our participation is recognisable as competence.

Identity—a way of talking about how learning changes who we are and creates personal histories of becoming in the context of our communities.

We found these four attributes very helpful for organising insights we gleaned from published research on moderation and its challenges (Hipkins & Robertson, 2011). In a subsequent section of this paper, each of these attributes will be briefly described and illustrated with reference to teachers whom we observed working together to moderate student work in a school that had an established reputation for good assessment practice. First, we briefly introduce the school and our approach to the fieldwork that yielded the descriptions that follow.

Observing moderation conversations at Flanshaw Road School1

Our rationale for the small empirical component of this project was that it would be most helpful for other schools if we were able to provide examples of moderation processes that were working well. As a small-scale exploratory study, it was not our intention to make generalisations about moderation practices more widely. We investigated the moderation practices at Flanshaw Road School in Waitakere City, Auckland. This school is ethnically diverse, decile 5 and has around 400 students from Years 1–6. The principal approached us with an invitation to visit: the school has worked hard over a number of years to develop as a community of practice in which every teacher keeps on developing their skills for strengthening student achievement in writing and hence collectively lifting their overall achievement statistics. Moderation of writing is seen as just one of a suite of practices that has contributed to that success.

The fieldwork was conducted over two days during November 2011. In this time we observed three moderation meetings, all with a focus on making judgements about individual pieces of student writing that had been gathered the week before our arrival. We observed conversations between: five teachers of Years 1–2 students, with two of this group in their first two years of teaching; four teachers of Years 5–6 students; and an after-school meeting with almost all teaching staff present. As we had anticipated, moderation conversations between willing and engaged participants are rapid-fire, dynamic and complex. Making sense of such a conversation requires sustained concentration and focus. From our literature review we had developed an observation checklist (see Table 1) and we used this as a basis for making descriptive notes about the practices we observed. We were not concerned with making evaluative judgements about the individuals who took part: rather, we were focused on learning opportunities and interactions. We also conducted postmoderation interviews with the principal and four teachers to explore their perspectives on the moderation practices we had observed and their implications within the wider context of the school. We requested that these interviews span a range of teacher expertise: longer serving and early-career teachers, those with particular expertise in the development of student writing competencies and those who did not specialise in this area.

Table 1 Observation criteria

Concept

Key questions

Meeting structure

How much time is given to the meetings? How often do they take place?

Where do they take place? (conversation conditions)

How is the group led? Is there a facilitator? If so, what is their role? Are they someone internal or external to the school?

Is there any protocol guiding the moderation discussions?

Group identity

Who is included in the group (e.g., experienced/inexperienced moderators, same/different school, subjects taught, primary and/or secondary teachers)? Who is missing from the group?

Are students involved in any way?

Group dynamics

What sorts of emotions are evident?

How does the group establish trust? (Does the facilitator create a safe learning environment?)

(How) do group members support one another?

Whose voice is heard/not heard?

How are suggestions treated?

Active engagement

Who is contributing? What sorts of contributions do they make? (observations, questions, comments)

What is the focus of each contribution? (students’ work, the standard being used to make the judgement, guidelines, a specific student)

Do discussions remain focused? Is there any process or support for this to happen?

How invested in the process are the group members?

Shared meanings

What different concrete and tacit resources are drawn on? In what way are these used?

(How) are meanings shared?

(How) does the group arrive at a consensus?

Whose views do others challenge and whose do they support? On what grounds do they do so?

Whose views are prioritised when reaching a decision?

Deep learning

Are individual members and the group given time for critical reflection? What occurs during this reflection time?

Is discussion focused on surface-level features or the deeper purposes and qualities of learning?

Do group members challenge their own and others’ beliefs?

Is the overall emphasis on learning or compliance?

Ongoing impact of moderation

Are opportunities for putting learning into place discussed?

Is there a process for reassessing other student work in light of what has been learned?

We analysed our field notes by considering how the different components of Wenger’s community of practice model (practice, community, meaning and identity) had been exemplified in the case study. We subsequently developed a diagram that organised the complexity of the moderation process within the wider professional practices that were established in this school. This diagram will be presented and discussed later in the paper. We were in no doubt that this was indeed a dynamic community of practice and we now describe the different components of Wenger’s theory in action in this school.

Moderation as practice: Learning as doing

A community of practice is not merely a community of interest—people who like certain kinds of movies, for instance. Members of a community of practice are practitioners. They develop a shared repertoire of resources: experiences, stories, tools, ways of addressing recurring problems—in short a shared practice. This takes time and sustained interaction. (Wenger, 2006, p. 2, emphasis added)

The literature has several clear messages about the nature of moderation practice that productively supports teacher learning. To be useful for such learning, moderation needs to be ongoing (Maxwell, 2002), thereby providing regular opportunities to practise new skills (Strachan, 1996). Sharing and challenging one’s own and others’ beliefs can make people feel vulnerable and anxious, so it is important that practice opportunities are conducted in a trusting and supportive environment even when challenge and tension are part and parcel of the practice (Wenger, 1998). Healy and Bush (2010) suggest that it is important to have protocols for guiding moderation discussions so that teachers can share their views equally and discussions remain focused. Strong facilitation by a leader who is well-informed about productive conversation sequences helps to pace conversations in ways that maximise teacher learning opportunities yet avoid frustration for participants.

It was evident that moderation meetings were seen as a high priority for staff at Flanshaw Road School. Formal moderation meetings to judge students’ progress in writing occurred at least once a term, and included syndicate meetings (where teachers of the same year levels met) and whole-school meetings (that involved all staff members across year levels). Moderation meetings for syndicate teams sometimes occurred during school hours, and the school has developed ways to free teachers to attend these meetings. Whole-school moderation meetings occurred after school and were attended by almost all teaching staff. The following characteristics enabled and supported robust moderation practice:

Moderation meetings were held regularly and were a high priority: The school has invested time and energy in developing and refining matrices for making progress in writing. It has also invested in professional development about effective teaching of writing. Moderation practices are regularly reflected on and revised by classroom teachers and school leaders. Each of the moderation meetings we observed lasted approximately an hour. Teachers who still had questions stayed beyond the moderation meetings to further discuss examples with their colleagues. Informal moderation conversations are common among teachers, and many teachers said they regularly talk to their colleagues about their assessment practice.

Clear expectations and goals were set for moderation meetings: Teachers held a common understanding about the aims and importance of the moderation meetings. The after-school meeting was structured along assessment for learning lines, with the principal identifying the overall learning intention to frame participation: “We are developing a collective understanding of the ways in which colleagues make decisions about the grading of writing samples.” Discussion tasks and success criteria were then outlined, adding to the sense of focus and purpose. In all of the moderation meetings we observed teachers were focused on moderation discussions from the start to the end.

All teachers were actively involved in moderation meetings: All teachers at the school participated in these focused, fast-paced moderation meetings. The syndicate meeting with teachers of the Years 1–2 students (where there were two fairly new teachers) was facilitated by an experienced teacher. This facilitator ensured that every participant had space to discuss their students’ work. She said her aim was to encourage teachers to become more confident in taking responsibility for decisions rather than relying on her for advice. She acknowledged that this takes trust, time and experience. All the teachers in the Years 5–6 syndicate meeting were experienced and there was no obvious leader. Neither was the staff meeting centrally facilitated. Everyone participated in small groups and took turns at presenting and discussing examples, many of which had been selected in the syndicate meetings as being in need of a wider opinion. Several teachers with deep expertise (including the ESOL teacher and another teacher with a strong academic background in writing) were available to the small groups when called on.

Matrices guided the discussions: Matrices of writing progress developed within a wider cluster of schools had been refined for Flanshaw Road School and served as key enablers of meaning-making conversations during moderation meetings. Teachers challenged one another to provide evidence from their student work samples and to explain how these fitted various aspects of the criteria. As the teachers debated with one another they developed ways to deal with difficult cases: in essence, they were developing a shared practice. The teachers also discussed how these moderation decisions would affect their future judgements and how their emergent understandings could be applied to their teaching. One teacher told us, “There’s no point in doing it unless you are going to use it.”

The case study data supported literature that suggests that there should be ongoing opportunities to practise moderation. However, the interactions we observed also suggested that moderation meetings may not need to be as carefully facilitated when all members are highly experienced teachers who have a well-established shared understanding of the purpose and process. The community’s history of working together is important and it is this aspect we address next.

Moderation as the making of meaning: Learning as experience

Through active and dynamic negotiation of meaning, practice is something that is produced over time by those who engage in it. In an inalienable sense, it is their production. (Wenger, 2010, p. 180)

Through the moderation process, “meanings are produced which may support, enhance or challenge current understandings and values held by the teachers” (Klenowski & Adie, 2009, p. 20). Challenges that teachers must confront might include: whether or not assessment should recognise effort (as well as achievement); whether assessment could de-motivate some students by identifying and documenting poor levels of achievement relative to their peers and to expected levels of progress; how conditions under which an assessment is completed might influence performance; beliefs about what an “average” student should be able to achieve; perceptions of the ability of individual students, or groups of students; and knowledge of specific challenges student(s) might have faced (Wyatt-Smith, Klenowski, & Gunn, 2010).

From 2007 to 2009 Flanshaw Road School was part of an Extending High Standards Across Schools (EHSAS) school cluster. The cluster took e-asTTle (a national assessment tool for writing) and a suite of New Zealand Curriculum Exemplars as the foundations on which they developed their own criteria for judging the progress that students make with their writing. School-wide ownership of these criteria is strong: they are posted in every classroom as a very public point of reference for students and parents, and they are the focus of achievement conversations and goal-setting practices. Despite this familiarity and history of active use, the teachers we observed needed to negotiate a number of complex issues as their decision making unfolded. This was not a one-way process of meaning making. The criteria themselves were an evolving, living reference and dilemmas addressed during moderation conversations sometimes resulted in adjustments to them. Issues that were debated in the conversations we observed included:

The relative amount of attention to pay to “deep” and “surface” features of writing: This tension was debated in all of the moderation meetings we observed, and much time was dedicated to these discussions. The early-career teachers, in particular, needed a lot of reassurance and clarification about how to weight these attributes when confronted with an insightful or creative piece of writing that was marred by careless errors. They tended to come back to this issue repeatedly and they did not appear to be especially convinced by the judgement of their more experienced peers.

How a recent teaching focus might have influenced students’ efforts: Classes that had recently focused on specific surface features (e.g., spelling, punctuation, sentence structure) unsur­prisingly tended to produce writing that was more accurate for those features. The samples being judged had been generated during a timed test-like task and there was a lot of discussion about whether taking extra care over surface features prevented some students from completing the whole task, or displaying some of the deeper features specified in the criteria. One teacher of older students noted that his recent teaching focus on how to use commas when writing lists had resulted in one of his students overusing this feature in the writing sample.

How to fairly judge the efforts of ESOL students: Some students who were second language speakers of English, particularly in the junior classes, had resorted to supplementing their written text with drawings that effectively conveyed their ideas. Teachers debated whether these should be considered in relation to their development of ideas or whether only the written words “counted”. We also observed a discussion, borne of direct practical experience, about specific ways in which the structure of the first language of Tuvaluan students could get in the way of their written English expression.

What to do when one sample was of variable quality: Teachers of the older students debated several examples where the first and second pages of writing could be judged as evidence of achieving different criterion levels. Again, this could be because students rushed toward the end, but we also heard discussion about some students who tended to “run out of steam” in tasks where sustained concentration was required. In such cases, the student’s teacher tended to take the role of advocate, pleading the case, but they seemed to do so in the expectation that their peers would take a more dispassionate view, thereby helping them to reach a fairer decision.

Form or flair? Some students were known to be cautious and averse to making mistakes. However their “safe” writing was not as interesting as that produced by students who went for flair and interest, sometimes at the expense of accuracy. Sometimes teachers were aware that writing with more flair was a specific goal for one of their students and they hoped to see such efforts rewarded.

The adequacy of the criteria: Teachers of the older student group discussed whether the criteria in the matrices flowed logically from one level to the next and the difficulty of making decisions about borderline cases. There were many discussions about where the different levels started and ended. One teacher initially said that it would be preferable to have fixed criteria that flowed well. However, he later commented that he did not think that this was possible and that there were benefits of working with criteria that could continue to change and evolve. Some criteria were seen as more difficult to apply to certain genres of writing. Another issue was that an older student working below the level of their peers could be judged to be writing at the same level as a junior student working above the level of their peers, even though many aspects of their work were not directly comparable because of their age/experience differences.

The impact of the task on the students: One teacher of younger students discussed how she had used a new planning template for the first time as her students began the assessed writing task. One student had struggled with this: “Something unfamiliar throws them in a loop.” A more experienced teacher explained the importance of scaffolding students and not introducing new ideas about the process of writing during an assessment task.

The influence of knowledge of the student: The teachers often made comments about wanting their students to do well and being disappointed if they did not. Sometimes they offered information about a student’s personality or about the amount of effort they had put into their writing. The facilitator of the junior syndicate dealt with this by bringing the discussion back to focus on the criteria. The teachers of the older students discussed how hard it was not to be influenced by knowledge of the students. They had previously tried blind marking (however, they could still recognise whose work they were assessing) or had swapped their class’s samples with another teacher to mark.

Whether evidence could come from class work as well as writing sample: The teachers of the younger students sometimes discussed how they knew their student could use a particular writing feature (e.g., full stops) but they had not displayed this in the writing sample. Sometimes there was discussion about whether they could bring a different sample of writing from that student. It appeared that most teachers thought that other evidence could sometimes be taken into account. This was not discussed by the teachers of the older students.

Our case study illustrates why moderation to reach an agreement about student work is by no means a straightforward process. It is important that there are safe spaces created for ideas to be debated and challenged within moderation meetings, and that the different areas of expertise that teachers bring are acknowledged. As they work together, teachers build a history of decisions that will serve as an important point of reference in future moderation meetings.

Moderating within a community: Learning as belonging

The history of practice, the significance of what drives the community, the relationships that shape it, and the identities of members all provide resources for learning—for newcomers and oldtimers alike.(Wenger, 2010, p. 182)

Moderation conversations are productive when members of the community feel safe to support and challenge one another as they debate to reach a consensus about the meaning of student work. It is of utmost importance that teachers do not feel as if they are working in isolation, and positive feelings about the work can be an important outcome of productive exchanges (Fullan, 2001, as cited in Barrett, 2008). The wider learning culture at Flanshaw Road School was characterised by valuing honesty and a no-blame, learning-focused environment. Teachers were comfortable challenging themselves and one another and there was a strong sense of trust within the group. Embedded within this wider high-trust culture, one teacher described the moderation meeting as “just colleagues talking about learning”.

Price (2005) notes that both expert and novice assessors should be a part of the assessment community because everyone needs to be socialised within the particular learning community. Furthermore, being able to access a diversity of perspectives and experiences during the conversation is a fundamental precondition for learning to emerge in any type of community (see, for example, Davis, Sumara, & Luce-Kapler, 2008). The clear implication is that moderation tasks should not be assigned only to the more experienced teachers (as might happen if only efficiency is being sought).

A range of participants

Next, we outline how opportunities are created for a range of participant groups to be involved in the wider processes around moderation as part of the Flanshaw Road School extended family. Diversity of perspectives can be broadened even beyond expert and novice teachers (although their roles remain central) to involve other groups with an important stake in the decisions being made.

Teachers bring different expertise and experience

Different areas of expertise and experience were acknowledged and utilised in the moderation conversations we observed. Teachers from the senior school syndicate looked to two of their peers who had strong academic knowledge of literacy practices. The expertise of the ESOL teacher was also highly valued and the school has found a way to free up her time to work with teachers across the school. Newer teachers are supported by more experienced staff, including team leaders and the leadership team.

Involvement of students

Although students were not directly part of the moderation meetings they were involved in the wider moderation process. The writing matrices/criteria are displayed in student-friendly language in every classroom. Both the teachers and students valued the criteria and they worked together to develop a shared understanding of what making progress in writing looks like. These shared understandings enabled conversations about next steps in a student’s learning and the teachers talked about the positive impact that moderation has had on teacher interactions with students as they set and work to reach their personal writing goals.

Wider community involvement

Via portfolios of work, two-way and three-way conferences, parents were proactively informed about their child’s learning successes and next challenges. The board of trustees and the senior leadership team were also kept updated about writing progress and any moderation challenges that had been difficult to resolve.

Beyond the individual school

The principal, along with several other members of the senior leadership team, continued to be an active participant in the local school cluster that had developed the writing matrices, and a second larger cluster of schools that work together as a learning network. In this way, they kept an outwards-focused point of reference, even as their internal practice continued to evolve.

In summary, the case study highlighted the benefits of moderation processes that directly or indirectly involve many different community members. Our observations endorse literature that points to the importance of having a diversity of perspectives within a moderation community in order for learning to occur.

Practice as identity formation: Learning as becoming

Sharing a history of learning ends up distinguishing those who were involved from those who were not. They share an enterprise, an understanding of what matters, relationships, as well as the resources that their history has produced. (Wenger, 2010, p. 181)

Within a sociocultural framing, identity can be thought of as constantly evolving via the stories we tell about ourselves and that others tell about us (Sfard & Prusak, 2005). When these stories concern acts of learning together and its consequences for individuals (e.g., new knowledge, skills, insights, values or dispositional changes, any of which could co-implicate a changing sense of self as a capable learner) then learning can be seen as involving a “transformation of identity, where identity is understood as evolving forms of competence” (Murphy & Hall, 2008, p. ix). A sense of “becoming” in the moderation context we have described might entail: building stronger competencies as an insightful and fair judge of student writing; becoming more skilful and responsive as a classroom teacher of writing; and seeing oneself as an ongoing learner with respect to professional knowledge of quality student writing in all its complexities (progression, genres, surface and deep features, interaction with other knowledge and skills etc.). Here identities as both a moderator and a teacher come together with an ultimate focus on improving student learning, with pedagogical self-evaluation and critical reflection being seen as important parts of the moderation process (Nixon & McClay, 2007).

How, then, can teachers best be supported so that they do come to view themselves as increasingly experienced moderators, and informed and skilful judges and scaffolders of high-quality student writing? Participation in appropriately structured and facilitated moderation meetings is necessary but not sufficient. Coming to view oneself as a competent moderator can take considerable time and practice (Klenowski & Adie, 2009) so continuity of moderation experiences is also important. Flanshaw Road School’s writing matrices, together with stories teachers recalled during their ongoing use (e.g., changes they had made, disputes over detail, tricky examples individuals had brought to past conversations) were an important carrier of continuity. As a focal point for attention and decision making, these matrices served as a stable but evolving record of the teachers’ learning. They were a concrete resource produced and sustained via their history of working together. Given their central role in the teachers’ meaning making, a strong sense of ownership of the matrices was very important. They had not been imposed by outsiders but rather evolved within the community of practice and its own extended networks.

A stable staff with a low rate of turnover is another carrier of continuity when teachers have shared understandings, experiences, expectations and values (Cowie et al., 2009). An habitual way of being when learning and working together is another potential carrier of continuity: the “how we do things here” element of practice in the school. It seemed to us that other aspects of professional learning were approached with the same high-trust energy and generosity of sharing at Flanshaw Road School. A number of teachers and the principal were actively engaged in tertiary studies in a chosen aspect of their professional practice.

The situated nature of a community of practice

The regime of competence of a community of practice translates into a regime of accountability—accountability to what the community is about, to its open issues and challenges, to the quality of relationships in the community, to the accumulated products of its history. (Wenger, 2010, p. 186)

Given the time New Zealand’s primary teachers are now destined to spend in moderation conversations, finding ways to turn these into rich professional learning conversations would appear to be no less than an urgent moral imperative (Fullan, 2010). Our case study set out to document the nature of moderation processes that allow new meanings and understandings to emerge and hence professional growth to take place in ways that directly contribute to an ongoing strengthening of professional practice. We saw it as an important research challenge to attempt to better understand what happens in the spaces between those involved in the moderation, and to locate moderation as a situated activity: part of the ebb and flow of the wider life of the school and all the social interactions and intended and unintended learning experiences that take place therein. It is these situated aspects of moderation practice and their associated accountabilities that we now address.

Drawing together the various threads of the fieldwork, we developed a diagram (see Figure 1) to represent the ways in which moderation processes at Flanshaw Road School were embedded within other aspects of school practice, each of which brought different sets of accountabilities, both to the individuals and the collective.

Figure 1 Moderation at Flanshaw Road School

The core moderation process (inside the dotted line) constituted three phases:

Phase One began with the staging of a memorable school-wide event on which the writing task would be based, and in which everyone took part. The intention was that students would be motivated by this event to produce the best writing sample they could. Accountability for making the event memorable was shared across the whole school community and at the same time became a shared bank of experiences on which everyone present could draw. Within their teams teachers developed age-appropriate guidelines for setting up and carrying out the assessment, to ensure there was consistency of conditions for all the students. Individual teachers were accountable for the ways they applied the guidelines to bridge between the event and the actual writing, which took place in each classroom at the same time the next day.

After individual teacher grading of the student samples, the actual moderation conversations documented above constituted the second phase. Teacher accountabilities in this phase were to their colleagues as increasingly competent judges of student writing, but also to the overall reliability of the grades generated for documenting school-wide patterns of achievement as the basis for reporting to parents and the wider school community, via the school’s board of trustees. New Zealand also has a policy of school target setting, followed by annual reporting to the Ministry of Education. In this way accountabilities flow beyond the local community, with the ultimate target of making responsible returns for the taxpayer investment in the education of the children at the school. The principal and teachers were proud of the school’s achievements in steadily raising the children’s writing levels. All the effort invested was seen to be paying off and this doubtless helped sustain the considerable effort and investment teachers made to the community of robust moderation practice. The challenge was seen as ongoing, however. There was no sense that the staff were resting on their laurels.

Postmoderation actions constituted the third phase and it was here that the teachers became directly accountable to their students. Back in their classrooms they discussed each child’s actual achievement on the task, progress made and next learning steps. As the teachers developed a deeper understanding of their students’ writing, so the students themselves also learned more about their writing progress and in turn shared this with their parents via two-way and three-way student-led conferences with parents, with the teacher present in support. In the diagram, we have positioned wider learning/identity consequences of this third phase outside the actual moderation process, to signal their potential to spill over into other aspects of learning.

All the processes described had backward as well as forward flows. Conversations within the community of practice informed understandings of previous processes; for example, the moderation discussions provided information about the successes and challenges of the process used for generating the writing task. These backward-looking accountabilities had a future-focused element. The learning that emerged was used to shape ongoing practice even as it reinforced the existing school culture and values that constituted the bedrock on which the whole community of practice had been built. Finally, the diagram shows the multiple points of connection between the core moderation activities (in the dotted box) and the wider school professional learning contexts, including a local cluster group. Moderation is a fully integrated part of the school’s professional learning, not a separate compliance exercise.

From a situated perspective, the key message of our case study is that a combination of structures and processes, together with their skilled and supportive enactment, enables moderation to successfully balance its multiple accountabilities. There can be no shortcuts if each community of practice is to learn and grow. Every relevant aspect of practice matters: school leadership; school ethos and ways of working; clear professional learning targets; local ownership of, and belief in, resources used for making judgements; transparency to and support from the local community; and more. Moderation cannot be “done to” schools if we want teachers to use it as a valued professional learning opportunity. It is a complex, dynamic process with emergent qualities that must come from within the community itself. Thus each school’s teachers, working as a community of practice, must be appropriately supported in carrying out their own moderation.

The “last word” in this article goes to the principal of Flanshaw Road School. She notes that her school’s journey had begun well before the high-stakes National Standards policy had been introduced. She feels unsure whether the process would have been as successful if it had been initiated once these higher stakes were already in place. For her and her teachers, assessment for learning purposes will always be to the fore. She acknowledges the efficiency of being able to use data for accountability purposes after they have been gathered in ways that inform and enhance the learning that students experience. But external accountability has never been the driver for Flanshaw Road School. Building a robust and dynamic community of moderators is first and foremost an integral part of a more holistic approach to lifting student achievement and it is important within-school leadership work. The principal believes that once transparent accountabilities have been established within schools, similarly developing them across schools will become a more achievable possibility. In this way, creating strong moderation communities inside each and every school is an important first step in realising the important goal of raising student achievement system wide.

Acknowledgements

The research for this paper was funded by NZCER’s Purchase Agreement with the New Zealand Ministry of Education. Our thanks go to everyone at Flanshaw Road School who smoothed the path of our fieldwork and provided helpful feedback on this paper.

References

Absolum, M., Flockton, L., Hattie, J., Hipkins, R., & Reid, I. (2009). Directions for assessment in New Zealand (DANZ). Wellington: Ministry of Education. Retrieved from Te Kete Ipurangi: http://assessment.tki.org.nz/Assessment-in-the-classroom/Assessment-position-papers

Barrett, J. (2008). The impact of the NCEA on teacher collegiality. Unpublished doctoral thesis, University of Waikato, Hamilton.

Cowie, B., Hipkins, R., Boyd, S., Bull, A., Keown, P., McGee, C., et al. (2009). Curriculum Implementation Exploratory Studies: Final report. Wellington: Ministry of Education. Retrieved from Education Counts: http://www.educationcounts.govt.nz/publications/curriculum/57760/1

Davis, B., Sumara, D., & Luce-Kapler, R. (2008). Engaging minds: Changing teaching in complex times (2nd ed.). New York: Routledge.

Fullan, M. (2010). All systems go: The change imperative for whole system reform. Thousand Oaks, CA: Corwin/Sage.

Healy, H., & Bush, H. (2010). Moderation: Making learning a priority in primary religious education. Journal of Religious Education, 58(1), 29–37.

Hipkins, R., & Robertson, S. (2011). Moderation and teacher learning: What can research tell us about their interrelationships? Wellington: New Zealand Council for Educational Research. Retrieved from http://www.nzcer.org.nz/research/publications/moderation-and-teacher-learning

Klenowski, V., & Adie, L. (2009). Moderation as judgement practice: Reconciling system level accountability and local level practice. Curriculum Perspectives, 29(1), 10–28.

Klenowski, V., & Wyatt-Smith, C. (2010). Standards-driven reform years 1–10: Moderation an optional extra? Australian Educational Researcher, 37(2), 21–39.

Maxwell, G. S. (2002). Moderation of teacher judgments in student assessment: Discussion paper. Brisbane: Queensland School Curriculum Council.

Murphy, P., & Hall, K. (2008). Learning and practice: Agency and identities. London: Sage.

Nixon, R., & McClay, J. K. (2007). Collaborative writing assessment: Sowing seeds for transformational adult learning. Assessing Writing, 12(2), 149–166.

Price, M. (2005). Assessment standards: The role of communities of practice and the scholarship of assessment. Assessment and Evaluation in Higher Education, 30(3), 215–230.

Sfard, A., & Prusak, A. (2005). Telling identities: In search of an analytic tool for investigating learning as a culturally shaped activity. Educational Researcher, 34(4), 14–22.

Strachan, J. (1996, December). Moderation: Swan or ugly duckling? Unpublished paper presented at the annual New Zealand Association for Research in Education conference, Nelson.

Wenger, E. (n.d.). What are three key success factors for communities of practice? Retrieved February 2012, from http://wenger-trayner.com/resources/key-success-factors/

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge: Cambridge University Press.

Wenger, E. (2006). Communities of practice: A brief introduction. Retrieved February 2012, from www.ewenger.com/theory/

Wenger, E. (2010). Communities of practice and social learning systems: The career of a concept. In C. Blackmore (Ed.), Social learning systems and communities of practice (pp. 179–198). London: Springer.

Wyatt-Smith, C., Klenowski, V., & Gunn, S. (2010). The centrality of teachers’ judgement practice in assessment: A study of standards in moderation. Assessment in Education: Principles, Policy and Practice, 17(1), 59–75.

Wylie, C. (2010). Focusing leadership on adult learning: The secondary school challenge. Journal of Educational Leadership, Policy and Practice, 25(1), 51–66.

Wylie, C. (2011, April). Opportunities for teacher collaborative practices in a self-managed school system: The New Zealand experience. Paper presented at the American Educational Research Association 2011 annual meeting, New Orleans. Retrieved from http://www.nzcer.org.nz/research/publications/opportunities-teacher-collaborative-practices-self-managed-school-system-new-z

Note

1The school is named with their consent.

The authors

Rosemary Hipkins is a chief researcher at the New Zealand Council for Educational Research.

Email: rose.hipkins@nzcer.org.nz

Sally Robertson is a researcher at the New Zealand Council for Educational Research.

Email: sally.robertson@nzcer.org.nz