One of the most important goals for teaching statistics is to prepare students to deal with the statistical information that increasingly impacts on their everyday lives. Students need to be able to critically evaluate statistical information and data-based arguments. The findings of this collaborative research study of Year 9 students suggest that all students can and should be exposed to critical thinking in statistics, and identify some ways that teaching of statistical literacy might be altered for greater effectiveness.
Developing statistical literacy with Year 9 students
Key points
•Most Year 9 students are adept at doing calculations and retrieving information from tables and graphs, but many still lack the statistical literacy to critically interpret and question statistical information.
•Changes to classroom pedagogy, such as the use of questioning and class discussion, can be effective in deepening critical thinking and student questioning.
•Literacy skills are a key component—students need to be able to interpret written and oral texts. They may need specific support with reading and writing.
•Understanding the context is important for interpretation. To scaffold students, teachers can start with familiar contexts and move to unfamiliar ones.
•This study developed a four-stage statistical literacy framework that can be helpful in diagnosing the stage that students are at, in order to focus teaching on moving them to the next level. The stages are: informal/idiosyncratic; consistent noncritical; early critical; and advanced critical.
One of the most important goals for teaching statistics is to prepare students to deal with the statistical information that increasingly impacts on their everyday lives. Students need to be able to critically evaluate statistical information and datābased arguments. The findings of this collaborative research study of Year 9 students suggest that all students can and should be exposed to critical thinking in statistics, and identify some ways that teaching of statistical literacy might be altered for greater effectiveness.
Background
“Left hand in IQ tests, dealt a bad hand in life”—New Zealand Herald headline.
Source: “Left hand in IQ tests” (2008)
Advances in technology and communication in our information-rich society have increased the extent to which students across the world are exposed to statistical messages in diverse contexts. Challenging statements and graphics such as those above regularly appear in the news media. However, students without statistical literacy may be misled or have difficulty in interpreting and critically evaluating these messages (Budgett & Pfannkuch, 2010). For instance, in the examples above they would need to check how typical is defined, and critically evaluate whether the headline is an accurate summary of the survey results.
There is a wide range of conceptualisations of statistical literacy. Chick, Pfannkuch and Watson (2005) describe one aspect of statistical literacy as “transnumerative thinking”, where students make sense of and use different representations of data to make sense of the world around them. Gal (2004) suggests that the type of statistical literacy students need is to be able to interpret results from studies and reports and to be able to “pose critical questions and activate a critical stance” (p. 51) about those reports. He argues that since “most adults are consumers rather than producers of statistical information” (p. 49), classroom instruction needs to focus more on interpretation of data than on generating data. Gal would like students to come away from a statistical literacy class with an ability to evaluate statements like those above and ask questions such as: Where did the data come from? What kind of study is it? According to Watson (2006, p. 11), statistical literacy is the “meeting point of the chance and data curriculum and the everyday world, where encounters involve unrehearsed contexts and spontaneous decision-making based on the ability to apply statistical tools, general contextual knowledge, and critical literacy skills”. Clearly, the type of statistical literacy that Gal (2004) and Watson (2006) identify is different from just being able to read and evaluate data and graphs.
Aspects of Gal’s notion of statistical literacy have been incorporated in The New Zealand Curriculum, which states: “Statistics also involves interpreting statistical information, evaluating datābased arguments, and dealing with uncertainty and variation” (Ministry of Education, 2007, p. 26). Although the term critical thinking does not appear in the achievement objectives, it is embedded in the thinking key competency of The New Zealand Curriculum (Ministry of Education, 2007).
The implementation of these extended notions of statistical literacy is likely to pose a challenge to teachers. The approaches and initiatives recommended in The New Zealand Curriculum (Ministry of Education, 2007) may not be fully implemented due to secondary teachers being trained in mathematics rather than statistics (Begg et al., 2004; Shaughnessy, 2007). In part this may also be due to the debate among educationalists and curriculum developers about the nature of statistics and mathematics and best practice for instruction in each domain (Begg et al., 2004; Gal, 2004). Rossman, Chance and Medina (2006) argue that mathematics strips the context in order to study the abstract structure and generalise, whereas in statistics, context is crucial for analysing data. They state that students need multiple opportunities to relate their comments to the context when drawing conclusions. The complexity of integrating contextual information at all levels and the required shift in emphasis from statistical procedures to a focus on interpretation and analysis may be the most challenging for teachers to unpack. Teachers need to understand what is meant by statistics and to understand the implications of statistical thinking, probability and statistical literacy for teaching and learning in their classrooms.
Our concerns about the importance of statistics in everyday life, and the lack of research in this area, led to our collaborative research study.
Research design and data collection methods
Research questions
Three interrelated research questions guided the study:
•How can we support students to develop statistical literacy within a datāevaluation environment?
•How can we develop a classroom culture where students learn to make and support statistical arguments based on data in response to a question of interest to them?
•What learning activities and tools can be used in the classroom to develop students’ statistical critical thinking skills?
The research approach
Although there have been calls in statistics education to engage students in solving statistical problems that require them to collect and explore data in meaningful contexts, research (Sharma, 2007; Shaughnessy, 2007; Watson, 2006) suggests that these activities may not be enough to develop students’ understanding and statistical reasoning. One way to develop an effective sequence of activities is through a research and development process called design research (Cobb, 2000). Design research is cyclic, with action and critical reflection taking place in turn. Design research generally involves cycles of three phases: a preparation and a design phase; a teaching experiment phase; and a retrospective analysis phase.
Preparation for the teaching experiment
This phase consisted of a review of the literature on statistical literacy and teaching experiments and the first attempt at formulating a hypothetical learning trajectory. The research team (teachers and researcher) proposed a sequence of ideas, skills, knowledge and attitudes that they hoped students would construct as they participated in activities. The team envisioned how dialogue and statistical activity would unfold as a result of planned classroom activities.
Teaching experiment
The teaching experiment was carried out in regular Year 9 mathematics and statistics classrooms and as part of regular mathematics teaching. There were two cycles of teaching experiments. The teacher’s goal was to improve the lesson design by checking and revising conjectures about the trajectory of learning for both the classroom community and the individual students.
Retrospective analysis
The research team performed a retrospective analysis after each lesson to reflect on and redirect the learning trajectory. In addition, the team performed an analysis of the hypothetical learning trajectory after an entire teaching experiment had been completed. The continually changing knowledge of the research team created continual change in the hypothetical learning sequence.
Data collection
During the teaching experiment, the data set consisted of pre- and post-teaching tests, video recordings of classroom sessions conducted during the teaching experiment, copies of students’ written work and sets of field notes from the classroom sessions. Semistructured interviews were also conducted with four students from each class while the teaching experiment was in progress. These interviews were scheduled after class sessions and focused on students’ interpretation of classroom events, with a particular emphasis on the identities they were developing as consumers of statistics. Each teacher-researcher kept a logbook of specific events that took place during the data collection period.
Data analysis
The research team discussed the lessons, read the transcripts, watched the videotapes and formulated conjectures on hypothetical learning sequences and students’ learning on the basis of episodes identified in the transcripts and video. The generated conjectures were tested against other episodes and the rest of the collected data. Written responses and interview data were coded independently by researcher and teachers based on the developmental hierarchies used in the research literature (Sharma, 2007; Watson, 2006). The coding was then revised as the hierarchies were modified based on the research findings.
Results
Statistical literacy is more than the ability to do calculations and read tables and graphs.
Our findings show that Year 9 students are actually quite good at this. For instance, all students could accurately extract information from the bar graph about favourite junk food (Figure 2) in our pre- and post-teaching test. However, few, if any, students could initially ask a question of the data, such as “How many people were asked?” or “Were they asked in the summer or winter?”
With suitable scaffolding and support students were able to interpret and critically evaluate statistical information and datārelated arguments. Additionally, they were able to discuss and communicate their understanding and opinions to others. This was done in part by providing thinking and questioning routines such as the “Questioning the Data Detective” poster (Figure 3), which is modified from the PPDAC (Problem, plan, data, analysis, conclusion) poster (based on the statistical enquiry cycle of Wild & Pfannkuch, 1999) already seen in many New Zealand classrooms. It was also achieved by providing scaffolding for the literacy and contextual knowledge demands of tasks.
Our findings show that questions can be a powerful way of scaffolding students when used to initiate and sustain discussion and to encourage students to think critically and share ideas. This is illustrated by one student’s comment:
The simplest question I want to ask is how they got the information. Now that we have talked about statistics … and now that we probably understand a bit about statistics, I would want to ask how they got the information.
While pre-prepared key questions can benefit students and lead to rich classroom discussions, coming up with on-the-spot questions can be difficult.
The classroom discourse was important for statistical literacy. Students should be able to discuss and communicate their understanding and opinions to others. Most of our classroom activities included group and whole-class discussion of the data. This typically involved a small-group activity in which the students worked on problems together and then reported back to the whole class. The teachers took time to remind the students how to work in groups (e.g., how to agree and disagree and how to present to the class). Our results show that students can be taught how to question and challenge in respectful ways as part of classroom discourse.
Students found group work useful:
When you are working alone you just get one point of view and when you are working in a group you get different perspectives of other ideas … how other people are thinking, learning in class.
Oh … just because when we work alone we might get it right, we might get it wrong, but if we work in a group we’ll get more ideas. We will be able to discuss it with the group.
Since all statistical messages are conveyed through written text (e.g., newspapers) or oral text (e.g., television), the understanding of statistical messages requires the activation of various literacy skills. Some messages require students to do a combination of deciphering text and understanding abstract symbolic information, such as in tables, graphs and statistical measures. Statistics has a particular language: some statistical terms create problems because they are familiar from everyday discourse but take on a different meaning in statistics. Students are also required to communicate their opinions clearly, both orally and in writing. As one student commented:
Because usually, like in normal maths, we don’t use literacy … like, we use addition, subtraction but we actually have some kind of literacy for the things we do in statistics.
In the teaching experiment, support was provided for both reading and writing in statistics. Supports included vocabulary acquisition, pre-reading and reading strategies, such as shared reading and scanning techniques. Writing support included on a focus on speaking before writing, the use of writing frames and cloze activities and composing responses individually and in groups.
Context is an important component of statistical literacy. Our findings show that students need exposure to both familiar and unfamiliar contexts. While engagement with context helps students develop higher order thinking skills, our results showed that contextual knowledge was a barrier for some students. Teachers were able to address this in two ways. The first was to start from familiar contexts before moving to unfamiliar contexts. For example, using Statistics New Zealand’s interactive boundary map software (Statistics New Zealand, n.d.) allowed teachers to use statistical data from the student’s own neighbourhood before asking for analysis of regional or national data. The other was to use contexts of interest to the students, ranging from the popularity of names in the Baby Name Wizard (http://www.babynamewizard.com/voyager) through to addressing more challenging issues and questions generated from students’ own lives. This process involved handing over some of the control and planning of lessons to students as the direction could not always be dictated by the teacher.
An important aspect of our unit was to expose the students to innovative technological tools that could be used to explore data and to test conjectures by evaluating data. Students needed access to the Internet to find source data and other information relevant to the task. They also needed access to statistics software to produce alternative graphs.
Statistical literacy framework
In preparing for the teaching experiment, we conducted whole-class performance assessments with groups of Year 9 students from the same school in which we worked. Based on the student responses, we developed a four-stage framework to diagnose students’ thinking in statistical literacy. The framework is based on Watson’s (2006) statistical literacy construct. We have reduced the six levels identified by Watson to four stages. The boundaries between the stages are not hard edges but rather provide a set of stages that give a convenient way of describing changes as students progress to higher levels of thinking. The aim is to furnish teachers with a tool that can be used to assess students’ statistical literacy.
The four stages are:
Stage 0–1 Informal/idiosyncratic
Students at this stage are exhibiting characteristics of prestructural or at most unistructural thinking. There is only an informal engagement with context, often reflecting intuitive nonstatistical ideas and beliefs, and where students provide random or inappropriate explanations. When making inferences, students focus on imaginative storytelling or inappropriate aspects. Questions asked are not based on the data or are focused on irrelevant contextual issues. Students are successful at some basic table and graph reading.
Stage 2 Consistent noncritical
Students at this stage are exhibiting characteristics of unistructural and multistructural thinking. Students focus on a single, relevant aspect or attempt to attend to one or more relevant aspects of the data, but have difficulty in integrating the aspects.
There is appropriate but noncritical engagement with context. There is accurate use of statistical skills associated with simple statistics and graph characteristics. Questions asked are valid but based on one aspect of the data.
Stage 3 Early critical
Students at this stage are beginning to exhibit characteristics of relational thinking. They can attend to more than one relevant aspect of the data and are beginning to integrate the aspects. There is critical engagement in familiar contexts. There is selective engagement with unfamiliar contexts with some justification.
Students demonstrate awareness of relevant features of displays and measures of centre and spread, but these are primarily based on either the data or the context, not both.
Stage 4 Advanced critical
Students at this stage are integrating statistical and contextual knowledge that exhibits extended abstract thinking. There is a critical, questioning engagement with context.
There is an understanding of the purpose of the data, data displays, measures of centre and inferences made. There is a critical evaluation of data collection methods, choice of measures and validity of findings that shows an appreciation of variation and the need to indicate the level of uncertainty when making predictions. Questions asked are based on relevant features of the data and the context using multiple perspectives.
Our use of the framework
Students were assessed at different stages of statistical literacy, from idiosyncratic nonstatistical reasoning through to critical statistical literacy. Our findings showed that students were at different stages. It was possible to progress them to the next stages of statistical literacy when teachers recognised the level that students were thinking at and then responded with appropriate support and questioning.
Limitations of the study
Design research is often criticised for its methods. The limitations can relate to technical and human aspects. On the technical side, the recording devices used in the study may not have captured everything that was said by the students and the teachers. Additionally, some students may have withheld their responses because they did not want their ideas captured on tape. On the human side, interview data may be subjective, and hence there are limitations associated with reliability. Students’ views, during interviews in particular, may have been influenced by our unequal relationship. As teacher-researchers we also assessed their work, so during the interviews students may have said things they thought their teachers wanted to hear. Another human limitation relates to researcher prejudices and biases. Since we were both the practitioners and the researchers, data collection and analysis could have been affected by our predispositions and biases.
To counter the above shortcomings, the research team collected data from a variety of sources using different data collection methods. This triangulation helped ensure the consistency of research findings. Trustworthiness was also achieved by subjecting all aspects of the research design to scrutiny and critique by colleagues.
Implications for teaching and research
Our findings have a number of implications for teaching.
The statistical literacy framework documented in this study can enable teachers to trace students’ individual and collective development in statistical literacy during teaching. The framework provides useful information regarding the type of statistical literacy that can be expected at different levels.
As well as statistical knowledge, literacy knowledge and skills are important for statistical literacy. Teachers need to help students access information. Pre-reading and vocabulary strategies can be used. Writing frames and prompts can be used to promote writing. Students can use words in a sentence and explain the meaning of the word to a partner. Students then read their sentences to class. The teacher can write some sentences on a whiteboard. As a class, they then classify the usage as everyday or statistical.
Context plays a particularly key role in the development of statistical literacy. Students need exposure to both familiar and unfamiliar contexts. Engagement with context helps students develop higher order thinking skills. Our results show that contextual knowledge may be a barrier for students. Students’ motivation towards statistical literacy could be influenced by the context in which the tasks are embedded. Teachers need to provide opportunities for students to work with real data and choose contexts that suit the specific needs of their students.
To discourage students from becoming too sceptical about statistics, it is important to provide examples where statistics are used correctly, not just to show the bad examples. For example, students could use the interactive boundary maps data from Statistics New Zealand (n.d.) to write a report that profiles their local neighbourhood. The interactive boundary maps show facts about geographic areas in New Zealand, including the number of people and households. All data displayed in the maps come from the 2006 Census, so the data are from a reliable and trusted source. The statements made are accurate and claims are supported. This is an opportunity to use good statistics in the classroom. The context is the student’s neighbourhood. This familiar context allows students to integrate contextual knowledge into statements that they make while minimising the need for students to make idiosyncratic or unsupported comments. Instead they can focus on making evidence-based comments. Students could ask more searching questions, such as “What are the (political, equity, environmental, social justice ...) implications of this?”
Classroom discussions are important for helping students to develop statistical literacy. Ample class time should be spent on discussion and reflection rather than presentation of information. It appears that the nature of the learning environment and classroom culture are major contributors to success for students, and teachers need to put a high priority on building a classroom climate that positively engages all students. Students need to understand the importance of sharing their opinions in order to advance their statistical ideas. It would be valuable for teachers to help students reflect on the purposes of explaining and justifying their thinking to others. This is consistent with the latest New Zealand curriculum document that promotes the ideals of having confident, critical and active learners of mathematics (Ministry of Education, 2007, p. 12).
The teaching activities that we developed in our study involved students interpreting and critically evaluating statements and reports made by others and communicating their decisions to peers. We also made sure that significant statistical ideas emerged as the focus of the conversations during group and whole-class discussions of students’ evaluations. The teachers viewed various datābased arguments the students produced as they completed the teaching activities as a primary source on which they could draw to initiate and guide whole-class discussions. By referring to the diverse ways in which the students had interpreted the data, the teachers guided the direction of discussions. In the case of the school survey activity (see Figure 4), for example, the issues that emerged as an explicit topic of discussion during the subsequent whole-class discussion included the contrast between small and large sample sizes, random sampling variation and missing information.
Scenario
A class wanted to raise money for their school trip to Rainbow’s End. They could raise money by selling raffle tickets for a game system. But before they decided to have a raffle they wanted to estimate how many students in their whole school would buy a ticket. So they decided to do a survey to find out first. The school has 700 students in Years 7–13, with 100 students in each level.
Seven students in the school conducted surveys.
1.Amy got the names of all 700 students in the school and put them in a hat and then pulled out 70 of them.
2.Mica asked 70 students at an after-school meeting of the computer games club.
3.Adrian asked all 100 children in Year 9.
4.Ravi surveyed 10 of his friends.
5.Claire set up a booth outside the tuck shop. Anyone who wanted to stop and fill out a survey could. She stopped collecting surveys when she got 60 kids to complete them.
6.Sila sent out a questionnaire to every kid in the school and then used the first 70 that were returned to him.
7.Joshua put the names of all the Year 7 boys in one hat and all the Year 7 girls in another hat. He pulled the names of five boys and five girls from each hat. He did the same thing for each year level until he had the names of five boys and five girls from each year level.
Who do you think has the best survey method?
Why do you think so?
The research team developed a statistical literacy poster, “Questioning the Data Detective”, to help students evaluate statistical reports and articles. The team revised this poster in light of the feedback received. This poster and other such critical thinking routines may prove useful resources for teachers working with interpreting the statistical literacy achievement objectives from The New Zealand Curriculum (Ministry of Education, 2007). We recommend that this type of critical questioning should be introduced in schools as there is a need for students to begin to question statistical reports at an early age.
Views about statistics teaching and learning have shifted considerably in New Zealand and internationally over recent decades, and it is important for teachers to be kept informed about changes in the ways that mathematical and statistical processes and thinking are being emphasised (Anthony & Walshaw, 2007). Our design research study has identified ways that teachers can encourage students to improve their statistical literacy and critical thinking. Not only does this help with statistics, but learning the process of critical thinking could potentially expand to other curriculum subjects and wider society.
Acknowledgement
This research project, “Developing secondary school students’ understanding of statistical literacy in a datāanalysis environment”, was funded by the Teaching and Learning Research Initiative and carried out at the University of Waikato.
References
Anthony, A., & Walshaw, M. (2007). Effective pedagogy in mathematics/pāngarau: Best evidence synthesis iteration [BES]. Wellington: Ministry of Education.
Begg, A., Pfannkuch, M., Camden, M., Hughes, P., Noble, A., & Wild, C. (2004). The school statistics curriculum: Statistics and probability education literature review. Auckland: Auckland Uniservices, University of Auckland.
Budgett, S., & Pfannkuch, M. (2010). Using media reports to promote statistical literacy for non-quantitative major. In C. Reading (Ed.), Data and context in statistics education: Towards an evidence-based society: Proceedings of the Eighth International Conference on Teaching Statistics (ICOTS8, July, 2010), Ljubljana, Slovenia. Retrieved from http://www.stat.auckland.ac.nz/~iase/publications/icots8/ICOTS8_7G1_BUDGETT.pdf
Chick, H., Pfannkuch, M., & Watson, J. (2005). Transnumerative thinking: Finding and telling stories within data. Curriculum Matters, 1, 86–107.
Cobb, P. (2000). Conducting teaching experiments in collaboration with teachers. In A. Kelly & R. Lesh (Eds.), Handbook of research design in mathematics and science (pp. 307–333). Mahwah, NJ: Lawrence Erlbaum.
Gal, I. (2004). Statistical literacy: Meanings, components, responsibilities. In J. Garfield & D. Ben-Zvi (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 47–78). Dordrecht, Netherlands: Kluwer.
Left hand in IQ tests, dealt a bad hand in life. (2008, 3 December). New Zealand Herald. Retrieved 21 March 2011, from http://www.nzherald.co.nz/lifestyle/news/article.cfm?c_id=6&objectid=10546184
Lenhart, A., Ling, R., Campbell, S., & Purcell, K. (2010, 20 April). Teens and mobile phones (Chapter 2). Retrieved 5 March 2011, from Pew Internet: http://www.pewinternet.org/Reports/2010/Teens-and-Mobile-Phones/Chapter-2/The-typical-American-teen-who-texts-sends-1500-texts-āmonth.aspx
Ministry of Education. (2007). The New Zealand curriculum. Wellington: Learning Media.
Rossman, A., Chance, B., & Medina, E. (2006). Some important comparisons between statistics and mathematics, and why teachers should care. In G. Burrill & P. C. Elliot (Eds.), Thinking and reasoning with data and chance (pp. 323–333). Reston, VA: The National Council of Teachers of Mathematics.
Sharma, S. (2007). Exploring pre-service teachers’ understanding of statistical variation: Implications for teaching and research. Australian Senior Mathematics Journal, 21(2), 31–43.
Shaughnessy, J. M. (2007). Research on statistics learning and reasoning. In F. K. Lester, Jr. (Ed.), Second handbook of research on mathematics teaching and learning (pp. 957–1009). Reston, VA: The National Council of Teachers of Mathematics.
Statistics New Zealand. (n.d.). Interactive boundary maps. Retrieved 22 March 2011, from http://apps.nowwhere.com.au/StatsNZ/Maps/default.aspx
Watson, J. M. (2006). Statistical literacy at school: Growth and goals. Mahwah, NJ: Lawrence Erlbaum.
Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry (with discussion). International Statistical Review, 67(3), 223–265.
SASHI SHARMA is a senior lecturer, Department of Mathematics, Science and Technology Education at The University of Waikato. Her research interests include statistics education, language and cultural issues and teacher education.
Email: sashi@waikato.ac.nz
PHIL DOYLE is a mathematics facilitator at Team Solutions—The University of Auckland.
Email: p.doyle@auckland.ac.nz
VINEY SHANDIL is the Assistant Head of Department of Mathematics at Marcellin College.
Email: shandilv@marcellin.school.nz
SEMISI TALAKIA'ATU is a mathematics teacher at Marcellin College.