You are here

A uniquely Aotearoa-informed approach to evaluating information using the Rauru Whakarare Evaluation Framework

Angela Feekery and Carla Jeffrey
Abstract: 

Evaluating information quality is a key skill students need to develop as they navigate the complex information landscape. Students need to develop an awareness of effective ways to evaluate information given the abundance of information (and misinformation) available online. This article introduces the Rauru Whakarare Evaluation Framework, which offers a kaupapa Māori-informed perspective designed to foster deeper engagement with the information evaluation process in secondary and tertiary institutions within Aotearoa New Zealand.

Journal issue: 

A uniquely Aotearoa-informed approach to evaluating information
using the Rauru Whakarare Evaluation Framework

Angela Feekery and Carla Jeffrey

Key points

Our students need support to develop critical information evaluation strategies to navigate our complex information landscape.

Common evaluation checklists fail to draw on rich indigenous connections that may encourage deeper engagement with quality information.

Our new Māori-informed holistic evaluation framework accommodates other ways of knowing and encourages all teachers and students to think differently about engaging with information.

Evaluating information quality is a key skill students need to develop as they navigate the complex information landscape. Students need to develop an awareness of effective ways to evaluate information given the abundance of information (and misinformation) available online. This article introduces the Rauru Whakarare Evaluation Framework, which offers a kaupapa Māori-informed perspective designed to foster deeper engagement with the information evaluation process in secondary and tertiary institutions within Aotearoa New Zealand.

Why don’t my students use better quality information?

Are you frustrated by the poor quality information students are using in assessments?

Have you observed that when you ask students to research a topic, quick Google searches are often the start and end point of the research process?

Do you find that your students (and perhaps yourself) struggle to navigate the unfiltered online information landscape where misinformation and fake news are rife?

Are your students weak at determining information quality?

If the answer to these questions is “yes”, then you may find our research insights useful. Our research team is focused on enhancing students’ information literacy capability in the senior secondary school and in first-year tertiary education. In our research, we define information literacy as involving “the processes, strategies, skills, competencies, expertise and ways of thinking which enable individuals to engage with information to learn across a range of platforms (both digital and traditional learning environments), transform the known, and discover the unknown” (Feekery, 2016, para 6). We are exploring the information literacy space by engaging in partnerships between teachers and librarians. Our aim is to ensure that students are exposed to new ways of thinking about information and that students are developing skills to become effective consumers and creators of quality information (Emerson, Kilpin, & Feekery, 2015a, 2015b).

Part of being information literate is being able to evaluate information quality. Our research on students’ information seeking and evaluation behaviours has revealed that many students lack a clear process for determining the quality of information they use to support their learning. Our secondary schools survey also indicates that teachers feel information evaluation is an important skill students need to develop to be successful learners.

This article introduces a new approach to evaluating information, the Rauru Whakarare Evaluation Framework, designed by the author team, Angela, an information literacy researcher, and Carla, a Māori librarian at Massey University, in consultation with Sheeanda McKeagg (Massey University Kaihautu Māori/Māori Services Manager) and Hinerangi Kara (Waikato University, Te Kaitakawaenga Māori/Māori Academic Liaison Librarian).

With this framework, we offer an alternative holistic approach to information evaluation that we envisage will encourage students to critically engage and make better decisions when selecting information to support their learning.

Why is evaluating information so challenging?

We know that effective source selection is determined by understanding that different types of information are created for different purposes, audiences, and contexts. Yet, students may find it difficult to determine the validity and credibility of electronic sources (Brabazon, 2006; Coonan, 2011). They often struggle to distinguish popular from scholarly information on the internet (McCartin & Feid, 2001). Being able to determine the relevance of a source for a particular task, and knowing what to reject, is also challenging (Head & Eisenberg, 2010).

In the pre-Google era, it was common for teachers to provide the bulk of material required in courses, and library catalogues and books were the primary source of supplementary information. Now we see increased expectations for students to access the majority of materials for assessments independently. Thus, developing an awareness of effective ways to evaluate information is especially important in light of the complexity and abundance of information available through the internet and library databases.

The diversity and breadth of online material inevitably creates issues of quality (Dalgleish & Hall, 2000). We often see students’ reference lists full of websites rather than quality sources that show a deeper engagement with a topic. This is problematic as many websites have a commercial focus and provide limited research-informed evidence compared to information produced by academic or professional experts.

As the volume of unfiltered information available online continues to increase, random internet searching for sources, and selection based on accessibility, will remain key concerns for secondary and tertiary educators. Tara Brabazon (2006), a scholar who writes about the negative impacts of the internet on learning, argues that educators need to (re)teach how to evaluate quality to limit unquestioning selection and acceptance of information sourced via Google. She stresses that “finding information is not synonymous with understanding information” (p. 163) and increased access to information does not necessarily promote high-quality research and writing.

Why do we need a new approach to evaluating information?

Teachers may already have in their kete a tool to teach information evaluation skills, or they might rely on school or university librarians to teach this for them. If information evaluation skills are explicitly taught, a common approach is using checklists that draw on common key criteria. Such criteria include: accuracy; author/authority; audience; bias (objectivity); clarity; coverage; credibility; currency; evidence; place published; purpose; reliability; and relevance. Common examples are the CARS (Credibility, Accuracy, Reasonableness, Support) checklist (Harris, 2018) and CRAP (Currency, Reliability, Authority, Purpose) test (Orenic, 2008).

The various information-evaluation checklists available online and through libraries present a number of problems. The first is that they tend to take a linear approach focused on individual criteria. Students are often focused on ease of access and surface credibility determined by using these checklists; therefore, they struggle to determine genuine credibility, relevance, or authority (Meszaros, 2010). Linear-evaluation approaches do not align with our more complex understanding of evaluation captured within the current holistic views of information literacy emerging in the new millennium (Association of College and Research Libraries [ACRL], 2015; Feekery, 2013, 2016; Secker & Coonan, 2013; Ward, 2006). These views see information literacy as a way of learning that fosters deeper engagement with information use by highlighting the importance of critical awareness of the origin of information and the purpose for which it was created.

“As the volume of unfiltered information available online continues to increase, random internet searching for sources, and selection based on accessibility, will remain key concerns for secondary and tertiary educators.”

A second and more important issue is that these linear models fail to draw on rich indigenous connections and kaupapa that may encourage deeper engagement with quality information. No such models specifically informed by kaupapa Māori principles exist within Aotearoa, and we have been unable to identify other indigenous approaches being adopted internationally.

To address this gap, the authors developed the Rauru Whakarare Evaluation Framework, emerging from a te ao Māori view and informed by a kaupapa Māori perspective, a “philosophical doctrine, incorporating knowledge, skills, attitudes and values of Māori society” (Moorfield, 2019, para 1). Taking such an approach normalises and legitimises Māori culture and knowledge (Smith, 2012).

The rauru whakarare pattern and the five Māori terms we adopted (outlined below) were identified as key concepts to underpin the new information-evaluation framework. It has been designed to foster a deeper engagement with the information-evaluation process in secondary and tertiary institutions within Aotearoa New Zealand.

The rauru whakarare pattern and concepts

The rauru is a spiral pattern used in Māori carving, commonly in the form of a single spiral (see Phillipps, 1948, for more information on Māori spirals). The rauru whakarare pattern is made up of smaller parts that can connect to make a unique pattern. It signifies interconnectedness, with jagged edges to emphasise that the process is not always smooth; the pattern only makes sense when all parts are considered together. This metaphor highlights the complex nature of the information-evaluation process and provides us with a means to engage critically and reflectively with the information-evaluation process.

Figure 1: Rauru whakarare pattern and framework concepts

We recognised that integrating Māori concepts into an information-evaluation framework creates a more meaningful, holistic view of the information-evaluation process than Western concepts allow. Māori terminology contains embedded spirituality and depth that often cannot be captured by a literal English translation. An integrated approach incorporating both Western and Māori ways of understanding what knowledge is, and how information is created and disseminated, moves us towards the third space of understanding between two knowledge domains, as advocated for by Brian Tweed (2016), who explored connection and tension between mātauranga Māori and mathematics curriculum when teaching Māori students.

The Rauru Whakarare Evaluation Framework embodies the connectedness of whakapapa (background), orokohanga (origins), mana (authority), māramatanga (content), and aronga (lens) of information we are accessing. These terms are not literal translations from English words, but were chosen to align with and extend the meaning of commonly used evaluation criteria. For example, New Zealanders understand mana comes from earned respect rather than credentials demanding authority, as is often the case in Western contexts.

These five key concepts are graphically represented with whakapapa at the heart of determining information quality. The framework aligns the concepts with evaluation criteria and questions found in the more common linear evaluation checklists. The concepts are as follows.

1.Whakapapa identifies and connects the various layers to consider when evaluating sources. It captures the source’s pedigree and connects it both to the topic and to the other sources being used. It asks questions about why the source was published, who it was created for, what the information context is, and how relevant it is to the Aotearoa context.

2.Orokohanga considers the source’s origin. Questions focus on where and when the information was published, and its currency in terms of date published and value to the discipline or profession.

3.Mana connects to the author or organisation’s credibility or standing within the community. We ask, “Why should I believe and trust in the views, values, and ideals of the creator of this information?” Mana also connects to the content and language accuracy in the source (e.g., the use of inclusive language, and no grammatical or spelling errors).

4.Māramatanga means “enlightenment” and emphasises that the source should positively impact the wider community of understanding. Māramatanga considers the appropriateness of the information for our purpose, audience, and context. It should add value to conversations taking place within a particular discipline area. It connects to understanding, usability, and relevance of the source. Information may have quality whakapapa, orokohanga, and mana, but we should also consider how closely it relates to our information need (e.g., the topic of an assignment).

5.Aronga represents the lens we use when looking at information sources. It also means considering the information creator’s lens and the impact of this on the source’s mana. Aronga connects to “perspective” or “direction”. It enhances our ability to recognise the information creator’s bias and perspectives. It also allows us to judge whether the authors are open to viewing a topic or issue through a different lens and recognise the validity of other perspectives, even if they don’t support them. (Feekery, 2018)

Figure 2: The Rauru Whakarare Evaluation Framework

“Within this framework, determining information quality is not governed by linear hierarchical principles, and each concept can be considered in various orders depending on the context and purpose of the information use.”

Within this framework, determining information quality is not governed by linear hierarchical principles, and each concept can be considered in various orders depending on the context and purpose of the information use. This framework encourages students to consider a range of usually individually unique characteristics as a whole. Using this kaupapa means students can start at any point and see how all ideas connect together while considering the contexts in which the information was originally produced and how it is currently being used for their task or purpose. Detailed descriptors for the framework concepts are provided on our website (www.informationliteracyspaces.wordpress.com).

How can the framework be used in practice?

We hope this kaupapa enables teachers, librarians, and students to shift from a tick-box exercise to a deeper, engaged, and instinctive way of approaching information evaluation. Our aim now is to introduce the framework to schools to foster a seamless, culturally appropriate transition between secondary and tertiary study. It is timely given the Ministry of Education’s change initiatives designed to foster equal value for mātauranga Māori (Māori knowledge) alongside other bodies of knowledge (Education Conversation, 2019).

We initially used this kaupapa in our university teaching as part of an online learning module focused on exploring professional information sources in business communication for first-year students. It has now been adopted widely in our university library introduction workshops for a range of disciplines, including health, science, humanities, and veterinary science.

We have already had several secondary schools’ and other tertiary institutions’ teachers and librarians ask to use the framework, so we are excited to see how the framework is adopted or adapted for various Aotearoa educational contexts. The National Library of New Zealand is also currently exploring ways in which the framework could be used in professional development with teachers and librarians in schools. We will be researching the continued response to the framework over the next 18 months.

By sharing this framework, we want to encourage educational institutions across Aotearoa to adopt a consistent approach and language to use when talking to students about information quality. Ideally, the whole school would engage in discussions about whether this framework can be adopted school-wide to provide a consistent message about information evaluation as soon as students enter high school (rather than using the kaupapa for one task, class, discipline, or year level).

We believe this approach would provide a solid foundation for students to inform their learning by selecting and using quality information sources. When students enter tertiary education, no consistent messages or training may be given on this vital aspect of learning because expectations are that students will have developed information literacy skills through their senior secondary years. Even for students not pursuing tertiary education, the skill for evaluating quality information will carry over into other aspects of their lives. Thus, the framework is not just constrained to use in educational contexts.

“By sharing this framework, we want to encourage educational institutions across Aotearoa to adopt a consistent approach and language to use when talking to students about information quality.”

We would encourage schools to make the Rauru Whakarare Evaluation Framework central to their whole-of-school information literacy policy, and to put in place strategies and necessary professional development to enable teachers (in all disciplines) and librarians to integrate the framework seamlessly into their curriculum.

While this kaupapa works within an Aotearoa context, we hope this will be a starting point for other indigenous groups to adapt the framework and adopt meaningful terms relevant to their diverse contexts.

As both authors work in the tertiary sector and the framework is so new, we lack specific insights for how the framework could be integrated into secondary school contexts. We will be exploring how the framework is being used in schools over the next 12–18 months and will be able to share our findings with you in a future article. As a starting point, we can offer the following suggestions for secondary-school contexts.

Librarians

Librarians are often the expert on information searching and evaluation processes. You could:

create or refocus library workshops on information evaluation processes for students that will introduce the Rauru Whakarare Evaluation Framework early in the high school learning experience

support teachers to explore the information literacy landscape within their discipline area. Work through the framework with some key sources teachers provide to students so the evaluation process can be modelled in the classroom.

Teachers

Teachers create the contexts in which information is explored and used. You could:

explore the Rauru Whakarare Evaluation Framework criteria in relation to disciplinary needs and be aware of the information types students need to access

use the framework to evaluate any information you provide to your students, and make this part of the conversation on source selection

consider how the framework can be applied as a working method within the structure of specific NCEA research-oriented assessment standards

consider ways to shift students away from content-focused information selection (“I used it because it says what I need it to”), towards developing a habit of focused evaluation when independently accessing information (“I can justify the use of this source because I strategically determined it is good quality, relevant information”).

“We welcome you to join us in our ongoing conversation about the framework and any other information literacy-related insights on our Information Literacy Spaces website and Facebook group.”

In this article, we have offered the Rauru Whakarare Evaluation Framework as a kaupapa Māori-informed approach to critiquing and engaging deeply with the abundance of information teachers and students can access. We have outlined opportunities for its use in secondary and tertiary learning contexts, and we are open to suggestions about whether the framework could be used in kura kaupapa or primary and intermediate school contexts as well. The framework is currently being translated into te reo Māori and will be shared on our website when available.

We would love to hear your thoughts on the framework and be informed about whether you choose to adopt it or how you adapt it for your learning contexts. We welcome you to join us in our ongoing conversation about the framework and any other information literacy-related insights on our Information Literacy Spaces website and Facebook group.

Acknowledgement

The model is available though a creative commons licence: Rauru Whakarare Information Evaluation Framework by Angela Feekery and Carla Jeffrey is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence. So, feel free to use the model and let us know how you are engaging with it by contacting us through our project website, Information Literacy Spaces.

This framework was developed as part of a TLRI-funded initiative (see http://www.tlri.org.nz/tlri-research/research-progress/cross-sector/transforming-information-literacy-spaces-support). For more information about our project, our website https://informationliteracyspaces.wordpress.com/ includes blogs by teachers and librarians engaged with our research and also provides a way of contacting us. You can also follow and engage with us on conversations on Facebook (https://www.facebook.com/ILspaces/).

We would like to thank reviewer Professor Lisa Emerson, Massey University, along with the anonymous peer reviewers.

References

Association of College and Research Libraries. (2015). Framework for information literacy in higher education. Retrieved from http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/infolit/framework1.pdf

Brabazon, T. (2006). The Google effect: Googling, blogging, wikis and the flattening of expertise. Libri, 56(3), 157–167.

Coonan, E. (2011). A new curriculum for information literacy: Teaching learning: Perceptions of information literacy: Theoretical background. Retrieved from http://ccfil.pbworks.com/f/emma_report_final.pdf

Dalgleish, A., & Hall, R. (2000). Uses and perceptions of the world wide web in an information-seeking environment. Journal of Librarianship and Information Science, 32(3), pp. 104–116.

Education Conversation. (2019). NCEA change package 2019 overview. Retrieved from http://www.conversation.education.govt.nz/assets/Uploads/NCEA-Change-Package-2019-Web.pdf

Emerson, L., Kilpin, K., & Feekery, A. (2015a). Let’s talk about literacy: Preparing students for the transition to tertiary learning. Set: Research Information for Teachers, (1), 3–8.

Emerson, L., Kilpin, K., & Feekery, A. J. (2015b). Enabling academic literacy: Smoothing the transition to tertiary learning [Teaching and Learning Research Initiative]. Retrieved from http://www.tlri.org.nz/

Feekery, A. J. (2013). Conversation and change: Integrating information literacy to support learning in the NZ tertiary context. Doctoral thesis, Massey University, Manawatu, New Zealand.

Feekery, A. J. (2016). Conceptualising the information literacy space [Blog post]. Retrieved from https://informationliteracyspaces.wordpress.com/2016/09/29/conceptualising-the-information-literacy-space/

Feekery, A. J. (2018). A uniquely Aotearoa-informed approach to evaluating information [Blog post]. Retrieved from https://informationliteracyspaces.wordpress.com/2018/07/26/a-uniquely-aotearoa-new-zealand-informed-approach-to-evaluating-information/

Harris, R. (2018). Evaluating internet research sources. Retrieved from http://www.virtualsalt.com/evalu8it.htm

Head, A. J., & Eisenberg, M. B. (2010). Truth be told: How college students evaluate and use information in the digital age. Project Information Literacy Progress Report. Washington, D.C.: University of Washington.

McCartin, M., & Feid, P. (2001). Information literacy for undergraduates: Where have we been and where are we going? Advances in Librarianship, 25, 1–27.

Meszaros, M. (2010). Who’s in charge here? Authority, authoritativeness, and the undergraduate researcher. Communications in Information Literacy, 4(1), 5–11.

Moorfield, J. (2019). Kaupapa Māori. Te aka online Māori dictionary. Retrieved from https://maoridictionary.co.nz

Orenic, K. (2008). The CRAP test. Retrieved from http://loex2008collaborate.pbworks.com/w/page/18686701/The%20CRAP%20Test

Phillipps, W. J. (1948). Māori spirals. Journal of the Polynesian Society, 57(1), 30–40.

Secker, J., & Coonan, E. (2013). Rethinking information literacy: A practical framework for supporting learning. London, UK: Facet Publishing.

Smith, G. (2012). Interview: Kaupapa Māori—The dangers of domestication. New Zealand Journal of Educational Studies, 47(2), 10–20.

Tweed, B. (2016). Tātai kōrero i ngaro, tātai kōrero e rangona. Legitimation and the learning of curriculum mathematics in an indigenous Māori school. Doctoral thesis, Victoria University of Wellington, Wellington, New Zealand.

Ward, D. (2006). Revisioning information literacy for lifelong meaning. The Journal of Academic Librarianship, 32(4), 396–402.

Dr Angela Feekery is a senior lecturer in communication at Massey University. She is passionate about enhancing students’ information literacy capability and understanding how educators can better support transition into tertiary learning.

Email: A.J.Feekery@massey.ac.nz

Carla Jeffrey is a business subject librarian at Te Kunenga ki Pūrehuroa, Massey University. She is a member of Te Rōpū Whakahau, the national body that represents Māori who work in the information profession sector. She is of Ngāi Tūhoe descent and is passionate about ensuring students succeed at their studies.