On or off screen: Reading in a digital age

On or off screen

Reading in a digital world

Jan Eyre

The alarm startles me from sleep. I heave myself out of bed to prepare for the day. Over breakfast, I check my smartphone, scanning the notifications for any important messages or interesting tidbits that have come in overnight. I flick to a news app to skim the headlines, diving deeper into any articles that catch my attention. Once on the train, I grab a book or an article from my backpack and immerse myself in it for the journey. At work, I open up my computer and scan my emails, deleting those I don’t need, prioritising the rest, or storing them in folders. I find a link to an interesting article and print it off to read later. I do some work on my computer, referring to the papers on my desk as well as documents stored online. At morning tea I grab the newspaper from the staffroom and browse it while I eat. Over a few hours, I have done a lot of reading, switching between paper and screen many times, making largely automatic decisions about which mode I prefer. And that’s all before 11 a.m.!

My work is mostly office based and I spend a lot of my time at a desk in front of a computer. Reading is part of my work. But I think my experience of switching between reading on screen and reading on paper many times during the day is a common one. We live in a digital world with anytime, anywhere access to a plethora of information on screen. Yet paper-based reading is still important, and most of us engage in it fairly regularly.

When I think about my choices between paper and screen, I realise they have a lot to do with what I’m reading and why I’m reading it. I prefer paper for anything that requires a period of focused, sustained reading, while screen reading is for short sharp bursts of information. If I start reading something online and realise I need to give it more thought, I frequently save it, or print it off to return to later.

I am not alone in this. Ask most people whether reading on screen is the same as reading on paper, and you will get an emphatic “No!” Probe a little deeper, and you will find that this “no” relates to a particular kind of reading: longer passages of multiple-page writing, such as articles and books. Even people with high levels of digital expertise often prefer to print off articles or will choose a paper version of a book rather than an e-reader. The reasons people give for this preference commonly include comments about the tangibility of paper (being able to flip backwards and forwards between pages; the feel of the paper in your fingers), and the ability to highlight sections of the text and make notes in the margin when reading an article.

It seems that many of us prefer to do a particular kind of reading on paper—the kind of reading that demands focused and sustained attention over multiple paragraphs of text, where we need to follow the complexities of a storyline or an argument and engage critically or emotionally with the ideas it puts forward. This kind of reading demands “deep” processing, in contrast with the surface-level skimming and scanning that is often typical of our screen reading (McNamara, 2007).

What does the research say?

There have been many research studies that compare reading online and reading on paper (Noyes & Garland, 2008; Mayes, Sims, & Koonce, 2001; Mangen, Walgermo, & Brønnick, 2013). The results have been variable, with some studies favouring online reading, some favouring paper-based reading, and some finding no difference between the two. It is hard to generalise from these studies, since technology is advancing so quickly and the kinds of texts and digital devices used in the studies varied widely. In this article, I will concentrate on some of the studies that might help to explain our preference for using paper for deep processing.

Wästlund, Reinikka, Norlander and Archer (2005) found that when participants read five different texts, each of around 1,000 words, then answered multiple-choice comprehension questions, their comprehension was lower when the texts were read online, compared to when they were read on paper. Those reading on computer also reported higher levels of tiredness and stress than those reading on paper.

Dr Anne Mangen of the National Center for Reading Education and Research at the University of Stavanger, Norway, has carried out a range of research on the effects of digitisation on the reading process. A recent study involved young adults reading one narrative and one expository text, each of around 1,500 words, on paper or on a computer, and then answering comprehension questions. Again, the results showed that students reading the texts on a computer screen scored lower than those who read the printed texts, this time on three categories of reading comprehension: access and retrieve, integrate and interpret, and reflect and evaluate (Mangen et al., 2013).

So it seems that when we read longer passages of text on screen, our comprehension of them may be lower than if we had read them on paper. But what about young people, the digital natives who have grown up in a digital world? Surely by dint of growing up in the era of digital technology they will be equally capable at reading on screen as they are on paper? It seems that this might not be so. Kerr and Symons (2006) conducted a study with 9- to 10-year-olds to see if their text recall and comprehension were affected when informational text was presented on computer. Sixty students read two passages of text, either on computer or on paper, and answered verbal questions. The results showed that children read more slowly on computer, and that they were more efficient at comprehending informational text that they had read on paper. One conclusion was that “higher-order reading skills may be affected by computer presentation of text” (Kerr & Symons, 2006, p. 14).

Of course, technology has moved on apace in the 11 years since Kerr and Symons’ study. It would be reasonable to assume that children today have greater exposure to digital technology and so are more used to reading on screen. Because of our commitment to research-based assessment, NZCER is deeply interested in these questions around onscreen reading. To investigate further, we recently completed a study that compared the results of the Progressive Achievement Test in Reading Comprehension (PAT: Reading Comprehension) in online and paper-based modes. We wanted to find out whether there were differences in the way that the individual questions functioned on screen and on paper, and whether student scores were equivalent across paper and online formats. To do this, we analysed around 200,000 assessment records.

PAT: Reading Comprehension is a standardised assessment developed for use in New Zealand schools. It is designed to provide formative information about students’ ability to make meaning from written text. There are seven different tests, each targeted at a specific year level from Year 4 to Year 10 (Test 1 targets Year 4, Test 2 targets Year 5, and so on). Each test consists of a series of reading passages and associated multiple-choice comprehension questions. The texts include poems, narratives, and transactional texts.

When we designed the online version of PAT: Reading Comprehension, we took great care to minimise the differences in format between the online and paper-based versions, and to control the factors that are thought to interfere with reading online. For example, we designed the online interface so passages of text could be read with minimal scrolling, and we provided options to alter the font and background colours to aid reading onscreen. In our analysis, we could see that this careful work has paid off: we found that the individual questions functioned just the same in an online format as they did on paper. Students tended to find the same questions easier or more difficult, regardless of mode.

So, both versions of the text offer robust and reliable evidence about students’ reading comprehension. But here is where it gets really interesting. We also found that, overall, students’ scores tended to be lower when they took the test online (up to four scale points, which typically represents around 6 months’ progress). The effect was slightly larger for Māori students and for students in decile 1 and 2 schools (Eyre, Berg, Mazengarb, & Lawes, 2017).

So what is going on? It seems from the results of these research studies that there is, indeed, something about the experience of reading texts online that affects reading comprehension.

What’s different about reading online?

Can we try and unpick what it is about the experience of reading online that makes it more difficult to engage in deep and focused reading? Research that investigates onscreen reading has given us some pointers. Firstly, reading online is more physically tiring than reading on paper. Our eyes work harder when we’re reading onscreen (Wästlund et al., 2005). Tiredness may also be the result of increased cognitive load when reading online. We have to attend both to the reading material and the demands of the digital interface (for example, when entering answers), resulting in a “dual task” situation (Wästlund et al., 2005, p. 390). As we tire and our concentration fades, our attention may wander.

Secondly, when we read online it is harder to make a mental map of the text (Mangen et al., 2013)—there are fewer spatial clues. For example, when reading a paper-based text, we might remember that a particular point that interested us was at the top of the second page. This mental map helps us to get an overview of the “shape” of the text and the way that the ideas are linked. On screen, especially where scrolling is involved, it is more difficult to form this mental map.

It seems from the results of these research studies that there is, indeed, something about the experience of reading texts online that affects reading comprehension.

Mangen (2008, p. 408) writes about the “intangibility” of online text, referring to its lack of visual and tactile clues. When talking about a preference for reading on paper, many people mention making notes, highlighting, or writing in the margins of a text. It seems this process of engaging with the text in a physical way helps us to make connections and process its deep ideas. Although there are an increasing number of technologies available that allow us to annotate and highlight digital texts, making actual marks on a piece of paper helps some of us to connect in a visceral way that is somehow connected with learning. Writing our thoughts by hand in the margins of a text helps us process the ideas deeply. In the words of the Literacy Learning Progressions, we are “writing to think about ... ideas” (Ministry of Education, 2010, p. 6).

Another reason cited by research for differences in screen and paper comprehension is our reliance on the bells and whistles of online media. When we access material online, especially material that has been designed specifically for an online format, we usually have the ability to engage with it interactively. We can click on hyperlinks, perhaps play an audio or video clip, or move on to another screen. In short, we are used to being able to plot our own path through the material, skipping and jumping from screen to screen, hunting for the information we need and being diverted down other paths. I’m sure I’m not the only one who goes onto the web for a specific purpose and emerges an hour later after following my nose through a fascinating rabbit warren of information. All this is both highly engaging and stimulating, offering the opportunity to make creative links across a diverse and broad range of information, but it is not the same as linear reading. With linear texts, we work through material from top to bottom in an orderly manner, building up connections between the main ideas as we go.

It’s possible that we bring expectations of being able to plot our own path to any online reading situation. This means that, when faced with an online reading assessment, we are subconsciously looking for the next interactive feature, the next way to change the screen, or the next path to follow. The result is a kind of divided attention, with part of the brain focused on the reading, and the other part focused on what we can move on to next.

In a similar vein, eye-tracking studies have shown that when we read online material, we engage in more skimming and scanning than we do when reading a print-based text (Neilsen, 2006). We tend to scan the screen in an “F”-shaped pattern: across the top, then down the left-hand side, with short forays across. This means we there are large parts of the screen that don’t receive any attention.

And there is also the issue of computer familiarity. The argument is that the less familiar we are with the digital environment, the less likely we are to perform well in an online assessment (Noyes & Garland, 2008). This could explain why, in our study of PAT: Reading Comprehension, we found that the difference in overall scores was greater for Māori students and for those in low-decile schools. According to the concept of the “digital divide” (Cullen, 2001), both these groups, on average, have lower access to digital technologies (Statistics New Zealand, 2012; Gibson, Miller, Smith, Bell, & Crothers, 2013) and therefore are likely to have lower levels of computer familiarity.

It is also important to think about how often students use an online format to read the kinds of texts that are typically used for reading comprehension exercises and assessments. If most reading of these text types in school is done on paper, and reading strategies are taught using paper-based text, how does this affect students’ ability to read the same texts online?

So, digital reading is different: What does this mean?

In an increasingly digitised world, large-scale assessments will continue to move online. We are seeing this in New Zealand, where many traditionally paper-based tests are already offered online or are in the process of being moved online. The New Zealand Qualifications Authority (NZQA), for example, has moved some external NCEA assessments to a digital platform (NZQA, 2017).

When thinking about assessments, it’s important to be aware of potential differences in the way we process information on screen and on paper, especially if we’re comparing scores from online and paper versions. Traditionally, assessments require us to read in a focused way, to make sure we get the full meaning of a question. When the assessment involves reading large passages of linear text, it’s possible that differences in reading comprehension between the two modes will have particular significance.

However, as the old saying goes, “knowledge is power”. Being aware of possible differences between reading online and reading on paper is the first step in supporting our students to thrive in a world where digital reading is increasingly common. This awareness can help us make advances in three crucial, interrelated areas: advances in learning and teaching; advances in assessment; and advances in equity of opportunity.

Advances in learning and teaching

Our students are engaged in two kinds of reading: reading online and reading on paper. They read on paper as they begin reading instruction at school, and in their day-to-day lives in and out of school. They read online as they research information from the web, as they interact with social media, and as they use online apps and games. It is likely that as they progress through and beyond school to the world of work and further education, digital reading will play an ever-more-prevalent role in their lives.

Perhaps it is time to think about how much school reading, including reading instruction, should be done online and how much from traditional, print-based media. How can we help students transfer the comprehension strategies they have developed with print-based media to a digital environment? How can we help them to switch on the comprehension skills they need when they are reading passages of online text? And if reading online requires a new set of reading strategies, how can we identify these and help students develop them?

There is growing interest and research activity worldwide in strategies to help students develop skills to engage critically with online text. Online services such as Newsela (newsela.com), which produces a range of news and other nonfiction articles at a variety of reading levels with accompanying comprehension activities, are examples of the kinds of tools that are developing to support learning and teaching.

Advances in assessment

Digital technology has the potential to transform assessment by offering innovative ways of collecting data to inform student learning (Leeson & Hattie, 2009). An inevitable first step in the journey towards this innovation is to move traditional paper-and-pencil tests online (Redecker & Johannessen, 2013). As I’ve stated above, an awareness of possible differences between online and paper-based scores is important, so that results can be interpreted meaningfully, fairly, and to students’ best advantage. It’s when we’re comparing scores across two modes, or allowing students a choice between online and paper versions of an assessment and then scoring them on the same scale, that the differences might be problematic. As more and more assessment moves to an online format and paper-based tests become a thing of the past, this issue is likely to disappear.

How can we help students transfer the comprehension strategies they have developed with print-based media to a digital environment? How can we help them to switch on the comprehension skills they need when they are reading passages of online text? And if reading online requires a new set of reading strategies, how can we identify these and help students develop them?

Meanwhile, an awareness of possible differences in how we read on screen and on paper can help us as we consider what it is important to assess and how best to assess it. Besides comprehension of traditional, linear texts, is it also important to capture, understand, and assess the new ways of engaging with text made possible by online, multimodal formats? How can technology help with this process?

Advances in equity

Finally, and most importantly, being aware of the potential differences between reading online and reading on paper can help us as we search to give students equitable access to educational opportunities. As digital technology becomes ever more prevalent, this means not only providing equitable access, but also making sure that all students develop the competence and familiarity to engage with it. Ensuring that all children have equitable access to digital technology, and equal opportunities to use it, is a must if we want to prevent the widening of the achievement gap (Eyre, 2015).

In an assessment situation, we need to think carefully about the abilities of our students and make sure they have the best opportunity to demonstrate the skills we are interested in. Before making a choice to use digital assessment rather than paper, we need to ask some questions. Is the assessment a good match for the skills we want to find out about? Do we use digital technology to develop and practise these skills in the classroom? How familiar and comfortable are our students with digital technology? How familiar and comfortable are they with the devices that they will use to complete the test?

So what’s the bottom line?

In our modern world, there is a place for both reading on paper and reading online. Many people prefer reading longer linear texts on paper, rather than on screen. There is some research evidence to show that deep comprehension of linear texts is more effective when they are read on paper than when they are read on screen. There is also some evidence to suggest that in assessments of reading comprehension students may score lower when the test is presented online than they would when the test is presented on paper. Several factors are thought to be at play, including familiarity with computers, increased cognitive load, and expectations of the online environment. An awareness of the impact of these factors can inform our thinking about teaching and learning, assessment, and equitable access to educational opportunities. Reading online is an essential skill in our modern, digital environment. It is time to start thinking about how we can integrate it into our classrooms in meaningful ways.

For more information about the PAT: Reading Comprehension study and a one-page report summary, see http://www.nzcer.org.nz/research/publications/mode-equivalency-pat-reading-comprehension

We are happy to answer any questions schools might have about the research. Please contact Cathie or Julie on educationadvisor@nzcer.org.nz

References

Cullen, R. (2001). Addressing the digital divide. Online Information Review, 25(5), 311–320. https://doi.org/10.1108/14684520110410517

Eyre, J. (2015). Dual modes for written assessments: Examining validity and reliability. Assessment Matters 9, 4–24. https://doi.org/10.18296/am.0009

Eyre, J., Berg, M., Mazengarb, J., & Lawes, E. (2017). Mode equivalency in PAT: Reading Comprehension. Wellington: NZCER. Retrieved from http://www.nzcer.org.nz/research/publications/mode-equivalency-pat-reading-comprehension

Kerr, M. A., & Symons, S. E. (2006). Computerized presentation of text: Effects of children’s reading of informational material. Reading and Writing, 19, 1–19. https://doi.org/10.1007/s11145-003-8128-y

Gibson, A., Miller, M., Smith, P., Bell, A., & Crothers, C. (2013). The internet in New Zealand 2013. Auckland: Institute of Culture, Discourse & Communication, AUT University.

Leeson, H., & Hattie, J. (2009). Commissioned Review Paper 12: The use of technology for assessment. Auckland: Visible Learning Labs, Faculty of Education, University of Auckland.

Mangen, A. (2008). Hypertext fiction reading: Haptics and immersion. Journal of Research in Reading, 31(4), 404–419. https://doi.org/10.1111/j.1467-9817.2008.00380.x

Mangen, A., Walgermo, B. R., & Brønnick, K. (2013). Reading linear texts on paper versus computer screen: Effects on reading comprehension. International Journal of Educational Research, 58, 61–68. https://doi.org/10.1016/j.ijer.2012.12.002

Mayes, D. K., Sims, V.K., & Koonce, J. M. (2001). Comprehension and workload differences for VDT and paper-based reading. International Journal of Industrial Ergonomics, 28(6), 367–378. https://doi.org/10.1016/S0169-8141(01)00043-9

McNamara, D. S. (2007). Reading comprehension strategies: Theories, interventions, and technologies. New York: Psychology Press.

Ministry of Education (2010). The literacy learning progressions: Meeting the reading and writing demands of the curriculum. Wellington: author.

Neilsen, J. (2006). F-shaped pattern for reading web content. Nielsen Norman Group. Retrieved from https://www.nngroup.com/articles/f-shaped-pattern-reading-web-content/

Noyes, J. M., & Garland, K. J. (2008). Computer- vs. paper-based tasks: Are they equivalent? Ergonomics, 51(9), 1352–1375. https://doi.org/10.1080/00140130802170387

NZQA. (2017). Digital assessment: 2017 NCEA digital trials and pilots. Retrieved from http://www.nzqa.govt.nz/about-us/future-state/digital-assessment-trials-pilots/

Redecker, C., & Johannessen, O. (2013). Changing assessment—towards a new assessment paradigm using ICT. European Journal of Education, 48(1), 79–96. https://doi.org/10.1111/ejed.12018

Statistics New Zealand. Household use of information and communication technology: 2012. Retrieved from http://www.stats.govt.nz/browse_for_stats/industry_sectors/information_technology_and_communications/HouseholdUseofICT_HOTP2012/Commentary.aspx

Wästlund, E., Reinikka, H., Norlander, T., & Archer, T. (2005). Effects of VDT and paper presentation on consumption and production of information: Psychological and physiological factors. Computers in Human Behavior, 21(2), 377–394. https://doi.org/10.1016/j.chb.2004.02.007