Gemma Fitzsimmons, Mark J Weal and Denis Drieghe Reading for Comprehension versus Skim Reading on the Web: How Skim Reading is Informed by Hyperlinks Gemma Fitzsimmons, Mark J Weal and Denis Drieghe The Influence of Hyperlinks on Reading on the Web: An Empirical Approach.

Reading Research A majority of the reading research involves reading for comprehension of single sentences. After a number of these sentences simple yes/no questions are asked ensuring that readers are indeed reading for comprehension However, in every day life our reading experiences are often quite different Format: Paragraph versus single sentence (e.g. early eye movements faster in paragraph reading, Radach, Huestegge & Reilly, 2008) Task: Reading for comprehension vs. proofreading vs. skim reading

Reading on the Web: The Influence of Hyperlinks One difference between reading off-line and reading hypertext that is likely to influence how people read on the Web is the presence of hyperlinks Hyperlinks denote a connection to other content somewhere else on the Web Hyperlinks are coloured and salient compared to the rest of the text (Low-level influence) Lexical information of the word that is hyperlinked If we consider high-level processes those based on information not contained within the lexical representation of a word, then there is high level information attached to a hyperlink. A hyperlink links one piece of information to another, perhaps on a separate page of the same Website, or a different Website

How do we Read Hypertext? Very little experimental research so far, mostly focused on Human Computer Interaction (HCI) Nielsen (1999) claims that “the mother of bad design conventions is the decision to make hypertext links blue”. However, he also admits that hyperlinks being denoted in blue is now a convention and we should not move away from it Carr (2010) suggests that hyperlinks are a distraction in text and hinder comprehension Closest thing to a prediction: E-Z Reader - Reichle et al (2009) suggest that higher-level, post-lexical processes intervene in eye movement control only when “something is wrong” and either send a signal to stop moving forward (longer fixation or re- fixation) or a signal to execute a regression => exclusively impact later eye movement measures.

Reading on the Web: Set of Experiments Experiment 1: We start with a simple experiment looking into saliency Experiment 2: We then look into the influence of hyperlinks on reading for comprehension Experiment 3: Pulling apart saliency and hyperlinks With the large amount of information available to us on the Web we need a strategy to sort through all of the text presented to us = skim reading Experiment 4: Reading for comprehension vs skim reading on the Web and examining whether hyperlinks can guide skim reading

All Experiments Eyelink 1000 Reading for comprehension - occasional yes/no questions (unless of course in condition of skim reading in last experiment) Linear Mixed Models In most of the analyses presented here the LMM’s were run on the raw fixation times and are presented as such for transparency. When the data either deviated from the normal distribution and/or log transformed data resulted in qualitatively different models, we present the log transformed data Random Factors: Participants and Items. “Full random structure”, pruned if the model did not converge. Initial models were all fixed factors with all higher order interactions. Where we did not have strong predictions about the higher-order interactions, we carried out model comparisons to see whether they added to the fit of the model and removed them if not

Experiment 1 Before examining whether hyperlinks influence reading behaviour, we first examine whether there is indeed a negative influence of the hyperlink being salient/coloured outside of hypertext, so when it would not be perceived as a hyperlink.

Experiment 1 Before examining whether hyperlinks influence reading behaviour, we first examine whether there is indeed a negative influence of the hyperlink being salient/coloured outside of hypertext, so when it would not be perceived as a hyperlink. 30 participants 30 sentences 5 conditions: Colour of the target word

Experiment 1 : Word skipping

Experiment 1: Fixation times Longer fixation times on Grey (early and late measures) and on Green (late measures)

Experiment 1: Discussion There was an influence of the colour in that coloured words were skipped less often than the black word Fixation times were not influenced by the colouring with the exception of grey (early and late fixation measures) and green (only in late measures) We had expected a longer fixation times on grey due to the reduced contrast associated with the condition (see also Drieghe, 2008; Reingold & Rayner, 2006). After measuring candela using a photometer we discovered that green actually also had reduced contrast compared to the other colours. But we are not sure why this would impact the later measurements exclusively => For the crucial blue versus black comparison: Less skipping, no effect on fixation times

Experiment 2 The coloured words are embedded in edited Wikipedia pages, essentially becoming hyperlinks We also included a frequency manipulation. This allowed us to examine interactions with lexical processing and to show the statistical power of our design if hyperlinks were read as normal words In this experiment, no clicking yet Word Frequency Word Type

Experiment 2 Stimuli 32 participants (30 took part in Experiment 1 which was run first) 20 edited Wikipedia pages 80 target words, one per sentence, 4 sentences per Wikipedia article

Experiment 2 Stimuli Demand/rigour Phone/scarf Salt/grit Sleep/inert

Experiment 2 Results No effects on Word Skipping, we did not replicate reduced skipping of a coloured word which was observed in Experiment 1.

Experiment 2 Results In early measures (FFD - SFD - GD) only effect of frequency, no effect of hyperlinking.

Experiment 2 Results Late measures (Go-past & TT): Interaction between Frequency and hyperlinking.

Experiment 2 Results Hyperlinked, low-frequency words were associated with longer late fixation duration measures. The effect in total reading time is not due to re-reading after they have left the hyperlinked word there was no difference in regressions in (but there was in regressions out).

Experiment 2: Discussion Experiment 2 – Longer times in late measures for Low frequent/Hyperlinked words Yes, hyperlinked words are processed differently to coloured words, but only for low-frequency words Extra high level information contained in a hyperlink, presumably the suggestion of additional information Potentially compatible with E-Z Reader (Reichle et al, 2009). Higher-level processes intervene in later measures only when the processing is difficult Parts of these findings were published in 2013 as a conference proceeding and were independently replicated by Gagl (2016) who found no effects of hyperlinking on early measures but did find longer re-reading times. His stimuli would have been in our Low frequency condition

Experiment 2: Discussion We did not replicate the finding from Experiment 1 that coloured words were skipped less We assumed the original finding (reduced skipping of a coloured word) is likely due to a low-level factor, i.e. saliency However, in the hyperlink experiments multiple words were coloured. This might have influenced the saliency If this is true, we might not find the reduced skipping of a coloured word when multiple words are coloured

Experiment 3 36 Participants 72 sentences Number of blue words (0/1/3) x Frequency (HF/LF)

Experiment 3: Word Skipping Effect of Frequency (HF > LF) Target word gets skipped less when in colour but only when it is the only word in colour. (see also Experiment 1).

Experiment 3: Fixation times Frequency Effect Replication of Experiment 1: No Effect of Colour Blue on Fixation times.

Experiment 3: Discussion Strong evidence that hyperlinks will not disrupt normal reading simply on the basis of being a coloured word When multiple coloured words, no effect on skipping rates and no effect on fixation times The only effect is reduced skipping of a coloured word but only when it is the only one (Experiment 1)

Reading on the Web: Set of Experiments Experiment 1: We start with a simple experiment looking into saliency Experiment 2: We then look into the influence of hyperlinks on reading for comprehension Experiment 3: Pulling apart saliency and hyperlinks With the large amount of information available to us on the Web we need a strategy to sort through all of the text presented to us = skim reading Experiment 4: Reading for comprehension vs skim reading on the Web and examining whether hyperlinks can guide skim reading

Just & Carpenter (1987)/ Just, Carpenter, & Masson (1982) Normal reading, skim reading and speed reading Skim readers were compared to normal readers: 2.5 times faster than normal readers Shorter fixations (100 ms less per fixation) Smaller frequency and word length effects in fixation times More often skipping over multiple words. Also, big word length effects on word skipping but no differences in skipping content vs. function words, which was present for the normal readers Reduced comprehension, clear trade-off

Frequency and Task Effects A High-Frequency word receives shorter fixation durations than a Low-Frequency word (Inhoff & Rayner, 1986). Compared to the frequency effect in normal reading: Skim reading: Reduced frequency effect (Just & Carpenter, 1987) Visual search for a word in a text: no frequency effect (Rayner & Raney, 1996) - rapid surface comparison vs. full lexical access Proofreading: Bigger frequency effect (Kaakinen & Hyönä, 2010) - postlexical checking of correct word form

Reading for Comprehension vs. Skim Reading on the Web Predictions: Faster reading times for skim readers Reduced comprehension for skim readers Hypothesis that when skim reading the hyperlinks would become anchor points to target because they might indicate the location of important information Increased skipping for skim reading compared to reading for comprehension but to a lesser extent for hyperlinks Frequency effects reduced for unlinked words during skim reading. But linked words might show a frequency effect similar to normal reading

Experiment 4 32 participants - 8 conditions 2 (Task Type) x 2 (Word Type) x 2 (Word Frequency) 40 texts - (20 read normally - 20 skimming, in that order). Instructions were to skim read as they would normally do when having to read a large amount of text Each text had 4 target words, leading to 160 target words. Average word length 5.6 characters Word Frequency Task Type Read normally Word Type Skim read

Experiment 4: Results Participants read faster when skim reading (39 seconds per trial) than reading for comprehension (20 sec, t(31) = 17.28, p <.001) Participants read faster when skim reading (39 seconds per trial) than reading for comprehension (20 sec, t(31) = 17.28, p <.001) Skipping probabilities Single Fixation Duration (94% when fixated) Go- Past Time

Experiment 4: Skipping probability Large skipping rates Main effect of frequency. Skipping HF > LF. Interaction between Word Type and Task Type: Unlinked Words get skipped more during skim reading

Single fixation durations Three way interaction, so we ran contrasts: Normal reading: Frequency Effect both for the linked and unlinked words (replication Fitzsimmons, Weal & Drieghe, 2013) Single Fixation Durations shorter in skimming For the Skim readers Frequency Effect for Linked but not for Unlinked words

Go-past Times Messy Data. You can see that we could not remove any of the interactions even though none are significant. Main effect of frequency Main effect of Task type with less rereading in skim reading. Although regressions were not rare when the target word was fixated (14.11%), the combination with the high skipping rates leads to not much trials with a regression.

Reading for Comprehension vs. Skim Reading on the Web Predictions: Faster reading times for skim readers Reduced comprehension for skim readers Hypothesis that when skim reading the hyperlinks would become anchor points to target because they might indicate the location of important information Increased skipping for skim reading compared to reading for comprehension but to a lesser extent for hyperlinks Frequency effects reduced for unlinked words during skim reading. But linked words might show a frequency effect similar to normal reading

Practical implications When designing a webpage be aware of the fact that readers will use hyperlinks to skim read through your text so try to make the links relevant to the content of the website Hyperlinks can assist the readers with navigating through the text

Other related work Importance and hyperlinks – do hyperlinks suggest important sections of text, can the reader use hyperlinks to inform reading/skim reading? Clicking and navigating – the impact of having to navigate through Webpages on comprehension

Thank you for your attention!