1. Western Knight Center: The Business of Online Journalism
April 16, 2004
Laura Ruel, Estlow Center, University of Denver

2. What do you see?
How much do we really know about how readers interact with our websites?
We know some things from looking at our usage logs -- seeing what pages they look at; noting what links they click on; etc.
We can learn some things from asking them questions, either via questionnaires or in focus group.
But there is much about website user behavior that we can't know unless we can actually look through a person’s eyes.
And that's what eyetracking research is all about. We're able to take a group of website users and with considerable precision record where on a web page they look -- to look at a website user session and see what they see.
We can then take that record of a user's eye movements on a website and combine it with other users' recordings.
Now we can identify patterns of web-page usage across a user population.
And what's really cool about this is that we can identify patterns that no amount of questioning of the web users could ever uncover.

3. What did we want to learn?
What are the most effective ways to present editorial content on news home pages?
What stories should be told using multimedia?
What can we learn about how users interact with ads on news Web sites?
Once users decide to enter an interior article page on a news website, what keeps them there?
What did we want to learn? A LOT, But with this study we focused on some basic questions.
IMPORTANT TO NOTE, there are many ways to go about finding answers to these questions, we used the tools of eyetracking to begin our quest!
What's the best way to present editorial content on my news home page?
• How does a user scan a page?
• Where are the best spots to place photos and the best photo sizes to use?
• What's the best form of site navigation?
• What items on a home page get the most looks, and how can you utilize this information?
• What's the best use of color?
• What are the visual referrals on home pages?
• How much content, how many stories are ideal on a home page. Should the page scroll?
• What's the most effective headline presentation method?
What stories should be told using multimedia?
• What types of stories are best told in multimedia formats?
• When does multimedia enhance reader recall of story information?
• What are a users typical viewing patterns when exploring a multimedia project, and how can a designer utilize this information?
• Why did producers of some of the most successful multimedia projects out there decide to present their stories that way?
What do I need to know about ads on my news Web site?
• What is the typical interaction between a news site visitor and the ads on the pages?
• What ads get the most looks? What ones don't?
• How do some of the common techniques -- rollovers, drop downs, pop-ups, static placement -- fare?
• What is the best ad sizes based on location in the site?
• What are the visual barriers to users looking at ads?
• What happens when ads look like news content?
Once I get users into an interior article page on my news website, how do I keep them there?
• What are the most effective online writing techniques?
• Do readers absorb subheadlines, bulleted lists and other writing techniques?
• How do ads on article-level pages affect how readers interact with the editorial content?

4. What was done before? Eyetrack I - Eyes On The News - 1990
Here’s a little history. Eyetracking has been around for quite a while. It’s used in many applications, for example:
A study of where a shopper looks while going down a grocery aisle.
A study of how eye movement changes when a driver is talking on a cell phone.
And of course, eyetracking has great application for analyzing media usage.
This is the 3rd time that the Poynter Institute for Media Studies has embarked on an eyetracking study.
The first time was in 1990, when Poynter and Gallup Applied Science used eyetracking to learn about the effects of introducing color to printed newspapers.
On the left is what test participants were subjected too – reading a mock newspaper while wearing a heavy headset with 2 cameras.
A decade later, Poynter and Stanford University researchers conducted the first eyetracking study of news websites in
As you can see from the photo on the right, the headgear was slimmed down somewhat a decade later. There was only one camera necessary, but you still had people reading in the unnatural position of having an awkward contraption on the head.
Eyetrack I - Eyes On The News
Eyetrack II – Eyetrack 2000

5. What did we do? Eyetools Insight Reporter
Now we get to 2003/04 – the current study.
This one was conducted by Poynter, the Estlow Center for Journalism and New Media at Denver University, and Eyetools, a California-based eyetracking software and tools company.
It’s been nearly 5 years since Eyetrack II, and much has changed in that time.
We’re now in the broadband era, where most people use broadband connections – at least at work, if not at work and at home. (Eyetrack II: mostly dial-up users; controversial finding about people not looking at photos – which may have been because dial-up users were conditioned to read text while photos slowly loaded.)
Websites are far more sophisticated today, and use of multimedia for online storytelling is common and growing fast.
Eyetracking technology is much improved:
No need for headgear! Test subjects simply sat at a computer and viewed websites and web content.
Tracking is very accurate – within a centimeter but typically less than that.
When a test subject takes a coffee or bathroom break, recalibration is automatic. Ergo, this is as much like real reading of a website as you can get in a lab setting.
We used Eyetools’ “Insight Reporter” eyetracking analysis software package. It’s capable of quickly analyzing enormous amounts of data.
Photo above is of Eyetools CEO Colin Johnson on the Tobii eyetracker. You can see the small video camera at the bottom of the monitor.

6. What are our goals?
Present these preliminary findings to the news industry
Solicit reaction, opinions, and suggestions on what we've done so far
Take your feedback and ideas and embark on Part 2 of this project
Take this opportunity to better understand what the news industry wants and needs
What are our goals?
Present these preliminary findings to the news industry
This is a pilot study – and we hope the beginning of a series of smaller eyetracking studies looking at thinner slices of what we covered in this pilot.
Solicit reaction, opinions, and suggestions on what we've done so far
Take your feedback and ideas and embark on Part 2 of this project
Take this opportunity to better understand what the news industry wants and needs

7. Procedures
Created a series of mock news websites based on what we determined were “typical” news Web page designs.
Controlled 5 specific variables on each set of the home pages.
Headlines with blurbs vs. headlines without blurbs
Size of headline type, large vs. small
Size of text type, large vs. small
Amount of content (scrolling vs. non-scrolling page)
Pop up ad vs. no pop up ad
Controlled variables on article level pages
Created article presentations in multimedia vs. text formats.
Wrote a recall comprehension test
Selected high-quality multimedia presentations to observe how participants interacted with the content.
Invited 50 participants to spend about an hour with this content
Tracked and analyzed eye movement
Gathered demographic and descriptive data from participants
Procedures
1. First, we created a series of mock news websites based on what we determined were “typical” news Web page designs.
Controlled 5 specific variables on each set of the home pages.
(Total of 10 web pages; 5 sets of 2)
Headlines with blurbs vs. headlines without blurbs
Size of headline type, large vs. small
Size of text type, large vs. small
Amount of content (scrolling vs. non-scrolling page)
Rollover-expand ad vs. conventional ad banner
Controlled variables on article level pages
Photo size
Text layout (1-column vs. 3-column)
Paragraph length
Subheads & no subheads
Use of summary graf headline treatment & none
Use of bullet lists for presenting information & none
We also tested some other web-page characteristics, but in a less rigorous way. We’ll have observations on:
Navigation placement
Comparison of home-page designs, including eye viewing patterns
Advertising placement & advertising formats
Home-page photo usage
These were realistic looking mock sites with lots of content.
Articles were real, and timely enough to be of interest.
(Also had a live wire feed of top daily stories.)
2. Created article presentations in multimedia vs. text formats.
Wrote a recall comprehension test
Control article read first, with questions. Then 1 text article and 1 multimedia article viewed, followed by questions.
Eyetracker was on during this part of test.
So, we can take a question that people missed and look at the eyetracking data specific to where the answer to that question is found.
3. Selected high-quality multimedia presentations to observe how participants interacted with the content.
Invited 50 participants to spend about an hour with this content.
Testing took place in San Francisco. We invited a mix of participants ages 18 to 55. They were paid $75 for their time.
To be accepted, they had to acknowledge being at least occasional users of online news websites.
Because of the eyetracker that we used, we screened out people who wear eyeglasses. Too difficult to get good data from eyeglass wearers.
Contact lenses OK.
We felt that excluding eyeglasses was a trade we were willing to accept for having a test environment that was very close to a person’s normal website viewing.
Otherwise, we’d have ended up using the headgear.
Tracked and analyzed eye movement – 50 minutes for each of 50 participants.
Last 10 minutes: Gathered demographic and descriptive data from participants.

8. Sample page This is one of our five mock-website home pages.
On this particular one, there were 2 variations.
1 page used smaller type throughout the page.
Other page used larger type. Otherwise, identical.

9. Sample page Here’s another home page.
This one was our experiment with a home page that all fit on a single screen without requiring any scrolling.
The matching page was the same design, but extended well below the bottom of this page.

10. Sample test pages
Here’s another set. This design is loosely modeled after fairly information-dense news sites – a very common design model in use by some major news sites.
The page on the left carried only headline links to inside stories.
The page on the right carried headlines & blurbs for each article link.

11. Tracking data: individual session
Here’s a video that shows how one participant’s eyes moved through the test Web site to the left.
Lets take a look inside the head of one of our test participants.

12. Tracking data: Knight fellow
Lets take a look inside the head of one our fellows here!
Describe your experience
Watch the video

13. A single-user session
This is what an individual session recording looks like.
This is a single user looking at a single page – a home page in this example.
It’s showing the person’s eye fixations and the paths – called saccaades – between fixations.
The circles are fixations – which are very brief pauses as the eye focuses on something.
The thin lines are the saccaades – the paths between fixations.
The thicker lines show where more eve movement is concentrated on the page.
The green dot on the page – in the upper left – is the entrance point. That is, the first place that this person’s eye fixated on something. In this case, that was the topmost headline.
The red dot pretty far down the page is the exit point.

14. A single-user session
This slide further explains the components of the single-user session.
Now, while these are fascinating to look at, single-user sessions aren’t all that useful.
When it gets interesting is when we combine, for a single page, all the single-user sessions for a page for all 50 test participants.
But rather than display this as a maze of lines from all those people, the eyetracking researchers use what’s called a heatmap. …

15. Sample test pages
Now, this is the set of home pages I showed you earlier.
The left page uses only headlines; the right page uses headlines & blurbs.

16. Heatmap data from those pages
And this is what the heatmaps look like for those pages.
What looks like a weather doppler radar map is an aggregate image of the fixations of all the participants who looked at this pages.
For these two pages, 25 people looked at each page.
The red, orange and yellow areas show where most people looked. The blue areas saw much less viewing. And the black/gray areas are where nobody fixed their eyes at any time during the page session.
Again, these represent fixations. So for those black/gray areas, people still ran their eyes across those regions of the page – but it was just passing over from one point of fixation to the next.

17. Heatmap explanation A couple other highlights from a heatmap image:
The purple X’s represent where a user clicked.
And the red lines at the bottom represent how far participants scrolled down the page. There are numbers of the red lines, and they represent specific participants.
On some other pages, these red lines are more interesting – especially for a page that requires a lot of scrolling to see everything.

18. The findings Report of our observations will be available in May.
Today I’ll provide a small preview of some things we’ve noticed so far.
We’d originally expected to release findings this month.
But our quest for accuracy – and a more-than-expected amount of data – necessitated us spending more time compiling the results for public release.

19. Sample finding Blurbs with headlines caused drop in reading down page.
We’re still analyzing our data about our two home pages – one using only headline links, the other using headlines & blurbs.
I can tell you an initial observation, which is that with a page using blurbs, viewing tended to be concentrated higher on the page.
People viewing the headlines-only page read down the page further and saw more headlines.
Blurbs caused people to hang around more higher up the page.

20. Sample observation
What is the typical scanning pattern for a home page?
For a “typical” news home page, eye fixations tended to hover around the upper left of most home pages first.
Then they move in a left-right pattern.
Then move down the page.
Of course, this varied according to the design of the page – and we tested several design templates.
And some variables changed this behavior. E.g., on the home page with blurbs I showed you earlier, this left-right pattern was obvious. But on headlines-only home page, the trend was more vertical and less horizontal.
Later, we’ll be able to give you patterns on article-level pages, and on multimedia editorial content.

21. Sample observation Static ads are seldom seen.
Our test participants tended to avoid looking directly at static banner advertising.
You can see a demonstration of that in the heatmap on the left. There’s a banner ad up top, which gets some eye fixations.
And there’s a large “skyscraper” ad in the left column, which gets very few fixations.
Users discern where ads are and ignore them.
In general, we found that certain pages placements did better than others – as you might expect.
Don’t put ads in the lower right and expect them to be looked at much, for instance.
We had pretty good success with some text ads that we put on one set of home pages. You can see this on the right side of the image on the right.
And a mouseover-expand ad did really well – not as well as the text ads, but better than any banner ad including animated ones.

22. Sample observation The effect of photos of viewing patterns.
Similar to what was found in the Eyetrack II study in , we’ve observed that people tend to fixate first on headlines.
For this pilot study, we didn’t have controlled-variable pages specific to photos, so we’re not going to be definitive with what we release about photos.
But we can offer some observations that we gleaned from the eyetracking analysis that should be useful to you.
And we expect to take these observations and initiate a controlled eyetracking mini-study of images.
One small observation I can offer is that the size of a photo makes a difference in the percentage of eye fixations we recorded, which is fairly obvious.
But we observed that the difference in a medium-sized photo vs. a large photo wasn’t that great. That is, you may not get much bang for the buck by increasing image size from medium to large.
However, increasing from small to medium image size, we did see a large increase in fixations on photos.

23. Sample observations Visual barriers affect eye viewing patterns.
Multimedia content does not always increase reader recall of story information.
Navigation placement – top, left, or right – gets similar fixation and usage rates in all positions. But top navigation performs best.
A few more tidbits:
1. An interesting phenomenon that we noted was the effect of visual barriers.
By barrier, I mean something like a rule or line, or white space.
In general, when there’s a barrier present separating editorial content from advertising, that seems to reduce the eye fixations to the advertising.
2. Multimedia content does not always increase reader recall of story information.
We’re currently studying how different multimedia elements succeed in aiding comprehension, and comparing that to text comprehension.
More to come on this in May.
3. Navigation placement. Top navigation seems to perform well.

24. Visit us in mid/late May
We’ve got lots more findings and observations.
is the place to watch.
Extensive coverage and industry analysis of findings at Poynter.org.
Tell us what we should do post-pilot study!