Human Vision and Information Visualization: A Collaboration Ruth Rosenholtz & Allison Woodruff SIMS-Information Visualization and Presentation November 2000

Outline The benefits of collaboration Collaboration 1: Understanding clutter Collaboration 2: Using thumbnails to search the web

The benefits of collaboration Knowledge of human vision useful for design. –What determines how easy it is to find and accurately interpret visual information? What is processed preattentively? How to best represent what kinds of information? What affects how accurately one comprehends a display? UI/infovis a potentially fruitful source of ideas for human vision research.

Barriers to collaboration Much current human vision research is aimed at uncovering mechanisms of the visual system. –This is not typically that useful for people who want predictive models of visual processing. Much infovis research (as a result?) makes no reference to human vision, or uses results inappropriate to the application.

Outline The benefits of collaboration Collaboration 1: Understanding clutter Collaboration 2: Using thumbnails to search the web

Clutter Makes a user interface less aesthetically pleasing. Makes it more difficult or more tiring to use. Makes it more difficult to understand a display and interact with it in a timely fashion. But what is it? Can we measure it? How do you reduce clutter?

Hypothesis: one component of clutter Clutter is when adding information to a display interferes with a user’s quick and accurate interpretation of the information in the display. Investigate: –What kinds of display manipulations interfere with the extraction of information from the display?

VIDA Manipulates the density in visualizations > Manipulating density is good because you fit more information in the display … but what affect does this have on perception of distribution?

Example: Replacing glyphs Added information: –Points in a scatter plot frequently replaced with symbols representing other information about those data points. Does this added information affect judgments about the scatterplot? –E.G. trend of plot, outliers, distribution of density of points in plot.

Example: Replacing glyphs Added information: –Points in a scatter plot frequently replaced with symbols representing other information about those data points. Does this added information affect judgments about the scatterplot? –E.G. trend of plot, outliers, distribution of density of points in plot.

Sample questions Does changing element: –Size –Shape –Color –... Affect judgments of: –Number, density, trend, location, or distribution of the data points by making the judgments either slower or inaccurate/more difficult?

Effects of size on judgments of number ?>?> ?>?> ?>?> fixed # # varies

Effect of size on judgments of number How many medium-sized disks do you need for it to appear that # small = # medium? # large = # medium? # mixed-size = # medium? Looking at quick judgments, here, not counting.

Results with small/large dots

Results with mixed-size dots

Preliminary results Users tend to overestimate the number of small dots relative to medium dots. Users tend to underestimate the number of large dots relative to medium dots. No bias in estimating the number of mixed-size dots, but users’ performance is more variable.

Outline The benefits of collaboration Collaboration 1: Understanding clutter Collaboration 2: Using thumbnails to search the web

Using Thumbnails to Search the Web Allison Woodruff, Andrew Faulring, Ruth Rosenholtz, Julie Morrison, and Peter Pirolli

Outline Introduction Related Work Thumbnail Generation Study Conclusions Comparing our enhanced thumbnails to plain thumbnails and text summaries

Motivation People spend a lot of time searching the Web We want to make search faster Options –Improve precision  Improve summaries 13 million queries/day Make them more accurate and make them easier to read quickly Probably need to clarify our model fairly early (search terms)

Text and Image Summaries Text summaries –Lots of abstract, semantic information Image summaries (plain thumbnails) –Layout, genre information –Gist extraction faster than with text Benefits are complementary We create textually-enhanced thumbnails that leverage the advantages of both text summaries and plain thumbnails URL, textual context for search term Text summaries don’t give you layout information, and they take longer to scan. Can’t usually read text in thumbnails.

Outline Introduction Related Work Thumbnail Generation Study Conclusions

Related Work Other thumbnails –Reduced image plus “dogears” (Cockburn et al.) –“Caricatures” showing selected features (Wynblatt and Benson) Use of thumbnails –Data Mountain, Web Forager, Pad, etc. Thumbnails for recall –Czerwinski

Outline Introduction Related Work Thumbnail Generation Study Conclusions

System Preprocessor modifies HTML –e.g., increase size of text, modify color of text Renderer creates scaled image of page Postprocessor transforms image –e.g., apply color wash, add text callouts

System Preprocessor modifies HTML –e.g., increase size of text, modify color of text Renderer creates scaled image of page Postprocessor transforms image –e.g., apply color wash, add text callouts

System Preprocessor modifies HTML –e.g., increase size of text, modify color of text Renderer creates scaled image of page Postprocessor transforms image –e.g., apply color wash, add text callouts

Design Goals Enhance features that help the user decide whether document is relevant to their query –Emphasize text that is relevant to query Text callouts –Enlarge (make readable) text that might be helpful in assessing page Enlarge headers

Design Issues: Visual Layering Problem: Callouts did not stand out – appeared to be part of original document Solution: Put them in a separate visual layer Hard to do by modifying HTML because HTML documents so diverse

Putting Callouts in a Separate Visual Layer Transparency Occlusion Junctions indicate the occurrence of these events.

Design Issues: Color Management Problems: Callouts need to be both readable and draw attention Solution: Desaturate the background image, and use a visual search model to choose appropriate colors –Colors look like those in highlighter pens Could show figures 1a/1c

Design Issues: Resizing of Text Problem: We want to make certain text elements readable, but not necessarily draw attention to them Solution: Modify the HTML before rendering the thumbnail Assuming text in original page is readable

Examples

Outline Introduction Related Work Thumbnail Generation Study Conclusions

Tasks Criteria: tasks that… –Are representative of common queries –Have result sets with different characteristics –Vary in the number of correct answers 4 types of tasks Picture: “Find a picture of a giraffe in the wild.” Homepage: “Find Kern Holoman’s homepage.” Side-effects: “Find at least three side effects of halcion.” E-commerce: “Find an e-commerce site where you can buy a DVD player. Identify the price in dollars.” Each of the 4 tasks had a different number of correct answers Range: 1/100 correct to 20/100 correct Some required genre classification. In others, semantic information in the text was especially useful. Some of the result sets were more textual than others. Tasks were chosen so that the result sets would have different characteristics. Study is based on search tasks we had the users perform.

Conditions Text summary –Page title –Extracted text with query terms in bold –URL Plain thumbnail Enhanced thumbnail –Readable H1, H2 tags –Highlighted callouts of query terms –Reduced contrast level in thumbnail For each of the questions we developed, we issued a query to Google. For each of the URLs Google returned as a search result, we created the following three types of summaries. Thumbnails get same number of pixels as text summaries.

Collections of Summaries 100 results in random order Approximately same number of each summary type on a page

Method Procedure –6 practice tasks –3 questions for each of the 4 task types e.g., each participant would do one E-commerce question using text, one E-commerce question using plain thumbnails, and one E-commerce question using enhanced thumbnails –Questions blocked by type of summary –WebLogger recorded user actions during browsing –Semi-structured interview Participants –12 members of the PARC community Entire process took about 75 minutes 18 questions, with 100 query results each 12 men and women from Xerox PARC All appropriate randomization was done so that different subjects did different questions with different conditions in different orders

Data Analysis Preliminary analysis: –Measure total search time on each trial –Normalize the data Some questions (“find …”) are slower than others For each question, subtract the mean search time for that question from the total search time –Average normalized data across subjects For each task type (picture/homepage/…) and each condition (text/plain/enhanced), find mean normalized search time

Results Average total search times, by task: –Picture: 61 secs –Homepage: 80 secs –E-commerce: 64 secs –Side effects: 128 secs

Results Normalized total search time (s)

Results Pooled Across All Tasks Subjects searched 20 seconds faster with enhanced thumbnails than with plain Subjects searched 30 seconds faster with enhanced thumbnails than with text summaries Mean search time overall was 83 seconds

Results: User Responses Participants preferred enhanced thumbnails –7/12 preferred overall –5/12 preferred for certain task types Enhanced thumbnails are intuitive and less work than text or plain thumbnails –One subject said searching for information with text summaries did not seem hard until he used the enhanced thumbnails. Many participants reported using genre information, cues from the callouts, the relationship between search terms, etc.

Outline Introduction Related Work Thumbnail Generation Study Conclusions

Relative performance of the summary types depends upon task In each task, enhanced thumbnails performed best, or tied for best Incorporating benefits of both text summaries and plain thumbnails led to consistently good performance for the enhanced thumbnails