What Do Exploratory Searchers Look at in a Faceted Search Interface? Bill Kules and Matthew Banta The Catholic University of America School of Library and Information Science

Outline 60-second demo Our goals Research questions Experimental design Results Future work

Our Goals Improve understanding of how faceted interfaces affect searcher actions and tactics Develop and validate a methodology for creating exploratory search tasks for evaluations of search systems

Research Questions How long do searchers look at the major elements (facets, results, query box, breadcrumbs, etc.) of the interface? In what order do searchers look at the major elements in a faceted search interface?

Experimental Design N=18 successful sessions –From 21 subjects recruited 1x2 within-subjects design Task types –Exploratory tasks (n=4) –Known item tasks (n=2) Counterbalanced within task type

Procedure Introduction 90 second training video Calibrate eye tracker Conduct 6 searches Questionnaire after each search Retrospective verbal report –Video of two searches with gaze data overlaid Final questionnaire

Six Tasks Exploratory A.Feminism in the United States B.Textile industry on three continents C.Great Britain and the colonies in the 20 th century D.History of the Olympic games Known item: find a book E.Firefly Encyclopedia of Trees F.Harry Potter and the Goblet of Fire

Exploratory Task Imagine you are taking a class called “Feminism in the United States”. For this class you need to write a research paper on some aspect of the U.S. feminist movement, but have yet to decide on a topic. Use the catalog to find two possible topics for your paper. Then use the catalog to find three books for each topic so that you might make a decision as to which topic to write about.

Equipment Configuration NCSU Catalog Research Testbed Tobii 2150 remote eye tracker –21” monitor –50 Hz sampling rate –Resolution 1024x768 Tobii Clearview v2.7.1 Gaze fixations –Minimum 100 ms –30 pixel radius Manually segmented AOIs

Query Facets Breadcrumbs Results Interface with Areas of Interest (AOIs)

How long did searchers look at the major elements of the interface?

Exploratory Search Tasks Known Item Search Tasks

For exploratory search tasks

In what order did searchers look at the major elements?

Transitions in Attention Between AOIs

Query Facets Breadcrumbs Results Interface with Areas of Interest (AOIs)

Post-search Interviews “The subject thing worked. I don’t normally do subject searches.” “I needed a subject and I didn’t want to look through 2000 books.” “shopping around” - selecting facets and then looking to see what was available for a particular subject

Limitations Training was provided Researcher-provided tasks in lab setting One high-level scenario Tasks constructed were focused on this study

Future Work Additional factors –Training – Next study supported by OCLC/ALISE grant and CUA purchase of an eye tracker –Number and size of facets –Domain, search knowledge Additional measures –Gaze behavior – e.g. fixation counts –“Traditional” measures – e.g. clicks Refine procedure for exploratory search task generation

Conclusions Facets played a major role in exploratory searches –Fixation time about ½ as much as on results –On first page about equal –Facet-result & result-breadcrumb ~ ½ of all transitions

Acknowledgements Rob Capra, Tito Sierra, Jason Casden, and Joe Ryan Doug Oard and members of the UMD HCIL – for the use of their facilities and eye-tracker This research was supported by a grant from Catholic University and in part by a grant from the NSF/Library of Congress (IIS ).