Incorporating Metadata into Search User Interfaces Marti Hearst UC Berkeley

Main Ideas Search is changing: More emphasis on flexibly showing next choices Less emphasis on ranking Web design is changing: More emphasis on dynamically determined views Less emphasis on pre-determined links Two key ideas: Task-specific design Harnessing the power of metadata

Outline Background Search Metadata Two examples Recipes Architectural Images Demo

Web search vs. Site Search Web search engines are providing source selection So … what happens when the user reaches the site? Follow Links … or … Search

Following Hyperlinks Works great when it is clear where to go next Frustrating when the desired directions are undetectable or unavailable Site Search This is not getting good reviews

Metadata Metadata is: Data about data Structures and languages for the description of information resources and their elements

Thesauri (Categories) A collection of selected vocabulary Broader, narrower, related-to relations Describe the content Medical text Anatomy, Disease, Chemicals, Procedures… Architectural images Location, Style, Materials, Period … Recipes! Cuisine, Ingredients, Season, Calories … These are often organized as hierarchical and faceted

New interfaces are mixing and matching thesaurus-style metadata Time/DateTopicRoleGeoRegion  The question: how to do this effectively?

What about Yahoo? Let’s try to find UCB

What about Yahoo?

Where is UCB?

Yahoo does use some metadata well Yahoo restaurant guide combines: Region Topic (restaurants) Related Information Other attributes (cuisines) Other topics related in place and time (movies)

Green: restaurants & attributes Red: related in place & time Yellow: geographic region

Combining Information Types Region State City A & E Film Theatre Music Restaurants California Eclectic Indian French Assumed task: looking for evening entertainment

Other Possible Combinations Region + A&E City + Restaurant + Movies City + Weather City + Education: Schools Restaurants + Schools …

Bookstore preview combinations topic + related topics topic + publications by same author topic + books of same type but related topic

Goals for Metadata Usage Well-integrated with search Provides useful hints of where to go next Tailored to task as it develops Personalized Dynamic

Recipe Example

soar.berkeley.edu/recipes

Epicurious Metadata Usage Advantages Creates combinations of metadata on the fly Different metadata choices show the same information in different ways Previews show how many recipes will result Easy to back up Supports several task types ``Help me find a summer pasta,'' (ingredient type with event type), ``How can I use an avocado in a salad?'' (ingredient type with dish type), ``How can I bake sea-bass'' (preparation type and ingredient type)

A View of Web Site Structure (Newman et al. 00) Information design structure, categories of information Navigation design interaction with information structure Graphic design visual presentation of information and navigation (color, typography, etc.) Courtesy of Mark Newman

Information Architecture includes management and more responsibility for content User Interface Design includes testing and evaluation Information Architecture vs. UI (Newman et al. 00) Courtesy of Mark Newman

Recipe Information Architecture Information design Recipes have five types of metadata categories Cuisine, Preparation, Ingredients, Dish, Occasion Each category has one level of subcategories

Recipe Information Architecture Navigation design Home page: show top level of all categories Other pages: A link on an attribute ANDS that attribute to the current query; results are shown according to a category that is not yet part of the query A change-view link does not change the query, but does change which category’s metadata organizes the results

Metadata usage in Epicurious PrepareCuisineIngredientDish Recipe

Metadata usage in Epicurious PrepareCuisineIngredientDish PrepareCuisineDish I Select

Metadata usage in Epicurious PrepareCuisineIngredientDish I > Group by PrepareCuisineDish

Metadata usage in Epicurious PrepareCuisineIngredientDish PrepareCuisineDish I > Group by

Metadata usage in Epicurious PrepareCuisineIngredientDish PrepareCuisineDish I > Group by PrepareCuisine I Select

Metadata Usage in Epicurious Can choose category types in any order But categories never more than one level deep And can never use more than one instance of a category Even though items may be assigned more than one of each category type Items (recipes) are dead-ends Don’t link to “more like this” Not fully integrated with search

Epicurious Metadata Usage Problem: lacks integration with search

The FLAMENCO Project FLexible Access using MEtadata in Novel COmbinations Main goal: Perform systematic studies to determine how metadata should be incorporated into search Answer questions such as: Given a set of user goals and a set of information with certain characteristics (size, inter-connectivity) How many metadata combinations to show? What level of detail to show? How best to preview and postview choices?

The FLAMENCO Project Focusing on very large collections whose items are not easily classified Medical text, image databases However, much should apply to website design as well

Evaluation Methodology Regression Test Select a set of tasks Use these throughout the evaluation Start with a baseline system Evaluate using the test tasks Add a feature Evaluation again Compare to baseline Only retain those changes that improve results

Application to Image Search

Image Search: What is the task? Illustrate my slides? “Find a crevasse” Keyword match works pretty well Find inspiration for an architectural design? General similarity: maybe But more control might be better

How different from medical example? More open-ended Easier to scan many images quickly Tertrain metaphor not used here Not narrowing down a large set Rather, always viewing more images A mechanism for “steering” through the metadata

Our Approach Architecture task: Emphasize images over text Use hypertext-style interface as a reasonable baseline for comparison Find out how much choice is too much Find out whether explicit metadata is better than implicit more-like-this

SPIRO: >40,000 art & architecture images Detailed metadata

SPIRO Query Form

SPIRO query on Subject: church

Another Example Greatbuildings.com Hyperlinks metadata together But a small collection ~1000 buildings ~4500 images total

Our Approach Create a system that allows experimentation with different interfaces Add functionality in a stepwise fashion Architecture task: Use hypertext-style interface as a reasonable baseline for comparison Find out how much choice is too much Find out whether explicit metadata is better than implicit more-like-this

Faceted Metadata Planalto Palace Parti Communiste Francais Pantheon Oscar Neimeyer Oscar Neimeyer Jacques-Gabriel Soufflot 20 th Century 20 th Century 17 th & 18 th C. Brasilia Paris Paris Stone Curvilinear Stone Image: Architect: Period: Location: Concept:

Planalto Palace Parti Communiste Francais Pantheon Oscar Neimeyer Oscar Neimeyer Jaques-Gabriel Soufflot 20 th Century 20 th Century 17 th & 18 th C. Brasilia Paris Paris Stone Curvilinear Stone Image: Architect: Period: Location: Concept: Faceted Metadata

Evaluation Methodology Solicit feedback from architects to determine if faceted metadata is helpful and how to present it Not evaluating if the current metadata in the system is the right metadata Lo-Fi evaluation of paper prototype 1 hour one-on-one with 3 professional architects Walk-through interactions on a paper computer, users think- aloud (audio recorded), questions about the experience Informal study of live version 1 hour one-on-one with 9 architects /grad students, 2 tasks (audio recorded) and a survey

Make iterative changes to the alternatives to identify useful aspects of the UI – not statistical analysis Learn to what extent the metadata is useful for searching How much text is too much? What kinds of queries will the users do? Explore how to clarify searching within results vs. starting a new search Goals for Study of Live Web Site

Informal Study of Live Version Comparison- a form-based UI to the same collection of images Task 1 – Free form search Participants told they are helping to test an image search engine Asked to talk about a project they’re working on and something they’d like to find Let them go at it and try to find images they’re interested in – they can ask any questions they want but no formal instructions given Task 2 – Treasure hunt Participants given paper copies of 3 images (but no metadata) and told to find them (range from easy to hard) Exit survey

About the System Web browser used to view Cold Fusion generated pages, queries made to a mySQL database 36,000 images from the UC Berkeley Architecture Slide library 4 facets of metadata about each image Image title, architect, period, location, and concept Concepts were not yet integrated

Results: Metadata is Helpful Very positive feedback about Flamenco All 9 participants named the metadata in the search results area as their favorite aspect of Flamenco Metadata was successful at giving hints about where to go next Perceived as useful “These are places I can go from here.”

Results: More Metadata Please Participants asked for more metadata Although there were complaints about the contents of the metadata, users still wanted more Longer lists of options (more hints) Users wanted more control to make very specific searches Half the participants requested the ability to control order of results with metadata Juxtapose visible images 2 different ways: Overview (one image from each project) vs. like together ( all images of a project next to each other) Different than ranking for text retrieval (precision, recall), but ordering does matter

Results: Complaints The UI was not successful at clarifying searching within results vs. starting a new search Only 2 of the 9 participants understood the distinction without discussion – but they want to do both The 1/3 of the participants who couldn’t find a treasure hunt image felt that Flamenco was slow Corroborates findings that perceived system speed is about finding what you want (Spool ‘00)

Enthusiasm for Metadata Contradicted my suspicions that a sledgehammer and pick- through-the-rubble approach would be preferable No-one thought there was too much text Adding more text preferable to adding another row of images

Conclusions Metadata is useful for exploratory tasks Good at giving hints about where to go next Architects want to use metadata to get more control Of results display Build complex queries Sometimes the right word is worth more than many pictures