Facets of user-assigned tags and their effectiveness in image retrieval Nicky Ransom University for the Creative Arts

Election night crowd, Wellington, 1931 Photographer: William Hall Raine Election night crowd, Wellington, 1931 Reference number: 1/ F Original negative Photographic Archive, Alexander Turnbull Library Tags William Hall Raine crowd men hats street night lighting faces sea of people people watching event election results populated

Background to research topic Growth in number of images online Accurate and comprehensive indexing is critical to make online content accessible But visual materials are difficult to index

Approaches to image indexing Concept-based indexing – assigning index terms to describe the subject of an image

Approaches to image indexing Concept-based indexing – assigning index terms to describe the subject of an image Search engine indexing – index terms automatically created from data related to an image

Approaches to image indexing Concept-based indexing – assigning index terms to describe the subject of an image Search engine indexing – index terms automatically created from data related to an image Content-based indexing – using automatic processing to index image attributes such as colour, texture and shape

CIRES: Content Based Image REtrieval System

User tagging

Research question To find out value of tags for image retrieval by investigating whether the terms used to describe images in tags are similar to the terms used to search for images. – Which image facets are described in user tags? – How do these compare to those found in image queries? – What are the implications for future use of tagging for online indexing?

Armitage, L., & Enser, P. (1997). Analysis of user need in image archives Journal of Information Science, 23(4), SpecificGenericAbstract Who?Individually named person, group or thing (S1) eg Napoleon Kind of person, group or thing (G1) eg Skyscraper Mythical or fictitious being (A1) eg King Arthur What?Individually named event or action (S2) eg London Olympics Kind of event, action or condition (G2) eg Football game Emotion or abstraction (A2) eg Anger Where?Individually named geographical location (S3) eg New York Kind of place: geographical or architectural (G3) eg Forest Place symbolised (A3) eg Paradise When?Linear time: date or period (S4) eg 2010 Cyclical time: season or time of day (G4) eg Spring Emotion/abstraction symbolised by time (A4) eg Father Time Shatfords matrix

Research methodology Small scale study using 250 images and associated tags on Flickr Tags categorised using facets from Shatfords matrix Comparisons made with results of previous research into user queries

Comparison of tags and queries (1)

Comparison of tags and queries (2)

Comparison of tags and queries (3)

Factors affecting results Limited sample size – only 250 images Use of Flickr as domain for study – only 38% of users apply tags Subjectivity of categorising tags – only one person assigning tags to categories Suitability of Shatfords matrix – 22% of terms could not be categorised Lack of online query studies with which to compare the results

Conclusion Broad similarities between the image facets used in queries and image tags But differences in the level of specificity Need to develop systems to bridge this gap Consider the value of tags for browsing systems