Bridging the Motivation Gap for Individual Annotators: What Can We Learn From Photo Annotation Systems? Tabin Hasan Dept. of Computer Science University of Trento 1st Workshop on Incentives for the Semantic Web ISWC 2008, Karlsruhe, October 26th, 2008 Anthony Jameson FBK-irst Trento Contents: 1.The Two Motivation Gaps 2.Bridging the … [See the title] 3.Implications for the Other Workshop Papers (With audience participation)

The Community Motivation Gap (common view) Creating metadata for yourselfCreating metadata for a community  Spread the work around  Provide individual as well as community benefit  Provide quick confirmation and benefit  Make incremental contributions usable  Show what contributions are needed  Show what the current user is especially qualified to provide  Provide a fun game  Minimize privacy concerns  Publicize contributions Common strategies:

The Individual Motivation Gap Creating metadata for yourselfExploiting your own metadata Creating metadata for yourselfCreating metadata for a community

A Photo Annotation Interface "Sri Lanka dusk photos” detected by PhotoCompas using contextual metadata. From Naaman et al. (2004)

External Resources "Sri Lanka dusk photos” detected by PhotoCompas using contextual metadata. From Naaman et al. (2004)

Algorithms, User Interface SAPHARI, from Suh and Bederson (2007)

User Input, Affordances of Situations ARIA: cf. Lieberman, Rosenzweig, and Singh (2001)

Tag Learning Learning tags from examples Geographic and event data sources Suggesting annotations of new photos Correcting incorrect suggested annotations Best when user is uploading many photos related to a given place / event

SOMNet Better support for choice of terms Encourage case entry when doctor has been dealing with the relevant information Use KB of previous cases to support autocompletion Enable “scratching own itch” while browsing Intelligently use relevant data in doctors’ own computers

Collaborative IR Augmentation Suggest mapping on basis of previous learning Facilitate contribution after noticing of gap Use machine learning to exploit KB of existing keyword queries and formal representations Support testing and debugging to require minimal user effort

Constitution-Based Game Make sure that the game is actually fun!

Inverse Search KB of aggregated information needs Support batch export of subsets of private data Algorithms for recommending information to be made public Facilitate export when private data is originally added

Community Motivation Gap Spread the work around so that each person needs to do only a bit –Collaborative IR augmentation »"collaborative" Provide individual benefit as well as community benefit –Collaborative IR augmentation »"immediate"

Community Motivation Gap Provide quick confirmation of contribution and quick benefit –Collaborative IR augmentation »"immediate" –SOMNet »People want to know their contributions are being taken seriously

Community Motivation Gap Ensure that even incremental contributions can be utilized –Collaborative IR augmentation »"incremental", "partial" Show what contributions are needed (and which ones the current user is especially qualified to provide) –Inverse search

Community Motivation Gap Provide a fun game in which people do the desired work –Constitution-based game Protect privacy –SOMNet »People wanted to avoid revealing gaps in knowledge –Inverse search »Encourage people to move knowledge from private to public when it's needed Publicize (and maybe publicly evaluate) contributions