1. Exploiting Flickr Tags and Groups for Finding Landmark Photos short paper at ECIR 2009 Rabeeh Abbasi, Sergey Chernov, Wolfgang Nejdl, Raluca Paiu, and Steffen Staab {abbasi,staab}@uni-koblenz.de, {chernov,nejdl,paiu}@L3S.de

2. Intro: Landmarks vs Non-Landmarks Problem Tag “Beijing”

3. Finding Landmark Photos Goal  Develop a method for easy classification of resources Idea  Exploit Flickr Groups (http://www.flickr.com/groups)‏ Method  Select groups related to positive and negative classes for training examples  Create and normalize feature space  Train the classifier  Classify unknown images Applications  Helps in improving search and browsing of resources related to particular class(es)‏ Query

4. flickr.com/photos/swamibu/2223726960/, flickr.com/photos/gunner66/2219882643/, flickr.com/photos/mromega/2346732045/, flickr.com/photos/me_haridas/399049455/, flickr.com/photos/caribb/84655830/, flickr.com/photos/conner395/1018557535/, flickr.com/photos/66164549@N00/2508246015/, flickr.com/photos/kupkup/1356487670/, flickr.com/photos/asam/432194779/, flickr.com/photos/michaelfoleyphotography/392504158/ Classifier Normalization Classification Model SVM +VE -VE Positive Training Examples Negative Training Examples Positive Flickr Groups Negative Flickr Groups

5. Problem decomposition

6. Tag Normalization for Classification U - users, T - tags, R - resources (photos) Normalized tag frequency of a tag t in a resource r Tag Frequency: TFr(t) = t tag counts per photo / total tags per photo Inversed Resource Frequency: IRF(t) = log (total number of photos / number of photos having t) Inversed User Frequency: IUF(t) = log (total number of users / number of user having t) Feature vector for a photo r: F(r) = [TFr(t1)*IRF(t1); TFr(t2)*IRF(t2); … ; TFr(tjTj) IRF(tjTj)]

7. Experiments with Normalization Schema Best schema: F(r) = TFr(t)*IRF(t)

8. Measuring Tag Representativeness City Tag Frequency, City User Tag Frequency, Confindence: t tag counts across landmark photos for a city CTF(t) = maximum tag counts across landmark photos for a city number of users having tag t across landmark photos for a city CUTF(t) = maximum number of users having tag t across landmark photos for a city CONF(t) = log (sum of confidence scores produced by SVM from all photos with t) RepresentativenessScore(t) = IRF(t) * IUF * CTF(t) * CUTF(t) * CONF(t)

9. Evaluation Experimental Setup Datasets  - Training Dataset (430k photos)‏  - Test Dataset (232k photos)‏ Comparison with state-of-art system WorldExplorer (Yahoo!) 20 Users evaluated both methods for 50 cities

11. Evaluation Prototype The users were asked to judge if a photo is a landmark or not Around 400-500 judgments per user (30 minutes per user). 20 users, each user evaluated two result sets (mixed together) for 10 randomly selected cities out of 50, each city is evaluated by 4 users in total

12. Micro-average Precision: World Explorer = 0.33 Our method = 0.37 Statistically significant

13. Macro-average Precision: World Explorer = 0.33 Our method = 0.34 Not statistically significant, variance is too high

14. Conclusions  Precision improvement of 12%, (80% users preferred our method 60% cities are better than WE with our method  L andmark finding based on photo classification can replace geo- tagging based methods in situations where geo-spatial information is not available The algorithm has a potential to be generalized beyond city landmarks for any topical photos, such as “cars", “mobile phones“, etc.

15. Thank You!