Yan-Tao Zheng1, Ming Zhao2, Yang Song2, Hartwig Adam2 Ulrich Buddemeier2, Alessandro Bissacco2, Fernando Brucher2 Tat-Seng Chua1, and Hartmut Neven2 1.

Slides:



Advertisements
Similar presentations
Distinctive Image Features from Scale-Invariant Keypoints
Advertisements

Using Large-Scale Web Data to Facilitate Textual Query Based Retrieval of Consumer Photos.
PhishZoo: Detecting Phishing Websites By Looking at Them
Location-Based Social Networks Yu Zheng and Xing Xie Microsoft Research Asia Chapter 8 and 9 of the book Computing with Spatial Trajectories.
Mining User Similarity Based on Location History Yu Zheng, Quannan Li, Xing Xie Microsoft Research Asia.
James Hays and Alexei A. Efros Carnegie Mellon University CVPR IM2GPS: estimating geographic information from a single image Wen-Tsai Huang.
Psychological Advertising: Exploring User Psychology for Click Prediction in Sponsored Search Date: 2014/03/25 Author: Taifeng Wang, Jiang Bian, Shusen.
DONG XU, MEMBER, IEEE, AND SHIH-FU CHANG, FELLOW, IEEE Video Event Recognition Using Kernel Methods with Multilevel Temporal Alignment.
Presented by Xinyu Chang
Neurocomputing,Neurocomputing, Haojie Li Jinhui Tang Yi Wang Bin Liu School of Software, Dalian University of Technology School of Computer Science,
Context-aware Query Suggestion by Mining Click-through and Session Data Authors: H. Cao et.al KDD 08 Presented by Shize Su 1.
Stephan Gammeter, Lukas Bossard, Till Quack, Luc Van Gool.
Ghunhui Gu, Joseph J. Lim, Pablo Arbeláez, Jitendra Malik University of California at Berkeley Berkeley, CA
Explorations in Tag Suggestion and Query Expansion Jian Wang and Brian D. Davison Lehigh University, USA SSM 2008 (Workshop on Search in Social Media)
Landmark Classification in Large- scale Image Collections Yunpeng Li David J. Crandall Daniel P. Huttenlocher ICCV 2009.
Recognition using Regions CVPR Outline Introduction Overview of the Approach Experimental Results Conclusion.
Tour the World: building a web-scale landmark recognition engine ICCV 2009 Yan-Tao Zheng1, Ming Zhao2, Yang Song2, Hartwig Adam2 Ulrich Buddemeier2, Alessandro.
Object retrieval with large vocabularies and fast spatial matching
ADVISE: Advanced Digital Video Information Segmentation Engine
A Study of Approaches for Object Recognition
Generic Object Detection using Feature Maps Oscar Danielsson Stefan Carlsson
Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,
CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.
Object Class Recognition Using Discriminative Local Features Gyuri Dorko and Cordelia Schmid.
Smart Traveller with Visual Translator for OCR and Face Recognition LYU0203 FYP.
Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos.
A fuzzy video content representation for video summarization and content-based retrieval Anastasios D. Doulamis, Nikolaos D. Doulamis, Stefanos D. Kollias.
DVMM Lab, Columbia UniversityVideo Event Recognition Video Event Recognition: Multilevel Pyramid Matching Dong Xu and Shih-Fu Chang Digital Video and Multimedia.
Large-Scale Content-Based Image Retrieval Project Presentation CMPT 880: Large Scale Multimedia Systems and Cloud Computing Under supervision of Dr. Mohamed.
Wang, Z., et al. Presented by: Kayla Henneman October 27, 2014 WHO IS HERE: LOCATION AWARE FACE RECOGNITION.
Collective Vision: Using Extremely Large Photograph Collections Mark Lenz CameraNet Seminar University of Wisconsin – Madison January 26, 2010 Acknowledgments:
Language Identification of Search Engine Queries Hakan Ceylan Yookyung Kim Department of Computer Science Yahoo! Inc. University of North Texas 2821 Mission.
CS 376b Introduction to Computer Vision 04 / 29 / 2008 Instructor: Michael Eckmann.
AUTOMATIC ANNOTATION OF GEO-INFORMATION IN PANORAMIC STREET VIEW BY IMAGE RETRIEVAL Ming Chen, Yueting Zhuang, Fei Wu College of Computer Science, Zhejiang.
School of Information Technology & Electrical Engineering Multiple Feature Hashing for Real-time Large Scale Near-duplicate Video Retrieval Jingkuan Song*,
Chapter 7 Web Content Mining Xxxxxx. Introduction Web-content mining techniques are used to discover useful information from content on the web – textual.
Web Document Clustering By Sang-Cheol Seok. 1.Introduction: Web document clustering? Why ? Two results for the same query ‘amazon’ Google : currently.
PageRank for Product Image Search Kevin Jing (Googlc IncGVU, College of Computing, Georgia Institute of Technology) Shumeet Baluja (Google Inc.) WWW 2008.
Beyond Co-occurrence: Discovering and Visualizing Tag Relationships from Geo-spatial and Temporal Similarities Date : 2012/8/6 Resource : WSDM’12 Advisor.
UOS 1 Ontology Based Personalized Search Zhang Tao The University of Seoul.
Internet Information Retrieval Sun Wu. Course Goal To learn the basic concepts and techniques of internet search engines –How to use and evaluate search.
Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao.
Video Google: A Text Retrieval Approach to Object Matching in Videos Josef Sivic and Andrew Zisserman.
Collective Vision: Using Extremely Large Photograph Collections Mark Lenz CameraNet Seminar University of Wisconsin – Madison February 2, 2010 Acknowledgments:
CANONICAL IMAGE SELECTION FROM THE WEB ACM International Conference on Image and Video Retrieval, 2007 Yushi Jing Shumeet Baluja Henry Rowley.
Detecting Dominant Locations from Search Queries Lee Wang, Chuang Wang, Xing Xie, Josh Forman, Yansheng Lu, Wei-Ying Ma, Ying Li SIGIR 2005.
Gang WangDerek HoiemDavid Forsyth. INTRODUCTION APROACH (implement detail) EXPERIMENTS CONCLUSION.
Scene Reconstruction Seminar presented by Anton Jigalin Advanced Topics in Computer Vision ( )
Image Classification for Automatic Annotation
Chittampally Vasanth Raja 10IT05F vasanthexperiments.wordpress.com.
Chittampally Vasanth Raja vasanthexperiments.wordpress.com.
Lecture 8: Feature matching CS6670: Computer Vision Noah Snavely.
Data Mining, ICDM '08. Eighth IEEE International Conference on Duy-Dinh Le National Institute of Informatics Hitotsubashi, Chiyoda-ku Tokyo,
GENDER AND AGE RECOGNITION FOR VIDEO ANALYTICS SOLUTION PRESENTED BY: SUBHASH REDDY JOLAPURAM.
Using Cross-Media Correlation for Scene Detection in Travel Videos.
Computer Vision Group Department of Computer Science University of Illinois at Urbana-Champaign.
1 Review and Summary We have covered a LOT of material, spending more time and more detail on 2D image segmentation and analysis, but hopefully giving.
SIFT Scale-Invariant Feature Transform David Lowe
Presented by David Lee 3/20/2006
Personalized Social Image Recommendation
Source: Procedia Computer Science(2015)70:
Self-Organizing Maps for Content-Based Image Database Retrieval
Marked Point Processes for Crowd Counting
Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science
CSDD Features: Center-Surround Distribution Distance
Aim of the project Take your image Submit it to the search engine
SIFT.
Data Mining, Machine Learning, Data Analysis, etc. scikit-learn
Paper Reading Dalong Du April.08, 2011.
Applying Agile Lean to Global Software Development
Presentation transcript:

Yan-Tao Zheng1, Ming Zhao2, Yang Song2, Hartwig Adam2 Ulrich Buddemeier2, Alessandro Bissacco2, Fernando Brucher2 Tat-Seng Chua1, and Hartmut Neven2 1 NUS Graduate Sch. for Integrative Sciences and Engineering, National University of Singapore, Singapore 2 Google Inc., U.S.A T OUR THE W ORLD : BUILDING A WEB - SCALE AND LANDMARK RECOGNITION ENGINE ICCV 2009

OUTLINE  Introduction  Approach ( Framework)  Experiments  Conclusion (Future Work)

I NTRODUCTION What is the motivation ? With the vast amount of landmark multimedia data on the web

I NTRODUCTION  Application  Provide clean landmark images for building virtual tourism of a large number of landmarks  Facilitate both content understanding and geo-location detection of images and video  Provide tour guide recommendation and visualization

 Issue must be tackled  No readily available list of landmarks in the world Explore two source : (1) geographically calibrated images in photo sharing websites (2)travel guide articles from websites  Even if, it’s still challenging to collect true landmark image Download landmark images from two sources: (1)photo sharing websites(2)Google Image Search  Efficiency is a challenge for a large-scale system Accomplish by three means: (1)parallel computing (2)efficient clustering algo. (3)efficient image matching by k-d tree indexing I NTRODUCTION

A PPROACH FRAMEWORK

Learning landmarks from GPS-tagged photos >> Perform the agglomerative hierarchical clustering on the photo’s GPS coordinates >> Validation criterion is unique number of authors of photos is larger than a threshold

Set of images A PPROACH FRAMEWORK Learning landmarks from travel guide articles >> with the hierarchy, we can extract city names from country in six continents >> satisfy following criteria, text is deemed to be a landmark candidate

Set of images A PPROACH FRAMEWORK Learning landmarks from travel guide articles >> Validating landmarks (1) if it is too long or most of its words are not capitalized (2) the number of unique authors of images in the cluster >> which reflects the popular appeal of landmarks

A PPROACH DISCOVER LANDMARKS IN THE WORLD Most of users are located in Europe and North America !!

A PPROACH LEARNING OF LANDMARK IMAGES Object matching based on local features  Detect interest point >> LoG filters [11]  Local descriptor >> SIFT [9]  Reduce the feature dimensionality to 40 >> Principle Component Analysis (PCA) [2]  The match interest points of two images are verified >> affine transformation [9]

A PPROACH MATCH SCORE  Match score which is the probability of a false positive Can be estimated by Bayes Theorem [2] By using a cumulative binomial distribution

A PPROACH MATCH REGION Classified into two types: match edge and region overlap edge ---- Match edge ---- Region overlap edge

A PPROACH G RAPH CLUSTERING Do not have a priori knowledge of the # of clusters >> k-means are unsuitable >> exploit the hierarchical agglomerative clustering [2] The distance of region

A PPROACH C LEANING VISUAL MODEL  Photographic v.s non-photographic image classifier  Based on Adaboost algorithm over low level visual features of color histogram and hough transform.  Adopt a multi-view face detector[15]

A PPROACH EFFICIENCY ISSUES  Make efficiency essential in two aspects: (1) the landmark image mining (2) landmark recognition of query images  Achieve efficiency in three measures:  Parallel computing to mine true landmark images  Efficiency in hierarchical clustering  Indexing local feature for matching Use k-d tree[1] ~0.2 sec in a P4 computer

E XPERIMENTS Validated Landmark CitiesCountries GPS-tag photos Tour guide corpus landmarks are found to be common in both lists >> land mark is a perceptional and cognitive concept

E XPERIMENTS  Evaluation of landmark image mining  1000 visual clusters are randomly selected 68 of them are found to be negative outliers (0.68%)  The classifier is trained based on ~5000 photographic and non-photographic images, while the face detector is base on [15]  After cleaning, cluster rate drops to 0.37%

E XPERIMENTS Positive testing 728 images from 124 randomly selected landmarks Negative testing Caltech-256 [5] Pascal VOC 07 [3] Evaluation of landmark recognitioin

E XPERIMENTS Recognition : local feature matching of query image against model images, NN principle A match is found when the match score is larger than the threshold Recognition accuracy: 80.8% fairly satisfactory Image content analysis and geo-location detection: 46.3% moderately satisfactory

C ONCLUSION FUTURE WORK  Conclusion  Build a world-scale landmark recognition engine  Utilize ~21.4M images to build up landmark visual model  Incorporates 5312 landmarks from 1259 cities in 144 countries  Future work  Multi-lingual aspect of landmark engine >> help to discover more landmarks and collect more clean landmark images in their native languages in the Internet

R ELATED WORK Thank You !! Related Work 3D visualization of landmarks