Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 1 Mihai.

Slides:



Advertisements
Similar presentations
Using Large-Scale Web Data to Facilitate Textual Query Based Retrieval of Consumer Photos.
Advertisements

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki
Relevance Feedback and User Interaction for CBIR Hai Le Supervisor: Dr. Sid Ray.
Image Retrieval: Current Techniques, Promising Directions, and Open Issues Yong Rui, Thomas Huang and Shih-Fu Chang Published in the Journal of Visual.
Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?
Query Chains: Learning to Rank from Implicit Feedback Paper Authors: Filip Radlinski Thorsten Joachims Presented By: Steven Carr.
Software Quality Ranking: Bringing Order to Software Modules in Testing Fei Xing Michael R. Lyu Ping Guo.
Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle Irwin King and Zhong Jin Nov
Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.
Morris LeBlanc.  Why Image Retrieval is Hard?  Problems with Image Retrieval  Support Vector Machines  Active Learning  Image Processing ◦ Texture.
Image Search Presented by: Samantha Mahindrakar Diti Gandhi.
1 Statistical correlation analysis in image retrieval Reporter : Erica Li 2004/9/30.
ACM Multimedia th Annual Conference, October , 2004
Relevance Feedback based on Parameter Estimation of Target Distribution K. C. Sia and Irwin King Department of Computer Science & Engineering The Chinese.
SVM Active Learning with Application to Image Retrieval
1 Integrating User Feedback Log into Relevance Feedback by Coupled SVM for Content-Based Image Retrieval 9-April, 2005 Steven C. H. Hoi *, Michael R. Lyu.
A novel log-based relevance feedback technique in content- based image retrieval Reporter: Francis 2005/6/2.
Object Class Recognition Using Discriminative Local Features Gyuri Dorko and Cordelia Schmid.
Presentation in IJCNN 2004 Biased Support Vector Machine for Relevance Feedback in Image Retrieval Hoi, Chu-Hong Steven Department of Computer Science.
1 An Empirical Study on Large-Scale Content-Based Image Retrieval Group Meeting Presented by Wyman
Dept. of Computer Science & Engineering, CUHK Pseudo Relevance Feedback with Biased Support Vector Machine in Multimedia Retrieval Steven C.H. Hoi 14-Oct,
Presented by Zeehasham Rasheed
Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.
Scalable Text Mining with Sparse Generative Models
A structured learning framework for content- based image indexing and visual Query (Joo-Hwee, Jesse S. Jin) Presentation By: Salman Ahmad (270279)
Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle Irwin King and Zhong Jin The Chinese.
Optimizing Learning with SVM Constraint for Content-based Image Retrieval* Steven C.H. Hoi 1th March, 2004 *Note: The copyright of the presentation material.
SIEVE—Search Images Effectively through Visual Elimination Ying Liu, Dengsheng Zhang and Guojun Lu Gippsland School of Info Tech,
Information Retrieval in Practice
Competence Centre on Information Extraction and Image Understanding for Earth Observation Matteo Soccorsi (1) and Mihai Datcu (1,2) A Complex GMRF for.
Wavelet-Based Multiresolution Matching for Content-Based Image Retrieval Presented by Tienwei Tsai Department of Computer Science and Engineering Tatung.
Multimedia Databases (MMDB)
UOS 1 Ontology Based Personalized Search Zhang Tao The University of Seoul.
Glasgow 02/02/04 NN k networks for content-based image retrieval Daniel Heesch.
COLOR HISTOGRAM AND DISCRETE COSINE TRANSFORM FOR COLOR IMAGE RETRIEVAL Presented by 2006/8.
Competence Centre on Information Extraction and Image Understanding for Earth Observation 29/03/07 Blind city classification using aggregation of clusterings.
80 million tiny images: a large dataset for non-parametric object and scene recognition CS 4763 Multimedia Systems Spring 2008.
Automatic Image Annotation by Using Concept-Sensitive Salient Objects for Image Content Representation Jianping Fan, Yuli Gao, Hangzai Luo, Guangyou Xu.
Beyond Sliding Windows: Object Localization by Efficient Subwindow Search The best paper prize at CVPR 2008.
Relevance Feedback: New Trends Derive global optimization methods: More computationally robust Consider the correlation between different attributes Incorporate.
Relevance Feedback in Image Retrieval Systems: A Survey Part II Lin Luo, Tao Huang, Chengcui Zhang School of Computer Science Florida International University.
Chapter 4: Pattern Recognition. Classification is a process that assigns a label to an object according to some representation of the object’s properties.
PSEUDO-RELEVANCE FEEDBACK FOR MULTIMEDIA RETRIEVAL Seo Seok Jun.
Non-Photorealistic Rendering and Content- Based Image Retrieval Yuan-Hao Lai Pacific Graphics (2003)
Visual Categorization With Bags of Keypoints Original Authors: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, C. Bray ECCV Workshop on Statistical Learning.
2005/12/021 Content-Based Image Retrieval Using Grey Relational Analysis Dept. of Computer Engineering Tatung University Presenter: Tienwei Tsai ( 蔡殿偉.
2005/12/021 Fast Image Retrieval Using Low Frequency DCT Coefficients Dept. of Computer Engineering Tatung University Presenter: Yo-Ping Huang ( 黃有評 )
1 A Compact Feature Representation and Image Indexing in Content- Based Image Retrieval A presentation by Gita Das PhD Candidate 29 Nov 2005 Supervisor:
Probabilistic Latent Query Analysis for Combining Multiple Retrieval Sources Rong Yan Alexander G. Hauptmann School of Computer Science Carnegie Mellon.
Semi-Automatic Image Annotation Liu Wenyin, Susan Dumais, Yanfeng Sun, HongJiang Zhang, Mary Czerwinski and Brent Field Microsoft Research.
Image Classification for Automatic Annotation
Towards Total Scene Understanding: Classification, Annotation and Segmentation in an Automatic Framework N 工科所 錢雅馨 2011/01/16 Li-Jia Li, Richard.
26/01/20161Gianluca Demartini Ranking Categories for Faceted Search Gianluca Demartini L3S Research Seminars Hannover, 09 June 2006.
A Genetic Algorithm-Based Approach to Content-Based Image Retrieval Bo-Yen Wang( 王博彥 )
Identifying “Best Bet” Web Search Results by Mining Past User Behavior Author: Eugene Agichtein, Zijian Zheng (Microsoft Research) Source: KDD2006 Reporter:
Statistical techniques for video analysis and searching chapter Anton Korotygin.
Relevance Feedback in Image Retrieval System: A Survey Tao Huang Lin Luo Chengcui Zhang.
Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC Relevance Feedback for Image Retrieval.
Competence Centre on Information Extraction and Image Understanding for Earth Observation PLATO for Information Mining in Satellite Imagery Soufiane RITAL,
Content-Based Image Retrieval Using Color Space Transformation and Wavelet Transform Presented by Tienwei Tsai Department of Information Management Chihlee.
1 A Methodology for automatic retrieval of similarly shaped machinable components Mark Ascher - Dept of ECE.
1 Kernel Machines A relatively new learning methodology (1992) derived from statistical learning theory. Became famous when it gave accuracy comparable.
Design and Use of Earth Observation Image Content Tools Mihai Datcu(1, 2), Daniele Cerra(1), Houda Chaabouni-Chouayakh(1), Amaia de Miguel(1), Daniela.
Visual Information Retrieval
Multimedia Information Retrieval
Ying Dai Faculty of software and information science,
Color Image Retrieval based on Primitives of Color Moments
Semi-Automatic Data-Driven Ontology Construction System
Color Image Retrieval based on Primitives of Color Moments
Presentation transcript:

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 1 Mihai Costache Télécom Paris

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 2 Outline  General problem description  Relevance Feedback in Satellite CBIR  SVM based search engine  Results  Discussions

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 3 Problem statement  Context  Huge volume of Earth Observation (EO) images  High volume of information due to the increasing resolution (0.6 – 2.5 metesr/pixel)  Need to perform  Automatic indexation and annotation of satellite image archives (Content Based Image Retrieval, Relevance Feedback)  Detection and recognition of structures and objects within the satellite images (classification)  Reasearch areas  Information theory and statistics  Machine learning and Bayesian Inference – Incremental learning  Discriminative learning

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 4 Satellite Content Based Image Retrieval System  CBIR attempts to automate the indexing process of images in the database  Descriptions based on inherent properties of images (texture, color, shape etc.)  Problems:  Gap between low-level descriptors and semantic image content  Different image semantic interpretations : Same image but two persons Same person but different moments of time  Thus small retrieval precision  Solution: incorporating human perception subjectivity Relevance Feedback

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 5 RF in SCBIR Systems Relevance Feedback  Finding relevant information in satellite CBIR by human interaction  The retrieval process is improved by human decision  The user scores the retrieved documents as relevant / irrelevant  Annotations used in the learning process  Structure of a satellite indexation system with RF Primitive feature extraction Classification Relevance Feedback (RF) Quadrature Mirror Filters (QMF) SVM RF-SVM Objective Subjective

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 6 Structure of a RF system  Human-machine communication in RF Subjective Objective

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 7 RF Algorithms  Query point movement - Try to find an optimal vector model for retrieval purposes  Rocchio’s Algorithm J. J. Rocchio, Relevance feedback information retrieval, Viper, MARS D. McG. Squire et al., Content-based query of image databases: inspirations from text retrieval, Y. Ruiu et al., Content-based Image Retrieval with Relevance Feedback in MARS,  Re-weighting algorithms - Try to find an optimal similarity measure  MindReader Y. Ishikawa et al., Mindreader: Query Databases Through Multiple Examples,1998.  RF – Support Vector Machine (SVM) S. Thong and Chang E., Support vector Machine Active Learning for Image Retrieval, M. Costache, H. Maitre and M. Datcu,Categorization based Relevance Feedback Search Engine for Earth Observation Images Repositories, 2006.

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 8 Important points in RF  Feature selection  Texture, shape, color etc.  Learning scheme  Bayesian, Kernel learning methods, Neural Networks, Decision Trees  Scalability  In terms of storage space and query processing times  Type of search:  Category search  Target search

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 9 Proposed search engine  Search engine concept based on RF algorithm  Field of application: Satellite CBIR  Search tool: Support Vector Machines (SVM)  Bayesian inference used to infer data models  Hierarchy of information Data driven User driven Learing/Unlearning

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 10 Support Vector Machine  Linear separation case  Labeled data training set  Find a separation surface  Decision function f = sign(g(x))  d + = distance from g to closest {+1}  d - = distance from g to closest {-1}  Margin area = d + +d - =  Find a separating hyperplane with largest margin xixi xjxj g > 0 g < 0 g = 0 margin area SV + SV - Most Relevant (MR) Most Ambiguous (MA)

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 11 SVM based Search Engine  Algorithm  Query selection  Ranking  User’s relevance: {+1,-1}  Retrieve the Most Ambiguous (MA) images (g~0)  Go to 3  Return the Most Relevant images (MR) (max g)  Model data (GMM) inference – generation of categories  No of Gaussian components K obtained by Minimum Description Length (MDL)  Parameters estimation by Expectation and Maximization (EM) Memory

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 12 System memory  Hierarchy of information  Classes Λ  Categories C  SV sets  Adding memory  Queries sessions for images within the database  Semantical annotation of the SVM generated classes  Categories generation  Save the set of Support Vectors (SV)  Algorithm  Query example  Compute the likelihod based on the saved data models  Choose the set of SV for which the likelihood is maximized  Start RF with the chosen SV set

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 13  D Query image  Θ Search engine  Λ  C  U airports,……, village,……….. Hierarchy construction

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 14 RF quality evaluation  Measure the capability of retrieving relevant data  Relevance has to be experienced  Relevance is personal & situational  Precision – Recall graphs  B – retrieved imagesA – relevant images  Recall: fraction of the relevant images which have been retrieved R = |B∩A|/ |B |  Precision: fraction of the retrieved images which are relevant P = |B∩A|/ |A|  Mean Precison  Combines precision, relevance ranking and overall recall

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 15 Satellite database  SPOT5 ©CNES scenes 3000x3000 pixels  Resolution 5m/pixel  Cropped 64x64 pixels small images  46 scenes used to create an 11 classes database  100 examples per class  Primitive features: QMF coefficients (11)  Gaussian kernel:  8 images per RF step  15 RF steps  Evaluation of the top 60 MP images  Clouds  Sea  Desert  City  Forest  Fields  Airports  Village  Savanna  Boats  Traffic circle

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 16 SVM-RF simulation results (1) – primitive feature evaluation  Database composed of 600 SPOT5 images divided in six classes  Used features: Gabor, Haralick, QMF and GMRF  Gaussian Kernel  System evaluation: Precision-Recall graphs

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 17 SVM-RF simulation results (2) – automatic indexation  Database composed of 1100 SPOT5 images divided in eleven classes  The system has memory  QMF features  System evaluation: Precision-Recall graphs Search engineSpeed of learningMean precision SVM based15 RF steps Category based1 RF step0.9368

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 18 SVM-RF simulation results (3) - classification  Classification (structure detection / recognition)  SPOT5 image over Paris  Cemeteries areas

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 19 Conclusion  Use of hierarchy of information suitable for EO image interpretation  Semantical annotations of the generated classes  Category Bayesian based learning enhances the search capabilities by speeding the learning process  SVM supports classification of EO scenes.

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 20 Thank You!