Presentation is loading. Please wait.

Presentation is loading. Please wait.

H euristic Pre-Clustering Relevance Feedback in Attention-Based Image Retrieval Wan-Ting Su, Wen-Sheng Chu and Jenn-Jier James Lien Speaker: Wen-Sheng.

Published byRoy Phillips Modified over 9 years ago

Similar presentations

Presentation on theme: "H euristic Pre-Clustering Relevance Feedback in Attention-Based Image Retrieval Wan-Ting Su, Wen-Sheng Chu and Jenn-Jier James Lien Speaker: Wen-Sheng."— Presentation transcript:

1 H euristic Pre-Clustering Relevance Feedback in Attention-Based Image Retrieval Wan-Ting Su, Wen-Sheng Chu and Jenn-Jier James Lien Speaker: Wen-Sheng Chu Robotics Lab. CSIE NCKU

2 Robotics Lab, CSIE NCKU System Interface Result View Heuristic Pre-Clustering View User can revise the clustering results manually User can change the positive group number on his/her own Query Image Positive Feedback Negative Feedback

3 Robotics Lab, CSIE NCKU System Overview Wavelet Transformation Low-Low Subband Attended View Extraction Image Database Ranking by Euclidean Distance User Feedback? Query Image END No Yes Best Matches PCA VQ User Re-clustering Ranking by GBDA Learning Offline Module : Attention-Based Image Retrieval Online Module : Heuristic Pre-Clustering Relevance Feedback Feature Extraction from Attended View Heuristic Pre-clustering

4 Robotics Lab, CSIE NCKU Wavelet and Attended View Extraction To reduce the computational cost Contrast extraction is applied to the wavelet coefficient in the LL-subband. contrast value of pixel p at image location (i, j) Gaussian distance neighborhood of pixel (i, j) attention center Got saliency map!

5 Robotics Lab, CSIE NCKU System Overview Wavelet Transformation Low-Low Subband Attended View Extraction Image Database Ranking by Euclidean Distance User Feedback? Query Image END No Yes Best Matches PCA VQ User Re-clustering Ranking by GBDA Learning Offline Module : Attention-Based Image Retrieval Online Module : Heuristic Pre-Clustering Relevance Feedback Feature Extraction from Attended View Heuristic Pre-clustering

6 Robotics Lab, CSIE NCKU Visual Features Extraction Table1. 32 low-level visual features FeaturesDimension Color mean, standard deviation and skew in HSV space 9 Standard deviation of the wavelet coefficients in 10 pyramid de-correlated sub-bands 10 13 statistical elements extracted from the edge map such as max fill time, max fork count, etc. 13

7 Robotics Lab, CSIE NCKU System Overview Wavelet Transformation Low-Low Subband Attended View Extraction Image Database Ranking by Euclidean Distance User Feedback? Query Image END No Yes Best Matches User Re-clustering Ranking by GBDA Learning Offline Module : Attention-Based Image Retrieval Online Module : Heuristic Pre-Clustering Relevance Feedback Feature Extraction from Attended View Heuristic Pre-clustering Got features! PCA VQ

8 Robotics Lab, CSIE NCKU Pre-Clustering Principal Component Analysis (PCA) + Vector Quantization algorithm (VQ)

9 Robotics Lab, CSIE NCKU User Re-clustering System Pre-clustering Result User Re-clustering Result

10 Robotics Lab, CSIE NCKU System Overview Wavelet Transformation Low-Low Subband Attended View Extraction Image Database Ranking by Euclidean Distance User Feedback? Query Image END No Yes Best Matches User Re-clustering Ranking by GBDA Learning Offline Module : Attention-Based Image Retrieval Online Module : Heuristic Pre-Clustering Relevance Feedback Feature Extraction from Attended View Heuristic Pre-clustering PCA VQ

11 Robotics Lab, CSIE NCKU Re-weighting Scheme Group-Based Discriminant Analysis (GBDA) Multiple positive and multiple negative classes Clustering each positive class Scattering the negative example away from each positive class Single Flower Bouquets of Flowers Negative Samples Positive Samples

12 Robotics Lab, CSIE NCKU GBDA S w : the sum of the within-class scatter matrix of the positive groups S PN is the sum of between-class scatter matrices of positive-to-negative m i : the mean of the i th positive class C i c: the number of positive groups D : a set of negative examples

13 Robotics Lab, CSIE NCKU Experiment Result (1) COREL image database QS2: 1000 images from 10 selected categories Each of 10 categories contains 100 images and is used as queries. 1. Sunset2. Flower3. Car4. Ape5. Mountain 6. Penguin7. Tiger8. Bird9. Horse10. Building Table 1. Image Categories in Query Set 2

14 Robotics Lab, CSIE NCKU Experiment Result (2) 20.00% 25.00% 30.00% 35.00% 40.00% 45.00% 50.00% 55.00% 60.00% 102030405060708090100 Scope Precision Attention-Based SystemGlobal

15 Robotics Lab, CSIE NCKU Experiment Result (3) 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 12345678910 Category ID Precision Attention-Based SystemGlobal

16 Robotics Lab, CSIE NCKU Experimental Results (4) Precision = 5/10 Precision = 7/20 Query Image First-time retrieval results

17 Robotics Lab, CSIE NCKU Experimental Results (5) Precision = 8/10 Precision = 17/20 First-time feedback results

18 Robotics Lab, CSIE NCKU Experimental Results (6) Precision = 10/10 Precision = 20/20 Second-time feedback results

19 Robotics Lab, CSIE NCKU Conclusion The major work in this study is integrating attention-based image retrieval with the relevance feedback algorithm using multiple positive and negative groups. The system guides the user in clustering positive feedbacks by providing heuristic pre-clustering results. Then the user can revise the clusters manually.

20 Robotics Lab, CSIE NCKU Experiment Result - Video Demo

Download ppt "H euristic Pre-Clustering Relevance Feedback in Attention-Based Image Retrieval Wan-Ting Su, Wen-Sheng Chu and Jenn-Jier James Lien Speaker: Wen-Sheng."

Similar presentations

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki 2008-11629.

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki
Introduction Distance-based Adaptable Similarity Search

Introduction Distance-based Adaptable Similarity Search
Content-Based Image Retrieval

Content-Based Image Retrieval
Presented By: Vennela Sunnam

Presented By: Vennela Sunnam
Query Specific Fusion for Image Retrieval

Query Specific Fusion for Image Retrieval
ARNOLD SMEULDERS MARCEL WORRING SIMONE SANTINI AMARNATH GUPTA RAMESH JAIN PRESENTERS FATIH CAKIR MELIHCAN TURK Content-Based Image Retrieval at the End.

ARNOLD SMEULDERS MARCEL WORRING SIMONE SANTINI AMARNATH GUPTA RAMESH JAIN PRESENTERS FATIH CAKIR MELIHCAN TURK Content-Based Image Retrieval at the End.
Face Recognition and Biometric Systems

Face Recognition and Biometric Systems
Learning similarity measure for natural image retrieval with relevance feedback Reporter: Francis 2005/3/3.

Learning similarity measure for natural image retrieval with relevance feedback Reporter: Francis 2005/3/3.
Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle Irwin King and Zhong Jin Nov. 2001.

Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle Irwin King and Zhong Jin Nov
Integrating Color And Spatial Information for CBIR NTUT CSIE D.W. Lin 2003.8.26.

Integrating Color And Spatial Information for CBIR NTUT CSIE D.W. Lin
IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim

IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim
Young Deok Chun, Nam Chul Kim, Member, IEEE, and Ick Hoon Jang, Member, IEEE IEEE TRANSACTIONS ON MULTIMEDIA,OCTOBER 2008.

Young Deok Chun, Nam Chul Kim, Member, IEEE, and Ick Hoon Jang, Member, IEEE IEEE TRANSACTIONS ON MULTIMEDIA,OCTOBER 2008.
Effective Image Database Search via Dimensionality Reduction Anders Bjorholm Dahl and Henrik Aanæs IEEE Computer Society Conference on Computer Vision.

Effective Image Database Search via Dimensionality Reduction Anders Bjorholm Dahl and Henrik Aanæs IEEE Computer Society Conference on Computer Vision.
1 Learning on Relevance Feedback in Content-based Image Retrieval Hoi, Chu-Hong ( Steven ) Supervisor: Prof. Michael R. Lyu Venue:RM1027 Date: 11:00a.m.

1 Learning on Relevance Feedback in Content-based Image Retrieval Hoi, Chu-Hong ( Steven ) Supervisor: Prof. Michael R. Lyu Venue:RM1027 Date: 11:00a.m.
NCKU CSIE Visualization & Layout for Image Libraries Baback Moghaddam, Qi Tian IEEE Int’l Conf. on CVPR 2001 Speaker: 蘇琬婷.

NCKU CSIE Visualization & Layout for Image Libraries Baback Moghaddam, Qi Tian IEEE Int’l Conf. on CVPR 2001 Speaker: 蘇琬婷.
March 15-17, 2002Work with student Jong Oh Davi Geiger, Courant Institute, NYU On-Line Handwriting Recognition Transducer device (digitizer) Input: sequence.

March 15-17, 2002Work with student Jong Oh Davi Geiger, Courant Institute, NYU On-Line Handwriting Recognition Transducer device (digitizer) Input: sequence.
1 Texture Texture is a description of the spatial arrangement of color or intensities in an image or a selected region of an image. Structural approach:

1 Texture Texture is a description of the spatial arrangement of color or intensities in an image or a selected region of an image. Structural approach:
ACM Multimedia 2004 12th Annual Conference, October , 2004

ACM Multimedia th Annual Conference, October , 2004
Texture Texture is a description of the spatial arrangement of color or intensities in an image or a selected region of an image. Structural approach:

Texture Texture is a description of the spatial arrangement of color or intensities in an image or a selected region of an image. Structural approach:
Fast Image Replacement Using Multi-Resolution Approach Chih-Wei Fang and Jenn-Jier James Lien Robotics Lab Department of Computer Science and Information.

Fast Image Replacement Using Multi-Resolution Approach Chih-Wei Fang and Jenn-Jier James Lien Robotics Lab Department of Computer Science and Information.

Similar presentations

Ads by Google