MPEG-7 DCD Based Relevance Feedback Using Merged Palette Histogram Ka-Man Wong and Lai-Man Po ISIMP 2004 Poly U, Hong Kong Department of Electronic Engineering City University of Hong Kong

MPEG-7 Dominant Color Descriptor A compact and effective descriptor Generated by GLA color quantization Maximum of 8 colors in storage Each color have a minimum distance (T d ) of 15 in CIELuv color space

Commonly used RF techniques and limitations Feature weighting relevance feedback technique Assumes a fixed feature space (histograms) Taking liner combinations on matching histogram bins. Simple approach: Histogram averaging +() / 2 =

Commonly used RF technique and limitations But DCDs of images might have different set of colors, similar images might not have any exactly matched colors. The number of colors in updated query may greatly exceed the limit of the number of colors defined by MPEG-7 Similar colors are separated. By definition of DCD, similar colors should be grouped together. H1H1 H2H2 H’

Limitation of feature weighting relevance feedback technique The Merged Palette Histogram Relevance Feedback The updated query contains common colors among selected images Represent the selected images efficiently

Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - initialize Obtain all DCD from selected images

Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 1 Link all DCD together ++ = 6 colors 8 colors6 colors20 colors

Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 2 Palette Merging Find the closest pair of colors based on Euclidian distance in CIELuv If minimum distance smaller than T d merge the color pair and sum up the percentages of merged colors Iterate until minimum distance > T d 20 colors 9 colors

Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 3 Approximation Cut the least significant colors if number of colors >8 9 colors8 colors

Proposed Merged Palette Histogram for Relevance Feedback Merged Palette Histogram Relevance Feedback (MPH-RF) process - 4 Re-normalization Adjust the histogram sum into 1 An updated query is generated Approximated MPHUpdated Query Histogram Sum =1

Experimental Results of MPH-RF Experiment Methodology ANMRR Image Database 5466 Image from MPEG-7 common color dataset (CCD) Pre-defined queries and ground truth set Relevance Feedback Ground truth images are selected as relevant images

Latest experimental results MPH-RF gives improvement on both similarity measure methods Combination of MPHSM and MPH-RF gives a significant improvement Three iterations of relevance feedback give a significant result *ANMRR – smaller means better InitialAfter 3 RF RF Improvement DCD- MPHSM DCD- QHDM

Experimental results Visual results – Query #50 from MPEG-7 CCD, MPHSM Visit for more resultshttp:// Query image Ground truth images Initial retrieval, 4 of 8 ground truths hit, NMRR=0.5 First RF retrieval, 6 of 8 ground truths hit, NMRR= Second RF retrieval, 7 of 8 ground truths hit, NMRR=0.1541

Experimental results Visual results – Query #24 from MPEG-7 CCD, QHDM Visit for more resultshttp:// First RF retrieval, 6 of 12 ground truths hit, NMRR= Query image Ground truth images Initial retrieval, 5 of 12 ground truths hit, NMRR=0.5125Second RF retrieval, 9 of 12 ground truths hit, NMRR=0.1963

Conclusions on Merged Palette Histogram Relevance Feedback A new MPH-RF for MPEG-7 DCD is proposed MPH-RF generates a new DCD query using palette merging technique Represents the selected relevant images naturally and effectively Experiment result also found that proposed method improve DCD-QHDM by and MPHSM by using MPEG-7 Common Color Dataset The proposed method also provide better perceptually relevant image retrieval.