V. Mezaris, I. Kompatsiaris, N. V. Boulgouris, and M. G. Strintzis Compressed-domain segmentation and ontologies for video indexing and retrieval V. Mezaris, I. Kompatsiaris, N. V. Boulgouris, and M. G. Strintzis & Aristotle University of Thessaloniki Informatics and Telematics Institute

Presentation Overview System Overview Compressed-domain Segmentation Algorithm Indexing Information Extraction Video Indexing and Retrieval Scheme Experimental Results Conclusions

Overview – Segmentation and Feature Extraction

Compressed-domain Segmentation Moving object segmentation and tracking Background segmentation Pixel-domain boundary refinement

Moving object segmentation and tracking Iterative macroblock rejection, to detect macroblocks possibly belonging to foreground objects Macroblock-level tracking, to examine the temporal consistency of the output of iterative rejection Clustering of foreground macroblocks to connected regions and assignment to foreground spatiotemporal objects

Moving object segmentation and tracking - example

Background segmentation Number of background objects is determined using the maximin algorithm and DC coefficients In I-frames, macroblock clustering using K-Means algorithm and DC coefficients In P-frames, tracking of background objects using macroblock motion vectors

Pixel-domain boundary refinement Creation of pixel-accuracy segmentation masks using a Bayes classifier Full decompression of the frame is necessary

Indexing Information Extraction MPEG-7 descriptors: Motion Activity Dominant Color GoF/GoP Color Contour Shape Motion Trajectory using “Local” Coordinates Motion Trajectory using “Integrated” Coordinates

Video Indexing Scheme

Video Indexing Scheme – Object Ontology

Video Retrieval Process

Experimental Results

Experimental Results

Experimental Results

Experimental Results

Retrieval Experiments

Conclusions Unsupervised algorithm for compressed-domain spatiotemporal segmentation Performs in real-time (5.02 ms per CIF I-/P-frame) Intermediate-level descriptors for user-friendly retrieval No key-frame/key-sequences or manual annotation required for query initiation Relevance feedback mechanism (SVM-based) for flexibility and efficiency