1. V. Mezaris, I. Kompatsiaris, N. V. Boulgouris, and M. G. Strintzis
Compressed-domain segmentation and ontologies for video indexing and retrieval
V. Mezaris, I. Kompatsiaris, N. V. Boulgouris,
and M. G. Strintzis
&
Aristotle University of Thessaloniki
Informatics and Telematics Institute

2. Presentation Overview
System Overview
Compressed-domain Segmentation Algorithm
Indexing Information Extraction
Video Indexing and Retrieval Scheme
Experimental Results
Conclusions

3. Overview – Segmentation and Feature Extraction

4. Compressed-domain Segmentation
Moving object segmentation and tracking
Background segmentation
Pixel-domain boundary refinement

5. Moving object segmentation and tracking
Iterative macroblock rejection, to detect macroblocks possibly belonging to foreground objects
Macroblock-level tracking, to examine the temporal consistency of the output of iterative rejection
Clustering of foreground macroblocks to connected regions and assignment to foreground spatiotemporal objects

6. Moving object segmentation and tracking - example

7. Background segmentation
Number of background objects is determined using the maximin algorithm and DC coefficients
In I-frames, macroblock clustering using K-Means algorithm and DC coefficients
In P-frames, tracking of background objects using macroblock motion vectors

8. Pixel-domain boundary refinement
Creation of pixel-accuracy segmentation masks using a Bayes classifier
Full decompression of the frame is necessary

9. Indexing Information Extraction
MPEG-7 descriptors:
Motion Activity
Dominant Color
GoF/GoP Color
Contour Shape
Motion Trajectory using “Local” Coordinates
Motion Trajectory using “Integrated” Coordinates

10. Video Indexing Scheme

11. Video Indexing Scheme – Object Ontology

12. Video Retrieval Process

13. Experimental Results

14. Experimental Results

15. Experimental Results

16. Experimental Results

17. Retrieval Experiments

18. Conclusions
Unsupervised algorithm for compressed-domain spatiotemporal segmentation
Performs in real-time (5.02 ms per CIF I-/P-frame)
Intermediate-level descriptors for user-friendly retrieval
No key-frame/key-sequences or manual annotation required for query initiation
Relevance feedback mechanism (SVM-based) for flexibility and efficiency