1. © ACTS-MoMuSys 1999. All Rights Reserved. VOGUE The Video Object Generator with User Environment Ecole Nationale Supérieure des Mines de Paris, France Instituto Superior Técnico, Portugal Universitat Politècnica de Catalunya, Spain University of Hannover, Germany

2. © ACTS-MoMuSys 1999. All Rights Reserved. Objective u To provide an interactive authoring tool to create video objects suitable for MPEG-4 encoding. u Characteristics:  Ease of use  No expert knowledge is needed  High performance

3. © ACTS-MoMuSys 1999. All Rights Reserved. Algorithms included u Spatial segmentation segmentation of still images u Temporal segmentation detection of moving objects u Tracking following the object through the video sequence The three algorithms are efficiently combined to provide the user with a performant tool

4. © ACTS-MoMuSys 1999. All Rights Reserved. Spatial segmentation Temporal segmentation Initial mask Fine segmentation Initial mask Tracking Correction of the results

5. © ACTS-MoMuSys 1999. All Rights Reserved. Graphical User Interface (GUI) u GUI provides interface to automatic segmentation + user interaction u GUI supports:  interaction between the different algorithms  general as well as algorithm specific user interaction u User interaction should be minimal but effective

6. © ACTS-MoMuSys 1999. All Rights Reserved. The Graphical User Interface

7. © ACTS-MoMuSys 1999. All Rights Reserved. The spatial segmentation u It is based on a family of nested partitions of the input image. u A certain level of the family is reached through the fusion of two or more regions of a finer level. u The finest level of the family corresponds to the fine mosaic resulting from flooding a gradient image from all minima. The coarsest level corresponds to the partition that contains the whole image.

8. © ACTS-MoMuSys 1999. All Rights Reserved. The spatial segmentation Original image Object Partition family Region selection

9. © ACTS-MoMuSys 1999. All Rights Reserved. The spatial segmentation u The whole family of partitions is created during a single morphological flooding of the input gradient image very fast calculation. u The hierarchy of fusions is stored in the form of a tree (typically less than 1000 nodes for QCIF images) and most of the calculations issued from the interaction with the user are carried out on this tree the results of the interaction are perceived as immediate by the user.

10. © ACTS-MoMuSys 1999. All Rights Reserved. Object Tracking: Approach Projection of a homogeneous color partition.

11. © ACTS-MoMuSys 1999. All Rights Reserved. Object Tracking: Partition Projection u Estimate the motion between the previous and current frames: u Link motion vectors. u Motion compensate the previous texture partition: u Backward block matching. u Fitting of the projected partition into a fine partition of the current image: u Two steps.

12. © ACTS-MoMuSys 1999. All Rights Reserved. Object Tracking: Partition Projection u In the fitting process, memory of the original label of the objects is kept (e.g.: the hat). u Small uncertainty areas usually remain. First step: GeometrySecond step: Geo. and color

13. © ACTS-MoMuSys 1999. All Rights Reserved. Temporal Segmentation

14. © ACTS-MoMuSys 1999. All Rights Reserved. Conclusions u A coherent approach has been chosen that combines morphological multiscale representations with motion analysis. u This allows for processing times which are compatible with user interaction on an ordinary PC. u The integration into a common user environment has lead us to analyze the degree of freedom to be left to the user.

15. © ACTS-MoMuSys 1999. All Rights Reserved. Future u VOGUE offers a solid ground to be developed in two directions:  As part of a professional MPEG-4 authoring tool  As an editing tool targeting the emerging mass market of numerical photography and video