3/6/2015 PortoICIAR’20041 Adaptive Methods for Motion Characterization and Segmentation of MPEG Compressed Frame Sequences C. Doulaverakis, S. Vagionitis, M. Zervakis, E. Petrakis Technical University of Crete (TUC) Chania Crete Greece

3/6/2015 PortoICIAR’20042 Problem Definition  Video segmentation  Abrupt (cuts) & Gradual transitions  Zoom, Pan/Tilt  Contribution: segmentation and identification of camera effects  Processing on MPEG video  Partially decompressed block data  DC intensity approximation for blocks  Coherent motion vectors for I, P, B frames

3/6/2015 PortoICIAR’20043 Twin Comparison (TC)  Shot boundaries peaks on histogram differences  Thresholds: Ta, Tb  Ta = μ + α σ  Τb = bμ  Requires pre- processing,does not adapt to signal Τα Tb

3/6/2015 PortoICIAR’20044 Sliding Window (SW)  Processing over W frames  One Threshold: Ta(i) = μ(i) + α σ(i)  Cut: < 5 frames  Gradual: > 5 frames  No preprocessing, adapts to signal Ta

3/6/2015 PortoICIAR’20045 Adaptive Method (AM)  Ta(i) = μ(i) + α σ(i) μ(i) =μ(i-1)-c(μ(i-1)–D(i)) σ(i) = |μ(i) 2 – λ(i)| 1/2 λ(i)=λ(i-1)–c (λ(i-1)–D(i)) 2 c=0.05  Ta is computed at each i and depends on previous values  No preprocessing, adapts to signal Ta

3/6/2015 PortoICIAR’20046 Direction Histogram  Histogram of angles of motion vectors  8 angles multiples of π/4 for moving vectors  Plus 0-th value for static vectors |v| < 1 static moving

3/6/2015 PortoICIAR’20047 Motion Characterization  Analysis of variance σ motion histogram  Normalized by number of intracoded vectors  Zooming: the vectors are spread uniformely (max σ)  Panning-Tilting: the vectors are concentrated at a single bin (min σ)  Static camera: the vectors are concentrated at bin 0

3/6/2015 PortoICIAR’20048 Example pan pan zoom static camera Ta

3/6/2015 PortoICIAR’20049 Video Segmentation Method  Cuts: the number of intracoded vectors in frame exceeds threshold  Gradual transitions: combines motion and intensity information  Difference of intensity histogram exceeds threshold  Magnitude of motion vectors exceed threshold

3/6/2015 PortoICIAR’ Experiments  Measurements over 17 videos  Competitive methods correspond to thresholding by TC, SW, AM  Each method is represented by its precision/recall curve as a function of the threshold parameter a

3/6/2015 PortoICIAR’ Abrupt Transitions (cuts)

3/6/2015 PortoICIAR’ Gradual Transitions

3/6/2015 PortoICIAR’ Future Work  More accurate threshold estimation  SW or AM gets trapped in local minima  Detection of Cuts is fairly stable  More elaborate methods for detection of gradual transitions and for cleaning- up false positives due to camera effects

3/6/2015 PortoICIAR’ Zoom Detection α=4.5 α=2

3/6/2015 PortoICIAR’ Pan/Tilt Detection α=2 α=4.5