Mega-boundaries TEMPORAL VIDEO BOUNDARY DETECTION

Mega-boundaries  Mega boundaries are defined between macro segments that exhibit different structural and feature consistency.  A good example of mega boundaries application is commercial detection

Commercial detection  Common method is detection of high activity rate and black frame detection coupled with silence detection  Lienhart 1997  Use of monochrome images, scene breaks, and action.  Blum 1992  Use of black frames and activity detector.  Iggulden 1997  Distance between black frame sequences.  Dimitrova 2002  Automatically spots repetitive patterns. Must be identified before recognizing

Commercial detection  Nafeh 1994  Learning and discerning of broadcast using Neural Network  Bonner 1982  McGee 1999  Novak 1988  Y. Li 2000  Agnihotri 2003

Features for Commercial detection  Mega boundaries detection method’s are based on what features we have on the test video  Unicolor Frame for commercial break  High visual activity  Letterbox format  Dataset of 8 hours of video from TV programs  Feature data consists of frames

Triggers and Verifiers  Trigger - Features that can aid in determining the location of the commercial break  Verifier – Features that can determine boundaries of the commercial break  We use the time interval between detected unicolor frame as triggers  Presence of a letterbox change or high cut rate expressed in terms of low cut used as verifiers.  Constrains on commercial breaks are longer than 1 minute and shorter than 6 minute.

Bayesian Belief Network Model  Directed acyclical graph (DAG)  The nodes correspond to variables  The arcs describe direct casual relationship between linked variables  The strength of these links is given by conditional probability distributions  P(x 1,..,x n )=P(X n |X n-1,..,x 1 )*... *P(x 2 |x 1 )P(x 1 )  Ω(x 1,..x n ) - Variables define as DAG  P(x i |∏ i )=P(x i |x 1,..,x n )  P(.|.) is a cpd (conditional probability density)  Using probability density function and chain rule

Bayesian Belief Network Model Probability for the verification node using pdf and chain rule Probability for potential commercial Probability for separator

Bayesian Belief Network Model Probability for sequence of black frames Probability for key frame distance

Evolved Algorithm  Challenge to create algorithm for all countries broadcast difference  Genetic algorithms implement a form Darwinian evolution.  Uses chromosome etc..  Eshelman’s CHC algorithm  CHC is general algorithm with 3 features  Monotonic  CHC prevents parents from mating if their genetic is too similar.  CHC uses soft restart

CHC for our Experiment  Default Parameters.  50 chromosomes, divergence rate of 35%.  Each parameter was coded as a binary string.  Each chromosome was decoded into set of parameters for the commercial detector and this detector was given a test video stream.  Correct label for the video frames were detected by human  Highest precision and recall was achieved with precision + recall

Results  First data set  8 hours of TV broad cast consisting of 13 TV programs  1.5 hours of 28 different commercials  Second dataset  4 hours of TV broad cast consisting of 11 TV programs  1 hour of 35 different commercials  FN,FP,TP,TN  Recall=TP/(TP+FN)  Precision=TP/(TP+FP)

Results  Using First data set  Results from first 4 experiment (recall and precision)  80.8% and 92.6%, 80.8% and 92.6%, 79.7% and 87.4%, 81.3% and 94.3%  Experiment 5 used experiment 4 and result was 88% and 90 %  Using second data set  This dataset was acquired after the algorithm  Test to the Genetic Algorithm  Results are shown in Figure.

Conclusion  Boundary segmentation in video  Visual scene segmentation  Multimodal story segmentation  Commercial detection  Questions ?