A Robust Scene-Change Detection Method for Video Segmentation Chung-Lin Huang and Bing-Yao Liao IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY

Outline IntroductionIntroduction Abrupt Scene-Change DetectionAbrupt Scene-Change Detection Gradual Scene-Change DetectionGradual Scene-Change Detection Experimental ResultsExperimental Results ConclusionConclusion

Introduction The main problem of segmenting a video sequence into shots is the ability to distinguish between scene breaks and normal changes that happen in the sceneThe main problem of segmenting a video sequence into shots is the ability to distinguish between scene breaks and normal changes that happen in the scene This paper combines the intensity and motion information to detect scene changes such as abrupt scene changes and gradual scene changesThis paper combines the intensity and motion information to detect scene changes such as abrupt scene changes and gradual scene changes

Previous Problems The two main problems in most existing algorithmsThe two main problems in most existing algorithms –they are threshold-dependent algorithms –they suffer false detection with scenes involving fast camera or object motion. This paper proposes a scene-change detection algorithm with three contributionsThis paper proposes a scene-change detection algorithm with three contributions –Relaxing threshold selection problem –higher detection rate (scene change should not be missed) –lower false alarm rate

ABRUPT SCENE-CHANGE DETECTION Method Method – Measurement of the Changes Between Frames Pixel-Based DifferencePixel-Based Difference Histogram-Based DifferenceHistogram-Based Difference – Static Scene Test – Scene Transition Classification – Detection Algorithm

Detection Algorithm First phaseFirst phase –locate the highest and the second highest peaks of DCimage difference in the midst of the sliding window, and then calculate the ratio n between the first and second peaks Second phaseSecond phase –Histogram Measure –Static Scene Test (Most of the false alarms declared by the histogram detector are due to sudden light changes, while the edge information is more or less invariant to these changes) –Scene Transition genuineAmbiguousNo Scene Change N high N low

Measurement of the Changes Between Frames Pixel-Based Difference:Pixel-Based Difference: – Histogram-Based Difference: (X 2 Test)Histogram-Based Difference: (X 2 Test) – Where Cx and Cy are the DC image of frames X and Y

Color Histogram Efficient representationEfficient representation –Easy computation –Global color distribution Insensitive toInsensitive to –Rotation –Zooming –Changing resolution –Partial occlusions DisadvantageDisadvantage –Ignore spatial relationship –Sensitive in illumination changes Choose illumination-insensitive color channelsChoose illumination-insensitive color channels

Example Color space selection & quantizationColor space selection & quantization –Use RGB channels Each channel is divided into 2 intervalsEach channel is divided into 2 intervals Total number of bins = 2 3 = 8Total number of bins = 2 3 = 8 –H(I): Color histogram for Image I –H 1 = (7, 7, 7, 7, 9, 9, 9, 9) Image 1 has 7 pixels in each C 1 to C 4, and 9 pixels in each C 5 to C 8Image 1 has 7 pixels in each C 1 to C 4, and 9 pixels in each C 5 to C 8

Static Scene Test DefineDefine –all objects present in the scene exhibit rather small motion compared to the frame size, and global movement caused by the camera is slow and smooth. MethodMethod –Edge Detection –Edge Dilation ResultResult –The transition of two consecutive frames with covering ratio larger than a predefined threshold is considered as a non- static or dynamic scene.

Example of edge detection Edge Detection Edge Dilation ( r=3 )

Scene Transition Classification Transition TypeTransition Type –1) static scene to static scene –2) dynamic scene to static scene or vice versa –3) dynamic scene to dynamic scene Dynamic-to-dynamic transition usually indicates continuous object or camera motion, rather than a real scene changeDynamic-to-dynamic transition usually indicates continuous object or camera motion, rather than a real scene change

Gradual Scene-Change Detection Gradual Scene-ChangeGradual Scene-Change –Dissolve –Fade-In( X = 0 ) –Fade-Out( Y = 0 ) Why not easy to detectWhy not easy to detect –Camera and object motions always introduce a larger variation than a gradual transition. Scene X to Y In Duration T

Intensity Statistics Model Normal CaseNormal Case –For any frames near the reference frame, their dissimilarity measure almost increases exponentially with their distance Gradual TransitionGradual Transition –The dissimilarity increases linearly with their distance during the transition –After the transition is over, the difference measures are randomly distributed

Normal SequenceGradual Transition Seed : the beginning frame of a gradual transition

N-distance measureN-distance measure –the difference measure generated by comparing a frame with itself and its successive ( N – 1 ) frames – – Ideal model of the N-distance measure

Gradual Scene-Change-Detection Algorithm 1) N-distance measure1) N-distance measure 2) Difference operation2) Difference operation 3)3) Low-pass filtering: Implement a simple low-pass filter to keep the low frequency segments and to remove the high-frequency segments

Gradual Scene-Change-Detection Algorithm If the number of zero crossings between frame k and frame l (k and l is zero crossing frame)If the number of zero crossings between frame k and frame l (k and l is zero crossing frame) – Zero-crossing rate calculation: – larger than a threshold : high frequency fragment – else : low frequency fragment (gradual scene-change segment) a local “ score ” record mechanisma local “ score ” record mechanism –Score i (q)q=0, 1, 2 …, N-1 High frequency fragment : Score i (q) = 0High frequency fragment : Score i (q) = 0 Low frequency fragment : Score i (q) = 1Low frequency fragment : Score i (q) = 1

N-Distance Measure Difference operation Local Score

Gradual Scene-Change-Detection Algorithm a global “ track ” record mechanisma global “ track ” record mechanism –Track(p)p = 1, 2, 3, …,L After every N-distance Measure of frame i, we can get the local Score i (q)After every N-distance Measure of frame i, we can get the local Score i (q) – Accumulate the Score record to the Track record the total number of frames in the video sequence

Improve Gradual Scene-Change- Detection Algorithm To develop a fast seed-searching processTo develop a fast seed-searching process – –we select one from every S consecutive frames for N-distance measure. Since gradual scene change does occur in segment 3 only, we need to ignore the scores in segment 1 due to correlation behavior of the reference frame and its neighboring frames.Since gradual scene change does occur in segment 3 only, we need to ignore the scores in segment 1 due to correlation behavior of the reference frame and its neighboring frames. The correlated distance in segment 1 is C

EXPERIMENTAL RESULTS (1)

EXPERIMENTAL RESULTS (2) Performance Measure Performance Result

EXPERIMENTAL RESULTS (3)

Conclusion This method avoided the false alarms by using the validation mechanism.This method avoided the false alarms by using the validation mechanism. It also proves that the statistical model-based approach is reliable for gradual scene-change detection.It also proves that the statistical model-based approach is reliable for gradual scene-change detection. Experimental results show that a very high detection rate is achieved while the false alarm rate is comparatively low.Experimental results show that a very high detection rate is achieved while the false alarm rate is comparatively low.