Download presentation
Presentation is loading. Please wait.
Published byGunner Barnfield Modified over 9 years ago
1
Learning Techniques for Video Shot Detection Under the guidance of Prof. Sharat Chandran by M. Nithya
2
Outline Introduction Types of Shot-break Previous approaches to Shot Detection General Approach - pixel comparison, histogram comparison… Recent Work – Temporal slice analysis, Cue Video Our Proposed approaches Supervised Learning using AdaBoost algorithm Unsupervised Learning using clustering Semi-supervised Learning combining AdaBoost & clustering Conclusion
3
Introduction 9,000 hours of motion pictures are produced around the world every year. 3,000 television stations broadcasting for twenty-four hours a day produce eight million hours of video per year. Problems: Searching the video Retrieving the relevant information Solution: Break down the video into smaller manageable parts called “Shots”
4
What is Shot? Shot is the result of uninterrupted camera work Shot-break is the transition from one shot to the next
5
Types of Shot-Break
6
Shot-Break Hard Cut Fade DissolveWipe
7
Hard Cut
8
Fade
9
Dissolve
10
Wipe
11
Shot Detection Methods
12
Goal: To segment video into shots Two ways: Cluster the similar frames to identify shots Find the shots that differ and declare it as shot-break
13
Pervious Approaches to Shot Detection General Approaches –Pixel Comparison –Block-based approach –Histogram Comparison –Edge Change Ratio Recent Work –Temporal Slice Analysis –Cue Video
14
Pixel Comparison Frame N Frame N + 1 x=1 y=1 | P i (x,y) – P i+1 (x,y) | D(i,i+1)= X Y XY
15
Block – Based Approach Frame N Frame N + 1 Compares statistics of the corresponding blocks Counts the number of significantly different blocks
16
Histogram Comparison
17
Edge Change Ratio
18
Comparison… MethodAdvantagesDisadvantages Pixel-Comparison Simple, easy to implement Computationally heavy, Very sensitive to moving object or camera motion Block basedPerforms better than pixel Can’t identify dissolve, fade, fast moving objects Histogram comparisonPerformance is better Detects hard-cut, fade, wipe and dissolve Fails if the two successive shots have same histogram. Can’t distinguish fast object or camera motion Edge Change RatiosDetects hard-cut, fade, wipe and dissolve Computationally heavy Fails when there is large amount of motion
19
Problems with previous approaches Can’t distinguish shot-breaks with Fast object motion or Camera motion Fast Illumination changes Reflections from glass, water Flash photography Fails to detect long and short gradual transitions
20
Temporal – Slice Analysis
22
Cue Video
23
Temporal – Slice Analysis
24
Cue Video Graph based approach Each frame maps to a node Connected upto 1, 3 or 7 frames apart Each node is associated with – color Histogram – Edge Histogram Weights of the edges represent similarity measure between the two frames Graph partitioning will segment the video into shots
25
Proposed Approaches
26
Use learning techniques to distinguish between shot-break and Fast object motion or Camera motion Fast Illumination changes Reflections from glass, water Flash photography
27
Supervised Learning
28
Feature Extraction 25 Primitive features like edge, color are extracted directly from the image These 25 features are used as input to next round of feature extraction yielding 25 x 25 = 625 features This 625 features can be used as input to compute 625 x 625 = 15, 625 features
29
How these features can be used to classify images?
30
Solution : Use AdaBoost to select these features. Oops!! There are 15, 625 features! Applying them to red, green and blue separately will result in 46, 875 features! Can we find few important features that will help to distinguish the images?
31
Input: (x 1,y 1 ) (x 2,y 2 ) …(x m,y m ) where x 1,x 2,…x m are the images y i = 0,1 for negative and positive examples Let n and p be the number of positive and negative examples Initial weight w 1,i = 1/2n if y i = 0 and w 1,I = 1/2p if y i = 1 For t= 1,…T: Train one hypothesis h i (x) for each feature and find the error Choose the hypothesis with low error value update the weight: w t+1,i = w t,i * t 1-e t where e i =0,1for x i classified incorrectly or correctly t =e t /(1-e t ) Normalize w t+1,I so that it is a distribution Final hypothesis is calculated as AdaBoost Algorithm
32
Supervised Learning Extract Highly selective features AdaBoost algorithm to select few important features Train the method to detect different shot-breaks
33
Unsupervised techniques Clustering
34
Unsupervised technique - clustering
35
Hard Cut Dissolve
36
Unsupervised technique Clustering method to cluster into shots Relevance Feedback
37
Semi-supervised Learning
38
Combination of Supervised and Unsupervised Few labeled data are available, using which it works on large unlabeled video Steps: AdaBoost algorithm to select features Clustering method to cluster into shots Relevance Feedback
39
Conclusion…
40
Problems with previous approaches: Can’t distinguish shot-breaks with –Fast object motion or Camera motion –Fast Illumination changes –Reflections from glass, water –Flash photography Fails to detect long and short gradual transitions Planning to use AdaBoost learning based clustering scheme for shot-detection
41
Thank you…
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.