Download presentation
Presentation is loading. Please wait.
1
Video Summarization via Determinantal Point Processes (DPP)
Boqing Gong University of Southern California Joint work with Wei-Lun Chao, Kristen Grauman, and Fei Sha
2
Background Basic idea of DPP Sequential DPP (NIPS 2014) Large-margin training of DPP Conclusion
3
Background Motivation: Representation: Subset Selection problem
Indispensable for fast video browsing and retrieval Representation: Key frames / segments extraction Subset Selection problem
4
Background Video summarization is hard: Naïve solution: Clustering
Individual selected frame: Representativeness Selected frames as a whole: Diversity Naïve solution: Clustering Competing !
5
Background Clustering works?
6
Video summarization: an overview
Video summarization is hard: What criteria lead to user perspective? What kind of models: Supervised learning ! Diverse subset with representative items
7
Background How to model subset selection problem?
Structured prediction, submodular functions Determinantal Point Processes (DPPs) [Alex Kulesza and Ben Taskar, 2012]
8
Background Basic idea of DPP Sequential DPP (NIPS 2014) Large-margin training of DPP Conclusion
9
Basic idea of DPP Idea: A point process based on matrix determinant. Formulation: M discrete items (binary decision)
10
Basic idea of DPP Why diverse? Extreme cases:
11
Basic idea of DPP Learning in DPP: 11
12
Background Basic idea of DPP Sequential DPP (NIPS 2014) Large-margin training of DPP Conclusion
13
Sequential DPP Motivation: Proposed Idea:
The temporal structure of video is missing Proposed Idea: Sequential DPP via Markov properties
14
… …
15
Sequential DPP Modeling the sequential structure:
Conditional DPP: still a DPP !
16
Sequential DPP Parameterization:
17
Inference and Learning
Allow brute-force search in small chunks Optimization:
18
Sequential DPP Experimental setting:
3 datasets: OVP (50), Youtube (39), Kodak (18) Fisher vectors + Saliency + Contextual features Evaluation: Recall, Precision, and F1 score Comparison: unsupervised methods & vanilla DPP
19
Sequential DPP Experimental Results:
20
Sequential DPP Experimental Results:
21
Sequential DPP Experimental Results:
22
Background Basic idea of DPP Sequential DPP (NIPS 2014) Large-margin training of DPP Conclusion
23
Learning parameters in DPP
Maximum likehood estimation Focuses on observed data only Large-margin training Maximizes margin between observed and undesired data Discriminative learning More flexible: incorporating evaluation metrics
24
Large-margin training of DPP
More discriminative and flexible
27
Conclusion Supervised learning for video summarization
DPPs: modeling diversity subset selection Video structure: Sequential DPP Parameterization: Neural networks Future work Better inference algorithms Models beyond DPP (submodular)
Similar presentations
