1. Yu-Gang Jiang, Yanran Wang, Rui Feng Xiangyang Xue, Yingbin Zheng, Hanfang Yang Understanding and Predicting Interestingness of Videos Fudan University, Shanghai, China AAAI 2013, Bellevue, USA, July 2013

2. Large amount of videos on the Internet – Consumer Videos, advertisement… Some videos are interesting, while many are not Motivation More interestingLess interesting Two Advertisements of digital products

3. Applications Web Video Search Recommendation System...

4. Predicting Aesthetics and Interestingness of Images – Datta et al. ECCV, 2006; Dhar et al. CVPR, 2011; N. Murray et al. CVPR, 2012… We are the first to explore the interestingness of Videos Related Work More interesting Less interesting ………

5. Flickr – source: Flickr.com Consumer Video – videos: 1200 (20 hrs in total) YouTube – source: Youtube.com Advertisement Video – videos: 420 (4.2 hrs in total) Two New Datasets

6. Collected by 15 interestingness-enabled queries – Top 10% of 400 as interesting videos; Bottom 10% as uninteresting – 80 videos per category/query Flickr Dataset

7. Collected by 15 ads queries on YouTube 10 human assessors (5 females, 5 males) – Compare video pairs Annotation Interface YouTube Dataset General observation: videos with humorous stories, attractive background music, or better professional editing tend to be more interesting

8. Aim: compare two videos and tell which is more interesting Visual features Audio features High-level attribute features Ranking SVM results Multi-modal fusion vs. Our Computational Framework

9. Feature Visual features Color Histogram SIFTHOGSSIMGIST Audio features MFCC Spectrogram SIFT Audio-Six High-level attribute features ClassemesObjectBankStyle Flower, Tree, Cat, Face… Rule of Thirds Vanishing Point Soft Focus Motion Blur Shallow DOF …

10. Prediction – Ranking SVM trained on our dataset Chi square kernel for histogram-like features RBF kernel for the others – 2/3 for training and 1/3 for testing Evaluation – Prediction accuracy The percentage of correctly ranked test video pairs Prediction & Evaluation

11. Prediction Accuracies(%) Visual Feature Results Flickr YouTube 74.5 67.067.1 76.6 68.0

12. Audio Feature Results Flickr YouTube 65.7 64.8 74.7 Prediction Accuracies(%)

13. Attribute Feature Results Flickr YouTube 64.3 74.8 64.5 Different from predicting Image Interestingness

14. Visual Attribute Prediction Accuracies(%) Visual+Audio+Attribute Results Flickr YouTube 71.7 78.6 76.6 68.0 2.6% 5.4% Audio Visual+Audio+Attribute Visual+Audio

15. Conducted a pilot study on video interestingness Built two datasets to support this study – Publicly available at: www.yugangjiang.info/research/interestingness Evaluated a large number of features – Visual + audio features are very effective – A few features useful in image interestingness do not work in video domain (e.g., Style Attributes…) Summary

16. Thank you ! Datasets are available at: www.yugangjiang.info/research/interestingness