1. Pascal Kelm Technische Universität Berlin Communication Systems Group Thursday, 2 April 2009 Video Key Frame Extraction for image-based Applications

2. Core partners: TUB (coord.), EPFL, TUD, QMUL Aims: Generation of tags and metadata (signal processing and/or users’ annotation) Keyframe extraction for image search engines Video classification, clustering and summarization Propagation of tags (object detection/ similarity) 2 Goals

3. Video Key Frame Extraction for image-based Applications Preparation: Setup database from unstructured channel “Travel” on YouTube.com –100 videos + affiliated metadata (keywords, comments, user information etc.) –Annotation of shot boundaries (by TUD and TUB) Demos : www.nue.tu-berlin.de 3 Database

4. Video Key Frame Extraction for image-based Applications 4 Youtube Downloader Tool also used by the IRP “Social Media Data Collection“

5. Video Key Frame Extraction for image-based Applications 5 Youtube Downloader Information also interesting for IRP “Social Media Data Collection“

6. Video Key Frame Extraction for image-based Applications -Integration of different expertise: -Image search engines, audio signal processing, video signal processing and text detection/ recognition 6 Overview

7. Video Key Frame Extraction for image-based Applications 7 Overview Shot Subshot Key Frame Video stream: 1.Temporal segmentation Detection of hard cut, fade and dissolve 2.Subshot boundary detection Special importance for single shot videos and for long shots Detection of new visual content appearing through camera operations or undetected transitions. 3.Extraction of key frames efficient key frame extraction as a handshake between existing image search engines and video domain

8. Video Key Frame Extraction for image-based Applications 8 Key Frame Extraction

9. Video Key Frame Extraction for image-based Applications 9 Key Frame Extraction

10. Video Key Frame Extraction for image-based Applications 10 Demo – Chapter Generation

11. Video Key Frame Extraction for image-based Applications 11 Demo – Storyboard-like Key Framing