On the use of hierarchical prediction structures for efficient summary generation of H.264/AVC bitstreams Luis Herranz, Jose´ M. Martı´nez Image Communication 2009

Outline Introduction System Framework – Skimming Unit (SU) – Coding order – Generation of the summary – Presentation of the summary Experiment

Introduction MPEG-2 prediction structure

Introduction Hierarchical prediction structure in H.264/AVC using P frames (delay 0)

Introduction Hierarchical prediction structure in H.264/AVC using B frames (delay 8)

System Framework Analyzing input bitstreamGenerating the summarized bitstream # frames selected Skimming unit (SU) -The selection of SUs -Temporal level

Skimming Units Skimming Unit (SU) Temporal layer Index 1 1

Skimming Units Overlapped SUs

Generation of the Summary Bit stream adaptation guided by skimming parameters

Presentation of the Summary Constant frame rate Variable frame rate – Signaling the changes of frame rate – Using a predefined set of SPSs

Summarization Modalities Frame selectionSkimming curve

Image Storyboard Clustered using K-means algorithm

Video Skimming Face detection Moving median filter (a) Frames with face detected (b) Skimming parameters frames per GOP (c) Skimming parameters -- 2 frames per GOP

Fast Preview Motion activity Median filter (a) Filtered activity (b) Skimming parameters frames GOP (c) Skimming parameters -- 2 frames GOP

Subjective Evaluation Visual distortionSemantic distortion

Performance Evaluation Coding efficiency and processing time

Performance Evaluation Bit stream extraction – Low computational cost – Faster generation of the bit stream

skim storyboard Fast playback