Marcus Barkowsky, Savvas Argyropoulos1 Towards a Hybrid Model Provide a structure with building blocks Provide a programming and evaluation environment Invite researchers to evaluate and improve their algorithms Check performance improvements

Marcus Barkowsky, Savvas Argyropoulos2 Image Characterization Sequence Characterization NR Image Distortion Metrics Packet Loss Analysis Bitstream Quality Indicators Temporal behavior of bitstream changes Saliency and Visual Attention Temporal Visual Attention Summarization and Mapping Optimization Reference decoder Quality of decoder implementation HMIX PVS Tracking of Codec- Prediction for Packet Loss Region Annex-B

Marcus Barkowsky, Savvas Argyropoulos3 Properties of Modules Input –PVS, HMIX, output of other modules, … Extent of output value –Pixel, Slice, Frame, Sequence, … –Example: distortion map (pixel), framerate (sequence) Expected range of complexity and quality output –high/medium/low image quality –high/medium/low accuracy –high/medium/low complexity –Information may be used: under temporal constraints (e.g. realtime) to switch off modules that do not match the detected quality range, e.g. JND algorithms in the presence of packet loss

Marcus Barkowsky, Savvas Argyropoulos4 Saliency and Visual Attention Object detection Background segmentation Texture, Luminance, Color, Masking Effects Motion based algorithms Perspective estimation Attraction of gaze by severe degradations PVS Summary of Spatial Degradations HMIX Motion Vectors eventually PVS from Reference decoder

Marcus Barkowsky, Savvas Argyropoulos5 Temporal Visual Attention Temporal visibility of degradations Influence of Scene Cuts on Attention Behavioral changes across sequence PVS HMIX Motion Vectors

Marcus Barkowsky, Savvas Argyropoulos6 Image Characterization Amount of Motion Spatial Frequency, e.g. flat regions Content type classification Detection of Faces, Persons,... PVS HMIX Motion Vectors Related to: Saliency and Visual Attention eventually PVS from Reference decoder

Marcus Barkowsky, Savvas Argyropoulos7 Sequence Characterization Mean motion, global motion uniformity Content type classification, e.g. “Cartoon” Behavioral changes across sequence PVS HMIX Motion Vectors

Marcus Barkowsky, Savvas Argyropoulos8 NR Image Distortion Blockiness, Blurriness on block level Blurriness, Sharpness on image level Frame rate Pausing and skipping, Rebuferring PVS HMIX QP, DCT Coeffs Related to Packet Loss Analysis Related to Packet Loss Analysis

Marcus Barkowsky, Savvas Argyropoulos9 Packet Loss Analysis Discontinuities, Artifacts Efficiency of Error Concealment in decoder Position and length in bitstream Hypothetical reference decoder, Leaky Bucket PVS HMIX Editors note: These blocks are not independant eventually PVS from Reference decoder

Marcus Barkowsky, Savvas Argyropoulos10 Tracking of Codec Prediction for Packet Loss Prediction Discontinuity Analysis Tracking via MV, Block Type Influence of QP PVS HMIX Editors note: These blocks are not independant

Marcus Barkowsky, Savvas Argyropoulos11 Bitstream Quality Indicators Framerate Picture Size MB type and QP MB type and Bitrate Motion Vectors and subblock pattern DCT coeff. distribution HMIX

Marcus Barkowsky, Savvas Argyropoulos12 Temporal behavior of bitstream changes Rapid changes in QP HMIX

Marcus Barkowsky, Savvas Argyropoulos13 Functions that produce a distortion map Temporal Error Visibility Spatial error visibility Optimization Saliency and Visual Attention Temporal Visual Attention Toolbox of mapping function, linear, sigmoid etc. OMOS Summation Match with Continuous Quality Evaluation of subj. experiments