Projet AAP FUI11 The NExt Video Experience

Project short presentation Catherine Serré – Technicolor R&D France2012 June 12

Project description French collaborative project (labelled by «Images et réseaux» and «Cap Digital» clusters) Duration 30 months From 2011-October-01 to 2014-March-31 Subject: HDR video for broadcasting Two main goals: Generate HDR video content Explore all the bricks of a HDR video distribution chain from content creation, through distribution, to visualization by the user NEVEx /06/12

Project description NEVEx /06/12

Content creation: Real scene capture The idea is to : reuse the Binocle 3D rig with a null baseline (aligned views) Use an appropriate neutral gray filter on one camera in order to generate two views of the same scene: an under-exposed view and an over-exposed view correct the geometric disparity between images (due to sensor, optics … disparities) with Binocle modified disparity killer application Perform the fusion of the two views to generate a HDR sequence Challenges: Research companies / academics and production companies don’t speak the same language nor share the same vocabulary Need camera and workflow characterization to produce linear content Fusion algorithm have to be robust to video characteristics (temporal luminance stability, robustness to flashes …) How much dynamic range increase (f-stop) is possible? NEVEx /06/12

Content creation: CGI content Create HDR CGI video sequences Generate different kinds of CG sequences which make tone mappers fail Use global illumination engines: physical luminances Control of the luminance and gradient levels in the CG sequences by changing: Light source Material Geometry Camera position Determine which tone mappers are relevant for each class of CG sequence NEVEx /06/12

HDR video processing Video fusion: Required for real content creation from two LDR views Challenges: Temporal aspect: luminance stability over time, robustness to flashes … Interactivity: Semi-automatic for post-production to preserve artistic intent Automatic for end device (distribution solution 2) Video Tone Mapping: Required for LDR display backward compatibility Challenges: Temporal aspect: luminance stability over time, robustness to flashes … Real-Time implementation at decoding stage and near real-time for post-production Interactivity: Semi-automatic for post-production to preserve artistic intent Automatic for end device (distribution solution 2). Guided Tone Mapping ? Inverse Tone Mapping: Short-term: alternative method to produce HDR video content Long-term: Conversion of legacy content into HDR for HDR ditribution chain NEVEx /06/12

HDR video quality assessment Subjective video quality evaluation for different bricks of the chain: Fusion Tone Mapping Inverse Tone Mapping Compression Format representation … Development of an objective HDR video quality metric NEVEx /06/12

Solution 1: reuse legacy infrastructure and demonstrate the benefit of HDR technology right now Pros: legacy infrastructure as is Cons: dedicated to LDR displays Solution 2: reuse legacy equipments and build an end-to-end HDR video chain Pros: LDR and HDR displays addressed legacy equipments Cons: Native HDR not addressed limited to 2 views (quality ?) processing power at decoding stage Challenge: video fusion and TM at decoding stage Solution 3: long-term native HDR video chain Pros: Efficiency Backward compatibility Cons: New equipment => new standard Challenges: New HDR representation: float ? Integer ? Reuse existing integer based compression techniques (MPEG …)? LDR backward compatibility Distribution / Demonstrators NEVEx /06/12