Subjectif tests requirements for HDR Jean-Charles Gicquel, Jérôme Fournier, Orange Labs VQEG, July 2014 presentation title

Introduction : Context High sensitivity of CMOS sensors Up to 16 f-stops (Sony F65) More and more video cameras Availability of displays with high dynamic range of luminance Up to 5000 Cd/m2 Just 2 references at the moment: Dolby and Sim2 Several demonstrations of such technologies Philips, Technicolor, BBC, Dolby,… Workshop DVB-EBU in London 2013 and Munich 2014 Subjectif tests requirements for HDR

Introduction : Context HDR and EDR under consideration for upcoming video standards UHD-phase2 DVB, ITU, SMPTE MPEG and EBU: need to have a common subjective method to evaluate fairly HDR technology Subjectif tests requirements for HDR

Introduction : Orange position Acquisition of a SIM2 display 2 months ago Characterization a the display Preparation of HDR, EDR contents and their SDR version in a format adapted to the HDR display Visualization of these HDR and EDR contents and their SDR versions on it (viewing session with experts and non-experts) First conclusions Various opinions Experts have mixed feelings regarding the type of contents: sceptical especially for movies contents Globally non-experts appreciate the improvements Subjectif tests requirements for HDR

Introduction : Orange position To evaluate the added value of HDR technology in terms of QoE To compare HDR and SDR references New perceptual dimensions to be assessed? realness, presence, visual discomfort/annoyance… To have a fair comparison of existing and ongoing technologies Impact on current end-to-end video architectures HDR video formats Components Bandwidth and associated video interfaces Transfer function and tone mapping issues Compression schemes SDR backward compatibility  Development of a new subjective test method required Subjectif tests requirements for HDR

Introduction : HDR assessment main issues Classification of test sources Mean, peak and minimum luminance of test material … Test bed definition for subjective assessment of HDR contents To display sources test material reducing potential degradations Format conversion: bit depth, color components, EOTF, TMO Video player To propose a method to adjust HDR screen features Specific to each HDR display? Test charts dedicated to HDR rendering? New subjective test method Viewing conditions  use case relationship (cinema, TV, mobile) Test protocol, subjective scales… Subjectif tests requirements for HDR

Technical architecture new requirements How to present HDR contents on HDR displays with different properties regarding the dynamic of luminance? Exchange file format: OpenEXR? Input file format (components, bit depth): display dependant? Max and min luminance (contrast) Psychophysical senstivity function in accordance with the use case (cinema, TV)? Weber, Barten, Moon & Spencer, …? Transfer function (luminance and color quantization) From 16 bits to 8, 10, 12 bits? What is the position of VQEG considering these issues? File format, conversion tools, selection of psychophysical function… Subjectif tests requirements for HDR

Technical architecture new requirements Additional keys issues How to present SDR content on HDR displays? SDR is the reference at the moment. It is needed to evaluate the added value of HDR compared to SDR How to present REC. 709 contents on HDR display? How to present HDR content on SDR displays? Evaluation of backward compatibility Tone Mapping Operator (Direct and inverse) for content generation Subjectif tests requirements for HDR

Test method (1/3) Feature Factors Related issues HDR Impact General Viewing Conditions Luminance and contrast ratio Peak luminance of display (Visual discomfort or annoyance => Sun glasses, visual fatigue for long duration visualization). Required contrast ratio adapted to eye performances. Yes Background and room illumination Define background and room illumination to avoid visual discomfort and annoyance during visualization: Still 10-15%? Do we have to consider different use cases (Indoor/outdoor with cinema, TV and mobile applications)? Display Physical interfaces, Input video formats, Transfer functions (luminance…) Calibration (luminance and colors) Viewing distance According to the fact that HDR increases the perceived resolution (higher contrast), do we need to reconsider PVD or DVD recommendations? ? Viewing position The peak luminance seems to be dependant of the viewing position in front of the display. Could we have more than one viewer? Subjectif tests requirements for HDR

Test method (2/3) Feature Factors Related issues HDR Impact Source signals Video format Which HDR exchange format? OpenEXR? Yes Video format conversion Components-based conversion (RGB to Log(Y),u,v Selection of test materials Video content complexity 2D video complexity is depending on texture and motion levels. What about HDR complexity? (min luminance, peak luminance, colors,…) Content acquisition and calibration Dynamic Range conversion (Tone mapping issue) Observers Number Stability and reliability of results ? Viewer’s screening Measurement of visual system performances? HDR impact on the HVS behavior? Subjectif tests requirements for HDR

Test method (3/3) Feature Factors Related issues HDR Impact Presentation What about the temporal presentation of SDR and HDR? Yes Test session duration Re-evaluation of duration for presentation, voting, session length Subjective scales Realness, Visual comfort, Visual annoyance, Video quality,... Comparison, Preference… Visual fatigue Objective and subjective measurement of visual fatigue induced by HDR Test Results analysis Viewer factors Rejection criteria, others? ? Multidimensional indicators analysis Statistical methods for analysis, e.g. relation, interaction and combination of subjectively measured QoE indicators Subjectif tests requirements for HDR

Orange proposal Test method To design a subjective test method to answer the following question: What is the added value of HDR in comparison with SDR for TV services in terms of QoE? Technical architecture based on a Sim2 display Test method Based on the Philips test method Different room illuminations EOTF adapted to HDR Contents Preference scale Tests material with various luminance levels (mean values?) Additional tests using others perceptual scales Questionnaires to understand viewers criteria to assess HDR contents Realness, visual comfort/annoyance, perceived video quality… Subjectif tests requirements for HDR

Thank you! presentation title