NTT Confidential Test plan for evaluation of video quality models for use with stereoscopic three-dimensional television content Taichi Kawano (NTT)

NTT Confidential Aim/Target of 3D objective metric project Aim: Develop FR media-layer objective metric for stereoscopic three-dimensional (3D) video quality considering picture quality only Target service: - 3D video distribution service - 3D Blu-ray / DVD Target user: Service / content providers 2

NTT Confidential Summary of discuss points 3 Discuss following 8 issues: 1. Input of objective quality model 2. Subjective rating test 3: Display Specification and Set-up 4. Source Reference Channel (SRC) 5. Hypothetical Reference Circuit (HRC) 6. ILG and proponents 7. Number of tests 8. Schedule

NTT Confidential Discuss point 1/8: Input of objective quality model Model input (3.2): Option 1: FS and SbS format videos are used, where model requires flag whether input is FS or SbS. Option 2: FS format is used for FS and SbS format video (i.e., Flag is not needed). Proposal for input of model: Frame sequential (FS) format Reason: If format is SbS, viewers see FS on 3D monitor (i.e., SbS is converted to FS). Proposal: 1) The SRC sequences need to have a resolution of 1920x1080 (interlaced or progresssive) per view. (agreed) 2) SbS may be considered as an HRC (agreed) 3) The model inputs are a sequence which is combining left and right view into a single file (postponed decision) 4) For displaying, a post-production step may be required to convert the format, e.g. for passive glasses displays (postponed decision) 4

NTT Confidential Discuss point 2/8: Subjective rating test Voting methodology (4.4): ACR-HR on picture quality only # Same as HDTV test Number of viewer (4.7): 24 # Same as HDTV test Viewing distance (4.3.1): 3H (H is picture height) for active shutter glasses displays, for passive displays, 5H will be used if no evidence is provided that a different distance is required (agreed) # Same as HDTV test Number of Viewer for single display (4.3.2): 1 to 3 simultaneous viewers, if not violating 30 degrees of angle and approved by experts (agreed) # If monitor size is small, 1 viewer use single display 5

NTT Confidential Discuss point 3/8: Display Specification and Set-up Display (4.3.3): Professional or consumer 3D monitor for TV/PC, display models shall be communicated prior to the experiment in order to allow rejection by experts 3D displaying system (4.3.3): ・ Active shutter 3D system with glasses ・ Passive polarized 3D system with glasses Allowable minimum monitor size (4.3.3): 23 inches (agreed) Display resolution (4.3.3): Native 2D Display resolution Full-HD (1920 by 1080) 6

NTT Confidential Discuss point 4/8: SRC Resolution (5.1): 1080p 1080i Frame rate (5.1): 25, 30 frames per second for progressive and interlaced, 24, 48, 50, 60 frames per second for progressive # 24 fps is used in 3D blu-ray # 25 and 30 fps is widely used in 2D video application How to get source sequences (5): TBD # Possible to reuse 3D sequences of other 3D project? Video format (6.2): Proposal for AVI format: Uncompressed AVI files in UYVY Reason: Although UYVY was used in HD test plan, I420 is widely used in market and color blurring does not occur in progressive format. 7

NTT Confidential Discuss point 5/8: HRC Coding Schemes (7.2.1): ・ MPEG-2 (Main level), may be considered for removal when testplans are created ・ H.264 (AVC high and main profiles) ・ H.264 (MVC Stereo high profile) HEVC # MPEG-2, AVC, and HEVC are applied to each view independently, the video is encoded as a single frame in SbS format Video bit-rate (7.2.2): Very coarse compression quality shall not be included in this experiment as it is unrealistic for 3D (inducing visual discomfort, switching off the second view will be preferred), HRC shall be generated such that MOS scores larger than 2 are expected for sequences without transmission errors, typical bitrate (without being restrictive) would be MPEG-2 : Mbps for each view or SbS AVC: Mbps for each view or SbS MVC: Mbps Video encoding mode (7.2.3): CBR, VBR Transmission errors: not allowed 8

NTT Confidential Discuss point 6/8: ILG and proponent Possible to set up ILG?: TBD # Potential ILG (declared in Rennes): AGH, Acreo, IRCCyN, NTIA # If impossible, only proponents run test Who propose model and conduct tests?: NTT, … # Potential proponents (declared in Rennes): Yonsei, SwissQual, Opticom, DT, Orange FT, NTIA, IRCCyN 9

NTT Confidential Discuss point 7/8: Number of test Number of datasets (4.1): Minimum of 3 # One for each frame rate (24, 25, 30 fps) => 3 tests # 3 frame rates ✕ 3 codecs ✕ (FS + SbS) => 18 tests How many tests for each proponent and ILG?: NTT: 1 or 2 tests XXX: * test YYY: * test ZZZ: * test … 10

NTT Confidential Discuss point 8/8: Schedule Approve test plan: June 2013? # Next VQEG meeting Declare intent to participate: July 2013 Submit model: Feb. 2014? # VQEG meeting after next Select SRC: Mar Generate PVS: Aug Run subjective test: Dec Statistical analysis: Jun Approve final report: Mar