Paul Eastman Chair, SAB Special Projects 14 June 2017 SAB Status Report Paul Eastman Chair, SAB Special Projects 14 June 2017

Special Projects Activity–WGs P1680.1 Standard for Environmental Assessment of Personal Computer Products (Rev.) P1680.2 Standard for Environmental Leadership Assessment of Imaging Equipment P1680.3 Standard for Environmental Leadership Assessment of Televisions P1680.4 Standard for Environmental Assessment of Servers P1680.6 Standard for Environmental Assessment of Complex Set Top Boxes (PAR pending) P1857 Advanced Audio Coding P3333 3D Model Creation Using Unprocessed 3D Medical Data

IEEE 1680.4 IEEE and NSF JTG-ESS Progress A draft Standard was completed and approved to move to first ballot by each SDO on schedule, six months from formation of JTG The first ballot round was approved by NSF and failed within IEEE, as did the first recirculation Over 1100 comments were received and addressed in the first ballot round and over 400 comments in the second round* P1680.4 Results First ballot round: 90% return, 36% affirmative Second ballot round: 93% return, 55% affirmative * IEEE balloters only, additional comments received through NSF process.

IEEE 1680.4 IEEE and NSF JTG-ESS Progress There is a large group (~20%) of balloters, representing a coalition of “right-to-repair” advocates, who will not approve the draft without substantive changes to one section of the Standard. Extended discussions on this section had been held within the JTG previously. All comments were reviewed and subject to consideration, following the standard procedures of the JTG. Limited revisions to this section were accepted. The second recirculation is underway and will likely fail within IEEE (see below). P1680.4 Results (as of 6/12/17) Third ballot round: 93% return, 58% affirmative

IEEE 1680.4 Consequences of Failure of the Third Ballot Round (Second Recirc) The Joint Development Agreement provides that upon failure of the Standard to be approved after the third ballot round, the SDO which has achieved approval (NSF in this case) can publish the standard independently In that event, the JTG may be dissolved and each organization may continue to develop the standard independent of the other Both SDOs share copyright of original material developed during the joint process, and have the right to use, further develop, and publish the language without permission of the other organization

IEEE 1680.4 IEEE Options upon Failure of the Third Ballot Round Continue to develop the standard independent of NSF with the existing 1680.4 WG and ballot group Dissolve the existing WG and reconstitute with a new structure (e.g. entity instead of individual) and issue a new call for balloters once the Standard is ready Discontinue activities on the Standard, as a separate standard covering the same scope will be available from another SDO. (In this instance, NSF has indicated that existing 1680.4 Working Group members could join the NSF continuous maintenance process for the Standard.)

IEEE 1680.4 Next Steps assuming Failure A meeting was held with IEEE SA and the Governance Team of the JTG on 6/8/2017, with the following steps proposed: NSF to determine what actions they will be taking in line with the Joint Development Agreement IEEE SA to determine the IEEE decision process for responding to publication by NSF A meeting to be scheduled for IEEE 1680.4 WG to review the results and assess whether or not the WG wishes to continue IEEE and NSF to work together on joint communication regarding next steps and the JTG

P1680.1 - Environmental Assessment of PCs, Tablets, and Monitors Technical Requirements in Working Draft were completed in February 2017 Includes criteria in 11 environmental assessment categories: Substance management, Materials selection, Design for end of life, Product longevity/life-cycle extension, Energy conservation, End-of-life management, Packaging, Life cycle assessment and carbon footprint, Corporate environmental performance, Corporate social responsibility IEEE P1680.1 WG approved draft for sponsor ballot March 9, 2017 WG Editorial Review and MEC review were completed by WG in March 2017 Footer text

P1680.1 - Environmental Assessment of PCs, Tablets, and Monitors Sponsor Ballot Launched on March 28, 2017 Closed on April 27, 2017 72 eligible people in ballot group Ballot return rate = 83% Approval percentage = 49% 1143 comments were submitted Footer text

Working Group (WG) Status IEEE P1680.1 Officers (stable since April 2015) Chair: Walter Jager (co-funded by GEC and ITI) Secretary: Casandra Robinson (NIST) Vice-Chair: Alexandra Degher (HP) June 16, 2017 Roster: 87 participants 44 WG (voting) members 4 IEEE staff

Recent Progress Ballot Comment Resolution since April 28, 2017 Ballot Resolution Support Team (BRST) created plan and triage for resolution of 1433 comments coordinate small teams to propose disposition of comments Complex/contentious comments discussed by WG for direction/guidance Some comments have complex technical aspects requiring team of experts to investigate and resolve Disposition and edits are approved by full WG

Next Steps (Plan) June 27-29, 2017: WG face-to-face meeting to resolve remaining technical issues Complete the disposition of comments Final editorial review of draft standard July 27, 2017: Final WG approval of draft for re-ballot July 28, 2017: launch re-ballot and submit documentation to REVCOM for Sept 7, 2017 meeting

Challenges and Actions A large number of environmental criteria in draft standard ~50% more criteria than 1680.1-2009 standard Makes standard more difficult to understand and implement Will be discussed at June f2f meeting Stakeholder balance and dominance Manufacturer representatives outnumber govt and NGO participants Ongoing challenge to have WG members consider expectations of all stakeholder groups. Tight timeline for ballot comment resolution Several organization-specific issues raised in comments Need to leverage previous WG discussions and decisions Accelerating SGs to achieve closure on criteria and quality New process being implemented to ensure that each criterion has an editor Parallel criteria development within each SG Consistent capturing of environmental benefit of criteria proposals helps drive closure on technical requirements More use of offline commenting Written comments help avoid circular discussions and drive closure on technical requirements and wording WG driving closure on criteria that have strong opinions from WG members Reduce amount of editorial work being done at WG for each criterion SGs to perform round of editorial commenting on each criterion before presenting for vote at WG New SG process to drive closure of editorial changes before criteria are presented at WG Agreement on CSR criteria to be included within scope of standard SG chair is needed for 4.3 Design for End of Life & 4.4 Product Longevity

IEEE 1680.1 – Estimated Timeline to Completion (update) 3Q2015 Phase 1 SGs complete criteria development (Energy, material selection, EOL, and packaging) 4Q2015 Phase 2 SGs launched for remaining criteria categories 2Q2016-3Q2016 SGs complete criteria development Editorial review 1Q2017 Compile and edit draft standard WG approval to sponsor ballot 2Q2017- 4Q2017 Sponsor ballot Ballot Resolution IEEE Publication Process

P1680.6 Complex Set Top Boxes The Working Group has bee holding monthly meeting via teleconference (using WebEx) since June 2015. 1.1 Project Number: P1680.6 1.2 Type of Document: Standard 1.3 Life Cycle: Full Use 2.1 Title: Standard for Environmental Assessment of Complex Set Top Boxes Includes devices with more than simple channel selection WG Chair is David Daniels has been replaced by Bill Skeates of Sky TV Presentation to CS/SAB 8 June 2016

P1680.6 - Environmental Assessment of Complex Set top boxes Criteria Development Part Completed : Environmental sensitive materials 5/13 Compete Material Selection 5/6 Complete Design for Life Extension 10/12 Complete Project Longevity 1/2 Complete Packaging 2/4 Complete Environmental Performance 8/11 Complete Power Consumption Complete End of Life 1/2 Complete Face to Face meetings planned: July 2017 Footer text

Working Group (WG) Status IEEE P1680.6 Officers (stable since June 2016) Chair: William Skeates (Sky) Secretary: Mark Schaffer (Knowsgreen) Vice-Chair: Mark Schaffer (Knowgreen) Editor Anna Ma (Sky) June 2017 9 voting members 1 observers 1 IEEE staff

IEEE 1680.1 – Estimated Timeline to Completion (update) 1Q2017 Phase 1 complete criteria development Energy Sensitive materials, material selection, env- design durability, 2Q2017 Phase 2 complete criteria for Packaging, environmental performance energy conservation end of life, 3Q2017 Complete contentious criteria development 4Q2017 Compile and edit draft standard SAB ballot 1Q2018- 2Q2018 IEEE Publication Process Sponsor Publication of standard

P1680.2 & P1680.3 Amendments 1680.2 - Standard for Environmental Leadership Assessment of Imaging Equipment 1680.3 - Standard for Environmental Leadership Assessment of Televisions Working Group Chair of both WGs: Jennifer Costley, NY Academy of Sciences Limited criteria in scope of amendments: Footer text

Schedule & Status Currently WGs voting to move amendments to ballot (closes June 11) Invitation to join balloting group closes June 17 Launch IEEE Sponsor ballot – late June 2017 Resolve ballot comments – Aug & Sept 2017 Ballot recirculation (assumes final ballot) – Sept 2017 Submit to IEEE SASB for Dec meeting – by Oct. 16th 2017 SASB approval – Dec 2017 Publish – Jan/Feb 2018

P1857 Purpose and Usage The P1857 family standards can be used for video and audio compression in any possible application systems where bandwidth and/or storage cost are sensitive, like internet video, broadcasting TV, surveillance video, etc.. P1857 video standard has developed a complete industry chain from chips, encoders to end products. It is adopted by China Central Television for their DTMB transmission, and full wireless coverage in China has been achieved. Other applications like ultra high definition, network video, surveillance video are scheduled. Presentation to CS/SAB 10 June 2017

P1857 Published Standards 1857-2013, Standard for Advanced Audio and Video Coding-video Standard was published on June 04, 2013. 1857a-2014, Amendment 1 of IEEE Std. 1857-2013 Standard was published on April 04, 2014. 1857.2-2013, Standard for Advanced Audio Coding Standard was published on Nov.12, 2013. 1857.3-2013, Standard for System of Advanced Audio and Video Coding Standard was published on Jan. 20, 2014. 1857.5-2015, Standard for Advanced Mobile speech and Audio was approved on 5 December 2015 and was published on Jan. 30, 2016. Presentation to CS/SAB 3 June 2015

P1857 Working Groups P1857.4, Standard for 2nd Generation IEEE 1857 Video Coding The PAR was approved on Dec 11, 2013. PAR Expiration Date: 31-Dec-2017 WG vote was conducted in December 2016 and comments were received to suggest scene profile be. Currently cross-checks are being conducted to estimate performance and other parameters. Advanced Scene Profile including tools of block-based background reference will be added on next meeting in June 2017. P1857.6, Standard for Digital Media Content Description The PAR approval date: 11-Dec-2013 WG Vote opens from May 12, 2017 to June 08, 2017 P1857.7, Standard for Adaptive Streaming The PAR approval date: 10-Dec-2014 PAR Expiration Date: 31-Dec-2018 WG vote opens from May 12, 2017 to June 08, 2017 Presentation to CS/SAB 3 June 2015

P1857 Working Groups P1857.8, Standard for 2nd Generation Audio Coding The PAR was approved on 05-Dec-2015 . PAR Expiration Date: 31-Dec-2019 VR audio is considered to be supported in this project. The audio rendering tools is scheduled to be added. P1857.9, Standard for Immersive Visual Content Coding The PAR was approved on 05-Dec-2015 PAR Expiration Date: 31-Dec-2019 The D1 on panoramic video coding is finalized at last meeting held in Beijing in March. The next stage is to improve video coding efficiency of panoramic 3D video. CPS Presentation to IEEE President 12 May 09

3D Human Factor Working Group 25

Working Group History > Entity-based IEEE-SA PAR submission: Feb. 18, 2011 IEEE-SA CAG approval: Mar. 7 – 8, 2011 IEEE-SASB approval: Mar. 29 – 31, 2011 WG - Call for participation: May. 27, 2011 Type of ballot : Entity Entity members: LG Electronics, Korea Telecom (KT), Korea Electronics Association (KEA), Huawei, Mitsubishi Electronic, Electronics and Telecommunications Research Institute (ETRI) 1st meeting in Seoul (Palace hotel) / June 14, 2011 2nd meeting in Seoul (Palace hotel) / Sep. 6, 2011 3rd meeting in Seoul (Palace hotel) / Dec. 19, 2011 4th meeting in Seoul (KEA) / April, 10, 2012 5th meeting in Seoul (ETRI) / July, 27, 2012 6th meeting in Seoul (Kintex) / Oct. 10, 2012 7th meeting in Seoul (Silconfile) / Jan. 11, 2013 26

Working Group History > Indi.-based Type change of ballot : Entity  Individual (Mar. 6, 2013) Revised P3333.1.1 & new P3333.1.2 approved : Mar. 27, 2014 Current individual members: 17 nations, 71 members (Active) Korea, UK, France, China, Japan, Malaysia, Taiwan *) For more detailed information, please refer to the WG roster < Shanghai, China >

Liaison & Collaboration IEEE-SA P3333 and ISO/IEC JTC 1/SC 29/WG 11 - (Nov. 7) Establishment of Category C Liaison Video Quality Expert Group (VQEG) - (Nov. 11, 2011) ISO-TC 159 WG12 – 3D Quality Assessment ITU-T SG9 Subjective assessment methods for 3D video quality

Roadmap (C/SAB/HFVE_WG) 2014 2015 2016 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec P3333.1.1(PAR ) WD v4.0 1st SB 2nd SB AP PB MT QoE and Visual Comfort Assessments of 3DContents PAR approved “IEEE Standard for Quality of Experience (QoE) and Visual-Comfort Assessments of Three-Dimensional (3D) Contents Based on Psychophysical Studies,” in IEEE Std 3333.1.1-2015, pp.1-46, July 10 2015 P3333.1.2(PAR ) Mod. PAR WD v1.0 Mod. PAR 1st SB in 2017 Quality Assessment of 3D Contents and UHD Contents CfP The 21st IEEE P3333.1 Human Factors for Visual Experiences WG Meeting * WD : Working Draft SB : Sponsor Ballot Request AP : IEEE-SASB(RevCom) Approval PB : Publication MT : Standards CfP : Call for PropMaintenanceosal

Roadmap (C/SAB/HFVE_WG) 2015 2016 2017 Mar Jun Sep Dec Mar Jun Sep Dec Feb April May June P3333.1.2(PAR ) WD v1.0 Mod. PAR 1st SB in 2017 Quality Assessment of 3D Contents and UHD Contents The 22nd IEEE P3333.1 Human Factors for Visual Experiences WG Meeting Finalization of P3333.1.2 Proposal of P3333.1.3 P3333.1.3(PAR ) Deep learning-based quality assessment of visual contents PAR voting * WD : Working Draft SB : Sponsor Ballot Request AP : IEEE-SASB(RevCom) Approval PB : Publication MT : Standards CfP : Call for PropMaintenanceosal 3

Scope Separation of P3333.1(Original) Standard for the Quality of Experience (QoE) and Visual Comfort Assessments of Three Dimensional (3D) Contents based on Psychophysical Studies Visual saliency prediction method Visual contents analysis method Spatial characteristics of 3D contents Temporal characteristics of 3D contents Subjective assessment method Continuous assessment methodology Paired comparison methodology P3333.1.2 Standard for the Perceptual Quality Assessment of Three Dimensional (3D) Contents based on Physiological Mechanisms Saliency variation according to the 3D distortions Quality assessment based on stereopsis, binocular rivalry and suppression 31

Suggestion for P3333.1.3 PAR P3333.1.3 (what we are preparing) Standard for the deep learning-based assessment of visual experience based on human factors Scope: Establish new deep learning-based metrics of content analysis and quality of experience (QoE) assessment for visual contents which have been partially considered in the previous two projects. Deep learning models for QoE assessment (multilayer perceptrons, convolutional neural networks, deep generative models) Deep metrics of visual experience from HD, UHD, 3D, HDR, VR, MR contents Deep analysis of clinical (EEG, ECG, EOG and so on) and psychophysical (subjective test, SSQ) data for QoE assessment Deep personalized preference assessment of visual contents

Published– P3333.1.1 Root PAR: P3333.1 (31-Mar-2011) 2nd revised PAR Approval Date: 27-Mar-2014 Previous Title: Standard for the Quality Assessment of Three Dimensional (3D) Displays, 3D Contents and 3D Devices based on Human Factors Revised Title: Standard for the Quality Assessment of Three Dimensional (3D) Contents based on Psychophysical Studies Published in July, 2015

Working Items – P3333.1.2 Request new PAR (06-Nov-2013) PAR Approval Date: 27-Mar-2014 PAR Expiration Date: 31-Dec-2018 Original title: Standard for the Perceptual Quality Assessment of Three Dimensional (3D) Contents based on Physiological Mechanisms Revised title: Standard for the Perceptual Quality Assessment of Three Dimensional (3D) and Ultra High Definition (UHD) Contents

Working Items – P3333.1.2 Current Status of WD Common idea: Image quality assessment metrics for 3D and UHD images Stereoscopic 3D UHD Metric FR-IQA (Cyclopean + FSIM-like) - Shanghai Univ. Cyclopean image -> Gradient, Phase congruency, Disparity RR-IQA (SSIM-like) - Kingston Univ. Luminance, Contrast, Fused edge Real-time Spatial Pooling - Yonsei Univ. Stereopsis / Binocular rivalry / binocular suppression Saliency / foveation IQA - Univ. of Rome Synthesized images for multi-view IQA – Tsinghua Univ IQA model based on depth distortion Sharpness assessment – Yonsei Univ. Perceptual resolution (Viewing geometry) DFT based local sharpness UHD image preference assessment – Yonsei Univ. Features of visual preference Visual preference assessment model DB Packet loss – Kingston Univ. Quality Discomfort, etc. Stereoscopic images plus depth - Tsinghua Univ Quality assessment

New item to be proposed – P3333.1.3 Proposed PAR : P3333.1.3 Title :Standard for the deep learning-based assessment of visual experience based on human factors Scope : Establish new deep learning-based metrics of content analysis and quality of experience (QoE) assessment for visual contents which have been partially considered in the previous two projects.

Next meeting schedule The 23rd IEEE P3333.1 Human Factors for Visual Experiences WG & P3333.1 Meeting Date : 2017 Location: TBD Subject : P3333.1.2 WD finalization & sponsor ballot in early 2017 P3333.1.3 PAR – Deep learning-based visual content assessment