1. IEEE 802.21 MEDIA INDEPENDENT SERVICES DCN: 21-19-0002-00-0000
Title: Introduction to VR SG
Date Submitted: January 7, 2019
Presented at IEEE session #89 in St. Louis, USA
Authors or Source(s):  Jeong, Sangkwon Peter (JoyFun)
Seo, Dong-Il Dillon (VoleR Creative)
Presenter:  Seo, Dong-Il Dillon (VoleR Creative)
Abstract: This document describes the current VR system layout and the industrial problems around so that the participants for the SG is aware of the problems they need to solve.

2. IEEE 802.21 presentation release statements
This document has been prepared to assist the IEEE Working Group. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.
The contributor grants a free, irrevocable license to the IEEE to incorporate material contained in this contribution, and any modifications thereof, in the creation of an IEEE Standards publication; to copyright in the IEEE’s name any IEEE Standards publication even though it may include portions of this contribution; and at the IEEE’s sole discretion to permit others to reproduce in whole or in part the resulting IEEE Standards publication. The contributor also acknowledges and accepts that this contribution may be made public by IEEE
The contributor is familiar with IEEE patent policy, as stated in Section 6 of the IEEE-SA Standards Board bylaws < and in Understanding Patent Issues During IEEE Standards Development
IEEE presentation release statements
This document has been prepared to assist the IEEE Working Group. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.
The contributor grants a free, irrevocable license to the IEEE to incorporate material contained in this contribution, and any modifications thereof, in the creation of an IEEE Standards publication; to copyright in the IEEE’s name any IEEE Standards publication even though it may include portions of this contribution; and at the IEEE’s sole discretion to permit others to reproduce in whole or in part the resulting IEEE Standards publication. The contributor also acknowledges and accepts that this contribution may be made public by IEEE
The contributor is familiar with IEEE patent policy, as outlined in Section 6.3 of the IEEE-SA Standards Board Operations Manual < and in Understanding Patent Issues During IEEE Standards Development

3. Use Cases

4. Case 1
Case 1
HMD is connected to a local content server such a PC or a gaming console by a wired or wireless network.
VR content is being rendered or decoded in the local content server and HMD is receiving the VR content through either a wired or a wireless network.
Local Content Server
LAN
HMD
* LAN: wired or wireless
※ Slide Source: Seo, Dong-Il Dillon, VoleR Creative( , IEEE th meeting)

5. Case 2
Case 2
HMD is connected to a remote content server such as cloud rendering by wired and wireless network.
VR content is being rendered or decoded in the remote content server and streamed to the HMD.
The remote content server is connected via WAN as the remote content server is located outside of the local area.
Remote Content Server
WAN
HMD
* WAN: wired + wireless
※ Slide Source: Seo, Dong-Il Dillon, VoleR Creative( , IEEE th meeting)

6. Case 3
Case 3
This is an extended version of case 1 – more than one VR system are connected to the remote content server.
HMD is connected to a local server like case 1 and the local content server is rendering or decoding VR content and send it back to the HMD.
The remote content server in this case is computing the content sent by the local content servers and redistributing the calculated data back to the local content servers rather than rendering the content in local content server.
Remote Content Server
WAN
Local Content Server
Local Content Server
LAN
LAN
HMD
HMD
* LAN: wired or wireless
* WAN: wired + wireless
※ Slide Source: Seo, Dong-Il Dillon, VoleR Creative( , IEEE th meeting)

7. Case 4
Case 4
This is an extended version of case 2 - more than one HMD are connected to the remote server.
HMD is connected to a remote server like case 2 and the remote server is rendering or decoding VR content and send it back to the HMD.
Remote Content Server
WAN
HMD
HMD
* WAN: wired + wireless
※ Slide Source: Seo, Dong-Il Dillon, VoleR Creative( , IEEE th meeting)

8. Diagrams Case 1 Case 2 Case 3 Case 4 * LAN: wired or wireless
Local Content Server
Remote Content Server
Remote Content Server
LAN
WAN
WAN
HMD
HMD
Local Content Server
Local Content Server
Case 4
LAN
LAN
Remote Content Server
HMD
HMD
WAN
* LAN: wired or wireless
* WAN: wired + wireless
HMD
HMD
※ Slide Source: Seo, Dong-Il Dillon, VoleR Creative( , IEEE th meeting)

9. Industry Problems VR sickness is a subset of Motion Sickness
The IG focuses on the VR HMD based VR Sickness only
Caused by the mismatch between the visually received information and the vestibular system
Various Standard Organizations such as IEEE 3079, IEEE 2048, Khronos Group, ISO/IEC JTC1/SC24, SC29 WG11(MPEG), W3C, ITU-T SG12 are developing technical standards for VR
Car Sickness
Air Sickness
Sea Sickness
Space Sickness
Motion Sickness
Cybersickness
Simulator Sickness
VR Sickness
HMD VR sickness에 초점을 맞추어야 한다.
이 슬라이드에서 VR Sickness가 one of Cybersickness라는 것은 크게 중요한 내용이 아니다.
※ VR: Virtual Reality
※ HMD: Head Mounted Display

10. Technical Requirements for VR QoE
Technicolor, Oct (m39532, MPEG 116th Meeting)
Requirement
details
pixels/degree
- 40 pix/deg
- no HMD is capable of displaying 40pix/deg today
video resolution
- 3 times 4K(3840x1920) vertical resolution = 11520x6480
framerate
- 90 fps
- a 90fps framerate offers a latency low enough to prevent nausea
3D Audio
- support of scene-based and/or environmental audio
- 360 surround sound, object-based audio, Ambisonics
motion-to-photon latency &
motion-to-audio latency
- how much time there is between the user interacts and an image / audio
- maximum 20ms
foreground & parallax
- objects in the foreground shall be far enough to prevent nausea
- if objects are too close it is likely they can become an important cause of nausea
- interactive parallax with background shall be present for such objects
- pic1 shows how it is possible to look behind the figure in the foreground
※ Sourced by Prof. Lyou, Eunseok (Gacheon Univ.)

11. Technical Challenges
Network requirements are directly related to MTP latency
For VR HMD based VR Content Service, the network should provide the following features:
Link Layer Issues: Jitter, Latency
High Layer Issue: QoS, QoE & Mobility
※ MTP: Motion to Photon
Tracker
CPU
GPU
Display
Photons
Optics
Head motion
User’s retina
Under 20 milliseconds of latency for comfortability

12. Industry Relevance
Securing QoE is important for the following industry participants (Content Delivers)
Mobile Network Infrastructure Providers
Immediate killer app for 5G is immersive media (VR & AR) for Telecom Companies
Need to have good VR QoE to grab more users to their 5G network
HMD Manufacturers
Stand-alone also known as all-in-one devices are designed to create a better QoE as it eliminates the cord that was attached to the device
Added processor and batteries in a limited physical case space increase the weight and the heat; hence network solution is required to for cloud VR service
Content Creators
Creating more compelling visual experience comparable to the modern PCs and gaming consoles for VR using the stand-alone device is very challenging
Current wired system limits the QoE of VR content; hence, needs a cloud VR
Platform Holders
Want to aggregate compelling VR content to grow their VR content ecosystem

13. IEEE 802.21 Interest Group Activities
For the last few months, IG produced the followings:
Produced a White Paper on problem statement, use case and analysis of existing technologies
Conducted a test to verify the application data throughput requirements (MPEG test procedure) in a constraint environment
Test Server : Linux Ubuntu
Test Sequences : Gaslamp, KiteFlite
Quantization Parameters : 22, 27, 32, 37
FPS : 30, 60, 90
Resolution : HD, UHD 4K, 8K
Encoder/Decoder: HEVC/H.265 Reference SW

14. Test Sequences (Defined by ISO/IEC MPEG Standard-I)
- Gaslamp
- KiteFlite
Application throughput (8K Res & 90 FPS): 40 ~ 160 Mbps
Test is conducted on non-real time application
Future test will investigate the data throughput at the network and the link layers
※ Source: Prof. Ryu, Eunseok (Gacheon Univ.)

15. SG Scopes Wire side Use case 1, 3 Wireless side Use case 2, 3, 4
Handover side
Use case 2, 3 and so on.

16. Next Steps for SG PAR’s scope fix to until April meeting