Zeqi Lai, Y. Charlie Hu, Yong Cui, Linhui Sun, Ningwei Dai Furion: Engineering High-Quality Immersive Virtual Reality on Today’s Mobile Devices Zeqi Lai, Y. Charlie Hu, Yong Cui, Linhui Sun, Ningwei Dai Tsinghua University *Purdue University MobiCom 2017

Furion Background Motivation: understanding the QoE gap between goal and reality Furion design Performance evaluation Conclusion

Furion Background Growing market penetration of VR Headset, Controller, Renderer QoE requirements Good responsiveness(e.g. latency < 16ms) High-Quality visual effects Mobility Today`s high-quality VR systems are tethered Can we enable high-quality VR on today`s mobile device and wireless network?

Furion Background Motivation: understanding the QoE gap between goal and reality Furion design Performance evaluation Conclusion

Furion Local Rendering Lower BRISQUE value, better visual quality. Visual quality ↑, FPS↓(per-frame latency ↑) Conclusion: local rendering is not doable

Furion Remote Rendering Rendering frames by a desktop(GTX 1080) and streaming frames to the phone via 802.11ac WLAN (available peak bandwidth :≈ 400Mbps) Transferring a high-quality frame takes about 200ms

Furion Will future hardware help? Future CPU/GPU is unlikely to help because of the power constraint.

Furion Will future networks help? Also unlikely to directly help due to the significant packet processing overhead on commodity phones 400Mbps TCP throughput already involves 27% CPU on Google pixel phone. Streaming high-quality VR frames in high FPS requires about 6Gbps throughput [MoVR, NSDI 2017]

Furion Background Motivation: understanding the QoE gap between goal and reality Furion design Performance evaluation Conclusion

Furion Remote rendering + video compression M: MJPEG, H: H.264, V: VP9

Furion Remote rendering + video compression M: MJPEG, H: H.264, V: VP9 How to get to 16ms?

Furion Key insight I: Key insight II: Mostly, VR content can be divided into foreground interactions and background environment. Key insight II: Interactions and environment have contrasting predictability and rendering workload.

Furion Cooperative rendering How can we reduce 48ms? Mobile GPU -> interactions (about 12ms) Remote GPU -> environment (about 48ms) Combine them on phone How can we reduce 48ms?

Furion The virtual world is discretized, and user movement is continuous and predictable Leveraging movement delay to pre-fetch adjacent frames Using pre-rendered and pre-encoded panoramic frames to batch all possible orientations.

Furion Decoding an encoded panoramic pictures takes 40ms Use parallel decoding.

Furion Background Motivation: understanding the QoE gap between goal and reality Furion design Performance evaluation Conclusion

Furion Performance evaluation Visual quality(the higher SSIM, the better) and FPS

Furion Performance evaluation Responsiveness

Furion Performance evaluation Resource usage

Furion Conclusion Developed Furion, a VR framework that enables high-quality, immersive mobile VR on today`s mobile phone and wireless network. Furion design is based on a split rendering architecture leveraging a key insight about VR, the contrasting of foreground interaction and background environment. Furion achieves acceptable QoE on commodity phones.