Introduction to VR Ching-Chi Lin Institute of Information Science, Academia Sinica Department of Computer Science and Information Engineering, National Taiwan University

Agenda Introduction Research Issues Demo

What is VR Virtual Reality(VR) replicates an environment that simulates a physical presence in places in the real world or an imagined world, allowing the user to interact in that world.[1] ◦ Artificially create sensory experiences, which can include sight, touch, hearing, and smell. [1]

The Matrix(1999)

Current Status Focus on: ◦ Sight ◦ Hearing

Side-by-side Image

Augmented Reality(AR)

Mixed Reality VR: virtual objects + virtual environment AV: real objects + virtual environment AR: virtual objects in real environment

VR Devices Head Mount Device(HMD): ◦ Oculus Rift ◦ Samsung Gear VR ◦ Sony PlayStation VR ◦ HTC Vive ◦ Google Cardboard ◦... Position Tracker, controller, joystick, …etc.

Oculus Rift

Samsung Gear VR

Sony PlayStation VR

HTC Vive

HTC Vive(Cont.)

Google Cardboard

VR Applications Games ◦ FPS, adventure, music, … etc. Media ◦ 3D movie, 360 ° videos ◦ VR movies Art Architecture …

Research Issues Improve user experiences ◦ Make the VR/AR effect more realistic. ◦ Reduce the dizziness ◦ Reduce random wobble of the picture.

Research Issues(Cont.) Mobile VR/AR ◦ Find a good balance between QoE and battery life of a mobile device. ◦ Dynamically scale resolution and refresh rate to reduce mobile GPU power consumption. ◦ Use user head movement information to decide the appropriate QoE metrics.

Research Issues(Cont.) Cluster-Assisted VR ◦ Move the computations from local device to a server cluster, such that users without high- end hardware equipments can also experience VR fluently. ◦ How to allocate sufficient computing resources to different VR applications in the cluster?

Research Issues(Cont.) Develop new VR/AR applications.

Our Current Working Items Short- to mid- term ◦ Reduce the quality when users quickly move their heads in order to maintain FPS. ◦ Compensate the “shake” due to environment or unaware movements from user. Long term ◦ An infrastructure for cluster-assisted VR. ◦ Improve power-efficiency on mobile VR.

Recruiting Master students/undergraduates who: ◦ 1. are interested in the topics mentioned above. ◦ 2. already have research issues/ideas about VR. Requirement: ◦ 1. programming skill  C/C++/Java/python/OpenGL/… ◦ 2. paper study and presentation ◦ 3. algorithm design or system implementation

Thank you!

Demo