VR/AR project Progress Report 2016/07/21

Live Reality Fusion Concatenate the live videos from two or more rooms together. Observer room + remote room(s) Remote room Observer Room

Short-term Goal “See Through Wall” Remote room Observer Concatenate R106 and R107 in IIS building. Remote room Static => Dynamic => Real-time Observer Still => look around => move around

Stereo Camera ZED stereo camera 3D Camera for Depth Sensing and Motion Tracking. https://www.stereolabs.com/ https://github.com/stereolabs/

Plan Offline construct the model of the remote room using ZED stereo camera. Stack ZED stereo camera on top of Oculus HMD. Capture the view of observer room. Compose what observer should see.

Steps Given the model and an image taken in the observer room from the user perspective. Find the wall to be replaced in the image. By depth map. “Hollowing out” the wall to be replaced. Construct a 3D world in Unity. Model of the remote room. The hollowed image as a plane.

Example

Current Status The stereo camera arrives today. Implementing Will construct the 3D model of the remote room. Implementing Wall/Background identification of a given image from the stereo camera. Passing hollowed image to Unity. Two cameras mode instead of VR mode in Unity.

Discussion