1. VR/AR project Progress Report
2016/07/21

2. Live Reality Fusion
Concatenate the live videos from two or more rooms together.
Observer room + remote room(s)
Remote room
Observer Room

3. Short-term Goal “See Through Wall” Remote room Observer
Concatenate R106 and R107 in IIS building.
Remote room
Static => Dynamic => Real-time
Observer
Still => look around => move around

4. Stereo Camera ZED stereo camera
3D Camera for Depth Sensing and Motion Tracking.

5. Plan
Offline construct the model of the remote room using ZED stereo camera.
Stack ZED stereo camera on top of Oculus HMD.
Capture the view of observer room.
Compose what observer should see.

6. Steps
Given the model and an image taken in the observer room from the user perspective.
Find the wall to be replaced in the image.
By depth map.
“Hollowing out” the wall to be replaced.
Construct a 3D world in Unity.
Model of the remote room.
The hollowed image as a plane.

7. Example

8. Current Status The stereo camera arrives today. Implementing
Will construct the 3D model of the remote room.
Implementing
Wall/Background identification of a given image from the stereo camera.
Passing hollowed image to Unity.
Two cameras mode instead of VR mode in Unity.

9. Discussion