1. Dongyan Xu, Daniel Aliaga, Scott Yost, Jonathan Deutsch

2. MRT Overview

3. Motivations To create a common plane for immersive demonstration and perception Allow users to focus on the tabletop instead of shifting their vision and attention back and forth between the computer screen and the physical objects

4. Advantages Cheaper than VR Works better than head mounting One focus (unlike video-conferencing) Can use computer-unfriendly objects

5. Conference XP Participants Venues RTP - Real-Time Protocol

6. DirectShow Insert source-filter to grab the image from the camera and do camera calibration Insert transform-filter for projector calibration

7. Raw Image

8. Camera Correction Camera is not perfectly aligned with deskop, has spherical distortion, etc. Create Tsai model for calibrated camera Sample Tsai model to warp the camera image into canonical space

9. Corrected Image

10. UI and Graphics

11. Raw Projector Output

12. Projector Calibration Projector is misaligned, projection area doesn’t perfectly overlap with camera field of view Project sequential pattern onto table, scan output via camera to detect projector “usable area” Can then warp images to projector such that they appear squared and aligned

13. Prewarped Output

14. Image Pipeline Camera Correction UI graphics Projector Correction Projector

15. One more problem! Problem: Camera projector feedback loop will saturate image Solution: Synchronize camera and projector

16. Synchronization! Immediately before the camera grabs an image, stop projecting onto the table Result: Camera sees only real objects, nothing from projector Accomplished through hardware synchronization pins on camera, software

17. Applications We provide image processing, networking code – you provide the app. Application #0: Digital Paper

18. Applications Focus: collaborative teaching – young children: origami, other art – school-age: chemistry, physics experiments

19. Applications Your ideas?

20. Interested? Contact: Prof. Dongyan Xu (dxu@cs.purdue.edu)dxu@cs.purdue.edu Prof. Daniel Aliaga (aliaga@cs.purdue.edu)aliaga@cs.purdue.edu