Projector Calibration of Interactive Multi-Resolution Display Systems 互動式多重解析度顯示系統之投影機校正 Presenter: 邱柏訊 Advisor: 洪一平 教授

Outline Introduction Related Work Calibration of Interactive Tabletop Displays Calibration of Interactive Cylindrical Displays Experiments and Error Analysis Conclusion 2

Introduction Multi-Resolution Display System Features A future personal desk Cost-effective large high-res display 3

i-m-Top Fovea Projector Mirror & PTU Peripheral Projector IR Camera IR LEDs 4 Introduction

5 Motivation Disadvantages of Manual Calibration Tedious and inefficient Technician dependent accuracy Goal Automatic Calibration Fast and accurate

Outline Introduction Related Work Calibration of Interactive Tabletop Displays Calibration of Interactive Cylindrical Displays Experiments and Error Analysis Conclusion 6

Related Work Automatic Projector Calibration Smarter Presentations Camera-assisted approach R. Sukthankar, R. Stockton, and M. Mullin, “Smarter Presentations: Exploiting Homography in Camera-Projector Systems,” In Proceedings of International Conference on Computer Vision (ICCV), C T P = CT

Related Work Automatic Projector Calibration Lee et al. Optical sensors embedded Gray code patterns Lee, J., Dietz, P., Aminzade, D., Raskar, R., and Hudson, S. "Automatic Projector Calibration using Embedded Light Sensors", Proceedings of the ACM Symposium on User Interface Software and Technology, October Gray Code Pattern

Outline Introduction Related Work Calibration of Interactive Tabletop Displays Hardware Configuration Implementation Calibration of Interactive Cylindrical Displays Experiments and Error Analysis Conclusion 9

Hardware Configuration Color Camera 10 Calibration of Interactive Tabletop Displays

Implementation Step1. Peripheral Projector Calibration Step2. Fovea Projector Calibration Step3. Infrared Camera Calibration Calibration of Interactive Tabletop Displays 11

Peripheral Projector Calibration Implementation Surface( S ) Peripheral Projector ( PP ) Our goal H PP S 12 Camera( C )

Peripheral Projector Calibration Implementation 13 White Pattern H C S

Peripheral Projector Calibration Implementation 14 H C S H C PP Visual Pattern Option 1: Circle Pattern Option 2: Concentric Circle Pattern Option 3: Middle Line Pattern

Peripheral Projector Calibration Implementation 15 H C S Gray Code Pattern H C PP H S H C S = H C -1

Implementation Peripheral Projector Calibration 16

Implementation Step1. Peripheral Projector Calibration Step2. Fovea Projector Calibration Step3. Infrared Camera Calibration 17 Calibration of Interactive Tabletop Displays

Implementation 18 Fovea Projector Calibration Surface( S ) Fovea Projector ( FP ) Cam( C ) Our goal H FP(θ, ϕ ) S

Implementation 19 Fovea Projector Calibration H C S H C FP(θ, ϕ ) Visual Pattern Option 1: Circle Pattern Option 2: Concentric Circle Pattern Option 3: Middle Line Pattern

Implementation 20 Fovea Projector Calibration H C S H C FP(θ, ϕ ) H S H C S = H C -1 Gray Code Pattern

Implementation 21 Fovea Projector Calibration Problem : Project to anywhere desired Range of PTU angle : pan(-3087~3087), tilt(-908~604) Calibration for all PTU angles is impractical Solution Sample PTU angles with fixed interval Interpolate mapping function ( f 1 and f 2 )

Implementation 22 Mapping Function Interpolation Record trajectories of fovea projection

Implementation 23 Mapping Function Interpolation Record trajectories of fovea projection

Implementation Mapping Function Interpolation Interpolate f 1 for one given surface coordinate p s Y X Surface 24

Y X Surface Implementation Mapping Function Interpolation Interpolate f 2 for one given PTU angle (θ, ϕ )

Implementation Fovea Projector Calibration 26

Implementation Step1. Peripheral Projector Calibration Step2. Fovea Projector Calibration Step3. Infrared Camera Calibration 27 Calibration of Interactive Tabletop Displays

Implementation Infrared Camera Calibration Surface( S ) H IRC S H S PP Source Image IR Camera(IRC) Warped Image 28

Implementation Infrared Camera Calibration 29

Summary Calibrate peripheral projector Calibrate IR cameras Calibrate fovea projector Obtain valid projector angle Interpolate mapping function Repeat for all sampled PTU angles 30 : Peripheral Projector Calibration : Fovea Projector Calibration : Infrared Camera Calibration Calibration of Interactive Tabletop Displays

Outline Introduction Related Work Calibration of Interactive Tabletop Displays Calibration of Interactive Cylindrical Displays Hardware Configuration Projector Calibration Experiments and Error Analysis Conclusion 31

Hardware Configuration 90 cm 85 cm 120 cm Surface Projector Surface Front ViewBack View Hardware Component 32 Calibration of Interactive Cylindrical Displays

33 Projector Calibration Piecewise Planar Mapping Texture mapping Using corresponding points between surface & projector Problem Projector intrinsics estimation Calibration of Interactive Cylindrical Displays

34 Projector Calibration Projector Intrinsics Estimation Step1. Initial guess Step2. Measure corresponding points m and M P0P0 P1P1 P2P2 P3P3 P4P4 P9P9 P8P8 P7P7 P6P6 P5P5 Projection Surface Marker : m : M a = [f x, f y, c x, c y ] Calibration of Interactive Cylindrical Displays

35 Projector Calibration Projector Intrinsics Estimation Step1. Initial guess Step2. Measure corresponding points m and M Step3. Estimate extrinsics E with a, m, M Step4. Update with E, m, M Step5. Repeat step 3, 4 until a = [f x, f y, c x, c y ] a = a + △ a || △ a || < ε Calibration of Interactive Cylindrical Displays

Projector Calibration Pre-Warping Source ImageWarped Image 36 Calibration of Interactive Cylindrical Displays

Outline Introduction Related Work Calibration of Interactive Tabletop Displays Calibration of Interactive Cylindrical Displays Experiments and Error Analysis Conclusion 37

ModePP FP (1 set) FP (36 sets) FP (300 sets) Manual1~2 40~60X Auto0.1 ~4.8~40 Unit : minute FP : Fovea Projector PP : Peripheral Projector 38 Experiments and Error Analysis - Calibration of Tabletop Displays Comparison of Calibration Time Manual vs. Auto

Error Analysis Calibration Error of Peripheral Projector 39 Experiments and Error Analysis - Calibration of Tabletop Displays

Error Analysis Calibration Error of Fovea Projector Without mapping function interpolation 40 Experiments and Error Analysis - Calibration of Tabletop Displays

Error Analysis Calibration Error of Fovea Projector With mapping function interpolation 41 Experiments and Error Analysis - Calibration of Tabletop Displays

42 Experiments and Error Analysis - Calibration of Tabletop Displays Applications are run after Calibration

Outline Introduction Related Work Calibration of Interactive Tabletop Displays Calibration of Interactive Cylindrical Displays Experiments and Error Analysis Conclusion 43

44 Conclusion We propose an automatic projector calibration method of interactive tabletop displays, which is both efficient and accurate The fovea projector is able to correctly project over the whole tabletop surface Calibration of cylindrical displays is achieved with semi-automatic approach

Thanks for your listening 45