1. A Unified Approach to Calibrate a Network of Camcorders & ToF Cameras M 2 SFA 2 Marseille France 2008 Li Guan Marc Pollefeys {lguan, marc}@cs.unc.edu UNC-Chapel Hill, USA ETH-Zurich, Switzerland

2. 2 ToF Camera (RIM sensor) Theory –Time of Flight Fig. from 3DV system website Products –Canesta cameras –Swiss Ranger –PMD cameras –ZCam

3. 3 Advantage –high frame-rate (50 fps.) –Depth image + amplitude image Drawback –low resolution (e.g. 176x144, SR3100) –depth measurement is still not stable Solution for reconstruction: –A network of ToF cameras & video camcorders –Challenges calibration robust shape estimation http://www.3dcgi.com/images/face_2d_3d.jpg 3D Sensors (cont.)

4. 4 Recovering sensor location, orientation and imaging parameters Traditional calibration target –Checkerboard Z. Zhang ICCV’99 J.-Y. Bouget’s toolbox –Laser pointer, etc T.Svoboda MIT press ’05 Svobod’s toolbox Our proposal –A sphere with unknown radius Calibration of the Sensor Network

5. 5 Video Camcorder –Observation: due to projective distortion, the image of a sphere is an ellipse, and sphere center is NOT the center of the ellipse, –An ellipse is defined with 5 parameters –If we know the intrinsics of the camera, it can be simplified to 3 Hough transform Sphere Center Extraction

6. 6 Hough Transform Given the undistorted optical center position, the ellipse detection is a 3-parameter Hough transform –Radius of the sphere tangent to the cone at plane Z=-1 –Row and Col of the sphere center in the image Fit the final result to get sub-pixel accuracy

7. 7 ToF Camera –Observation: intensity highlight in the “amplitude image” Detect & track the sphere highlight Fit parabolic surface to get sub-pixel accuracy Sphere Center Extraction (cont.) Camera optical center

8. 8 Setup –4 fixed position vision sensors 2 Canon HG10, 1920x1080, 25Hz 2 SR3100, 176x144,20Hz Calibration Result

9. 9 Radius recovery Scale recovery Sphere Radius & Scale Recovery R = 0.0248 S = 11.3386 R’ = RS =0.0248x11.3386 = 0.2824m Measured circumference = 1.7925m, the actual radius = 0.2853m

10. 10 Overview Robust Shape Estimation

11. 11 Sensor Fusion Framework Notations – as the binary state space – as the sensor models – as the sensor observations (L. Guan, J.-S. Franco, M. Pollefeys, 3DPVT 2008)

12. 12 Main Formula Bayes rule

13. 13 Results For MATLAB code, check out http://www.cs.unc.edu/~lguan Volume size 256 3 Threshold at 0.875 Computation Time ~ 3 min. (MATLAB)

14. 14 Summary & Future Work Calibration –Depth calibration Separate scale factor for each sensor reflection - depth accuracy analysis Reconstruction –More general sensor fusion –Ultimate challenge of outdoor environment Synchronization and video processing GPU Algorithm speedup