Yannick FranckenChris HermansPhilippe Bekaert Hasselt University – tUL – IBBT Expertise Centre for Digital Media, Belgium

Goal Geometric calibration of a camera w.r.t. a screen

Vision based HCI 3D reconstruction Motivation [Chen et al., SPIE 2002][Gorodnichy et al., SPIE 2002] [Francken et al., CVPR 2008][Nehab et al., CVPR 2008]

Planar mirror Related Work [Funk and Yang, CRV 2007] [Bonfort et al., ACCV 2006]

Planar mirror Spherical mirror –Corner reflections Related Work [Tarini et al., Graphical Models 2005]

Planar mirror Spherical mirror –Corner reflections –Edge reflections Related Work [Francken et al., CRV 2007]

Planar mirror Spherical mirror –Corner reflections –Edge reflections –Surface reflection Increased accuracy Less manual interventions Robust screen reflection detection Our Approach

Concept 1.Mirror detection 2.Screen pixel labeling 3.3D reconstruction

Mirror detection 1.Internal camera parameters K 2.Background subtraction 3.Edge extraction 4.Ellipse fitting 5.2D ellipse to 3D sphere

Screen pixel labeling

Reflection mask

3D reconstruction Reflected ray intersections Plane estimation Grid estimation Known parameters:

Reflected ray intersections Plane estimation Grid estimation Result: 2D pixel u  3D location x x = M. u 3D reconstruction Solution: Find 2D – 2D similarity transform

Overview x = M. u

Error as function of pattern refinement Results Accuracy –Ground truth –[Francken et al., CRV 2007] –Our approach

Error as function of sphere combinations Results

Error as function of sphere combinations Results

Error as function of sphere combinations Results

Error as function of sphere combinations Results

Screen-camera calibration using Gray codes –Increased accuracy –Less manual interventions –Robust screen reflection detection Conclusion

Gradient patterns –Speed! –Quality? Camera defocus –Which patterns are robust? Future Work

Questions? x = M. u