Omnidirectional camera calibration Ph.D. student Chang-Ryeol Lee November 26, 2013

Contents Camera model What is camera calibration? Camera calibration Tsai’s and Zhang’s methods Omnidirectional camera Calibration: A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion

Camera model * Prof. Kuk-Jin Yoon’s slides on High-level Image Understanding and Processing

What is calibration? * Prof. Kuk-Jin Yoon’s slides on High-level Image Understanding and Processing

Camera model * Prof. Kuk-Jin Yoon’s slides on High-level Image Understanding and Processing

What is camera calibration?

What is camera calibration?

Camera calibration We know positions of pattern corners with respect to world and image coordinate 3D corners 2D corners x y z {World} Grid size:3cm {Image}

Camera calibration Camera parameters Intrinsic parameter K Focal length, skew, principal point Extrinsic parameter P Rotation, translation

Tsai’s and Zhang’s methods Retrieve not only the intrinsic but also extrinsic parameters Set their Z-coordinate to 0 because the 3D points on the object are coplanar. A homography between m and M m ≅𝐊 𝐑 T | −𝐑 T t M=𝐊 r 1 r 2 r 3 t ′ X Y 0 1 =𝐊 r 1 r 2 t ′ X Y 1 𝐇= h 1 h 2 h 3 =λ𝐊 r 1 r 2 t ′

Tsai’s and Zhang’s methods Solving for Homography

Tsai’s and Zhang’s methods Any rotation matrix is an orthonormal matrix Define the symmetric matrix B as - 6 unknown variable r 1 r 2 t ′ = 𝐊 −1 h 1 h 2 h 3 r 1 𝑇 r 2 = ( 𝐊 −1 h 1 ) 𝑇 𝐊 −1 h 2 = h 2 𝑇 𝐊 −𝑇 𝐊 −1 h 2 =0 r 1 𝑇 r 1 = r 2 𝑇 r 2 = ( 𝐊 −1 h 1 ) 𝑇 𝐊 −1 h 1 = h 1 𝑇 𝐊 −𝑇 𝐊 −1 h 1 = h 2 𝑇 𝐊 −𝑇 𝐊 −1 h 2 𝐁= 𝐊 −𝑇 𝐊 −1 = 𝐵 11 𝐵 12 𝐵 13 𝐵 12 𝐵 22 𝐵 23 𝐵 13 𝐵 23 𝐵 33 = 0

Tsai’s and Zhang’s methods One image offers two linear equations in elements of B We can get intrinsic parameter through 3 or more images because unknown variable( 𝐵 11 ⋯ 𝐵 33 ) is 6. h 1 𝑇 𝐁 h 1 = h 2 𝑇 𝐁 h 2 , h 2 𝑇 𝐁 h 2 =0

Tsai’s and Zhang’s methods Derive equations for the internal parameters of K 𝐵 11 𝐵 12 𝐵 13 𝐵 12 𝐵 22 𝐵 23 𝐵 13 𝐵 23 𝐵 33 = 𝛼 𝑥 𝑠 𝑝 𝑥 0 𝛼 𝑦 𝑝 𝑦 0 0 0 −𝑇 𝛼 𝑥 𝑠 𝑝 𝑥 0 𝛼 𝑦 𝑝 𝑦 0 0 0 −1 𝐁 = 𝐊 −𝑇 𝐊 −1 𝑢 𝑦 = 𝐵 12 𝐵 13 − 𝐵 13 𝐵 23 𝐵 11 𝐵 22 − 𝐵 12 2 λ= 𝐵 33 − 𝐵 13 2 + 𝑢 𝑦 ( 𝐵 12 𝐵 13 − 𝐵 11 𝐵 23 ) 𝐵 11 𝛼 𝑦 = λ 𝐵 11 𝐵 11 𝐵 22 − 𝐵 12 2 𝛼 𝑥 = λ 𝐵 11 𝑠=− 𝐵 12 𝛼 𝑥 2 𝛼 𝑦 λ 𝑢 𝑥 = 𝑠 𝑢 𝑦 𝛼 − 𝐵 13 𝛼 2 λ

Tsai’s and Zhang’s methods After intrinsic parameters, the extrinsics readily follow r 1 r 2 t ′ = 𝐊 −1 h 1 h 2 h 3 r 1 = μ 𝐊 −1 h 1 r 2 = μ 𝐊 −1 h 2 t = μ 𝐊 −1 h 3 μ = 1 𝐊 −1 h 1 r 3 = r 1 × r 2

Omnidirectional camera Dioptric (fish-eye lens) Catadioptric (camera with mirror)

Omnidirectional camera Camera model Catadioptric case Dioptric case

A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion Don’t care whether catadioptric or dordioptric General method even NASA use this currently Key idea Camera modeling by Taylor expansion

A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion No focal length, skew, principal point Calibration parameters Taylor expansion coefficients a Rotation R Translation t

A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion Set their Z-coordinate to 0 by planarity of pattern

A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion Cross product of a corner point make three equations

A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion The 3rd equation can be linearly expressed on unknowns We know Homogeneous linear least square problem is solved by SVD

A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion 𝑟 3 is uniquely determined by 𝑟 1 , 𝑟 2 Extrinsic parameter is determined except for 𝑡 3 The 1st, 2nd equations are used to determine , where

A flexible Technique for Accurate Omnidirectional Camera Calibration and structure from motion Equations stacked with K observations of pattern is expressed linearly where Unknown parameter 𝑡 3 ,𝐚 are estimated 1st Kth

Question or Comments ?