Practical Camera Auto-Calibration Based on Object Appearance and Motion for Traffic Scene Visual Surveillance Zhaoxiang Zhang, Min Li, Kaiqi Huang and Tieniu Tan National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences Reporter: Chia-Hao Hsieh Date: 2009/11/3 CVPR 2008

Outline Introduction Methods – Extract foreground – Estimate vanishing points – Auto-calibration Experimental results

Introduction Recover intrinsic and extrinsic parameters of cameras – Based on appearance and motion of objects – Measure the camera height only Practical Camera Auto-Calibration Based on Object Appearance and Motion for Traffic Scene Visual Surveillance

Motion Detection Disadvantages of Gaussian Mixture Model – Fast illumination changes – Shadows Methods deal with the disadvantages – Model each pixel as the product of irradiance component and reflectance component – Model each reflectance component as a mixture of Gaussian

Outline Introduction Methods – Extract foreground – Estimate vanishing points – Auto-calibration Experimental results

Vanishing Points Estimation Helpful general properties – Moving objects move on the ground plane – Vehicles run along the straight roadway – Vehicles are rich in line segments along two orientations – Pedestrians walk with their trunks perpendicular to the ground plane

Vanishing Points Estimation Coarse Moving Object Classification – Two directions Velocity direction Main axis direction – Difference of direction K-Mean clustering Thresholding θ < 5° θ > 20° from moment analysis of silhouette

Vanishing Points Estimation Line Equations Estimation Vehicles are rich in line segments along two orientations – Histogram of Orientated Gradient

Vanishing Points Estimation Intersection Estimation – Least square – Levenberg-Marquardt – RANSAC – But… more and more frames Voting strategy – Each point lying on every line generates a Gaussian impulse in the voting space Vehicles: 2 vanishing points Pedestrians: 1 vanishing point

Outline Introduction Methods – Extract foreground – Estimate vanishing points – Auto-calibration Experimental results

Camera Calibration Recovery of K and R – 3 vanishing points 3 orthogonal directions of world coordinate system 3 DOF Assume α u = α v = f s = 0 Assume (u 0, v 0 ) is on the middle of image plane 1 DOF3 DOF

Camera Calibration K and λ i solved Solve R

Camera Calibration Recovery of T – Choose one arbitrary reference point (u 4,v 4 ) from image plane to correspond to the origin of the world coordinate system Camera height H is measured The optical center of the camera lies on the z = H plane Propose a method of complete calibration of surveillance scenes with three estimated orthogonal vanishing points and the measured camera height H

Experimental Results 720 × 576(u 1, v 1 ) = (217, 70) (u 2, v 2 ) = (1806, 31) (u 3, v 3 ) = (427, 4906) α u = α v = 884, (u0, v0) = (336, 226) vary in a small range less than 2%

Conclusions Practical camera auto-calibration method for traffic scene surveillance Completely recover both intrinsic and extrinsic parameters of cameras – Only the camera height H measured – Based on appearance and motion of moving objects in videos Accuracy and practicability Thank you!