1. Simultaneous surveillance camera calibration and foot-head homology estimation from human detection 1 Author : Micusic & Pajdla Presenter : Shiu, Jia-Hau Advisor : Wang, Sheng-Jyh 1. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition

2. Outline Introduction Human Detection Foot-head homology estimation Conclusion

3. Introduction This paper uses people to calibrate the camera Human contour detection (green) Refined human detection with camera calibration parameters (blue) Foot-head homology(o:foot,x:head)

4. Concept Objects are human Estimate camera parameters by observing a person standing at several positions 3-D scene2-D projection Image

5. System Flow Sequential Images Human Detection Foot-head Homology Estimation Output

6. System Flow Sequential Images Human Detection Foot-head Homology Estimation Output

7. Background Shape-based detector(Global search) – Detection rate drop significantly in presence of occluded humans Part-based detector(Local search) C. Beleznai and H. Bischof.,“Fast Human Detection in Crowded Scenes by Contour Integration and Local Shape Estimation”, In CVPR,2009.

8. Background Left - Shape based : Template matching with head and body Right - Part based : Obtain foreground image by background subtraction Segmentation of detected human

9. Result : Contour template

10. Human Detection Line edges model a human Offline: Create around 1000 human contours based on 3D model and moving and rotating camera

11. Draw foot-head lines in one image 2-D Image3-D scene

12. System Flow Sequential Images Human Detection Foot-head Homology Estimation Output

13. Background : Camera Model

14. Homography matrix 11 DOF One pair(2D-3D) of points 2 equation

15. Simple Calibration Example Measure 3-D position of special object points in 3-D scene (0,0,0) Correspond to camera 2-D point (0,30,0) (30,30,40) (u1,v1) x y z (u2,v2)

16. Foot-head Homology Estimation 1. Camera model : Shifted Homographies 2. Focal length, Rotation, Translation 3. Quadratic Eigenvalue Problem(QEP) 4. Foot-head Homology

17. Camera model Extrinsic parameters rotation R and translation t

18. Camera Parameters Assumptions intrinsic parameters – Square pixels – No principal point offset : Image coordinate at center point (principal point) – No skew : angle of horizon axis and vertical axis = 90’ Intrinsic parameters K = |f 0 0| |0 f 0| |0 0 1| x y 90’

19. x y z (x1,y1,0) (x1,y1,z0) (x2,y2,0) (x2,y2,z0) (x3,y3,0) (x3,y3,z0) If x1,x2,x3,y1,y2,y3 are known Six points => 12 equation Compute homography of H

20. x y z (x1,y1,0) (x1,y1,z0) (x2,y2,0) (x2,y2,z0) (x3,y3,0) (x3,y3,z0) If x1,x2,x3,y1,y2,y3 are unknown How to find homography of H?

21. x y z (0,0,0) (0,0,z0) (x2,y2,0) (x2,y2,z0) (x3,y3,0) (x3,y3,z0) (0,0,0) & (0,0,z0) two point are known 4 equation

22. x1 y1 z x y z (0,0,0) (0,0,z0) (0,0,0) (0,0,z0) (x3,y3,0) (x3,y3,z0) x1 = x+dx1 y1 = y+dy1

23. x2 y2 z x1 y1 z x y z (0,0,0) (0,0,z0) (0,0,0) (0,0,z0) (0,0,0) (0,0,z0) x1 = x+dx1 y1 = y+dy1 x2 = x+dx2 y2 = y+dy2

24. Shifted Homographies 3+3K unknown, Dof = 3+3K-1 unknown add equation K = 1 6 4 K = 2 9 8 K = 3 12 12

25. Shifted Homographies The 3D point X = (x, y, z,1) can be simplified assuming x = 0 r i : is the i-th column of R 6+3K unknowns, K : number of detections

26. Shifted Homographies

27. Finding Homographies This equation is extended with all the known point correspondences to form this equation: M contains all the point correspondences h contains h1, h2 and the unknown h3 of the homographies h is fixed for standard camera calibration

28. Focal Length 、 Rotation and Translation

29. form 3 equations Where equation unknown K = 1 3 4 K = 2 6 6

30. Minimum Solution : two detectors case The equations in (7) give Six equation with six unknown

31. Overdetermined Solution More than two homologies : solvable as a Quadratic Eigenvalue Problem (QEP) Find scalars λ and nonzero vectors x, satisfying (λ 2 D3 + λD2 + D1)x = 0 The authors create D1, D2, D3 using the known values in (7), λ = f.

32. Overdetermined Solution Solve With D1, D2, D3 very sparse containing only:

33. Solving QEP One approach to solving the QEP : Convert it to a linear system (remove the f 2 ): Solving ( A - f B ) v = 0

34. Foot-head Homology Result of QEP : K, R, t, f From this construct the homology H FH with u H ≃ H FH *u F – u H : image points of head – u F :image points of feet (x 0 k,y 0 k,0) (x 0 k,y 0 k,l) HkHk HkHk uFuF uHuH H FH Camera Image 3-D points

35. Result

36. Conclusion Use 3D-2D point correspondences (model to contour) Encode camera parameters that define relation between 3D 2D as a matrix H Solve H and get the camera parameters