1. Video Compass Jana Kosecka and Wei Zhang George Mason University

2. Motivation Information encoded in vanishing points and lines
Visually Guided Agents - Navigation
3D Reconstruction
Camera pose recovery
Man made environments
Many structural regularities
Parallel and orthogonal lines and planes
Reveal a lot about the scene structure
And camera pose
Information encoded in vanishing points and lines

3. Previous Approaches Vanishing point recovery approaches vary in
Line detection and representation
Grouping of line segments to vanishing directions
Choice of objective function and optimization technique
Assumptions about the camera
Antone’00, Collins’90,Caprile’90,Kanatani’93,
McLean’97,Tuytelaars’98
Our approach
Line represented in projective space
Grouping and VP estimation done simultaneously using EM
Efficient initialization using orientation histograms
Partial camera calibration

4. Line detection Stage Edge detection, non-maximum suppression
(traditionally Hough Transform – issues of resolution, threshold
selection and search for peaks in Hough space)
Connected components on edge pixels with similar orientation
- group pixels with common orientation
Line fitting Lines determined from eigenvalues and eigenvectors of A
Candidate line segments - associated line quality

5. Pinhole Camera Imaging Model
Perspective Projection:
Image points
Image lines

6. vp Vanishing point – intersecting point of two parallel lines
Given a set of line segments belonging
to the same vanishing direction vanishing
point can be estimated
LLS on Gaussian Sphere [Teller 00, Collins 97]
For calibrated camera
Given a set of line segments (parallel in 3D), belonging
to the same vanishing direction, vanishing point
can be estimated as LLS problem on a Gaussian sphere

7. Grouping and Estimation
Basic Premise - In man made environments majority of parallel lines
is aligned with the principal axis of the world
reference frame.
Given a set of line segments automatically group the line segments
into three dominant vanishing directions and estimate the associated
vanishing points

8. Probabilistic inference with an unknown model
Estimate simultaneously the coordinates of vanishing
points as well as probabilities of individual line segments
belonging to particular vanishing direction [Antone’00]
Bayes rule - Vanishing point posterior – line likelihood
Conditional mixture model
Number of models – one for each dominant direction and one for
outlier process

9. EM algorithm Objective function – estimate VP by maximizing posterior
Probability of the vanishing points given a set of line segments
where
yields:

10. Uncalibrated Camera Image points Image lines
Perspective Projection:
Image points
Image lines
Augmented minimization problem is not well conditioned
Line segments are not well separated on the Gaussian Sphere

11. Initialization All the measurements need to be normalized
Use approximate camera intrinsic parameter matrix
Normalize all the coordinates by
Compute initial groups of lines
Initialization
Compute initial vanishing directions

12. Initial groups are determined from the peaks of the line orientation
histogram
Find zero crossings of the histogram curvature
Line orientation –Pi, Pi
Line orientation –Pi, Pi

13. Compute initial estimates of vanishing directions
E-step for each line estimate likelihood
M-step estimate so as to maximize complete log-likelihood
k=1,…,m
Iterate until you reach the equilibrium
For finite vanishing points refine final estimates using
nonlinear ML objective function [Hartley& Zisserman,00]

14. EM Iterations
number of models is adjusted during the iteration

17. Applications: Single View Analysis
uncalibrated single view geometry
exploit information about vanishing points
for partial calibration and reconstruction

18. Vanishing points constraints
three vanishing points in the image plane
orthogonal
zero skew
two vanishing points in the image plane – recover focal length
linearly

19. Applications: Matching and Mosaic Construction

20. Efficient Technique for Estimation of Vanishing points
Conclusions
Efficient Technique for Estimation of Vanishing points
two passes over image and 3-7 iterations of EM on average
Constraints of man-made environment were used in
different stages of image processing pipeline
Ongoing work
Pose recovery – recovery of relative orientation
Partial camera calibration
Reconstruction from a single view
Image matching