18 th August 2006 International Conference on Pattern Recognition 2006 Epipolar Geometry from Two Correspondences Michal Perďoch, Jiří Matas, Ondřej Chum Center for Machine Perception Czech Technical University, Prague

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague2/16 Problem formulation Problem: Wide-baseline stereo with severe viewpoint and scale change. Message of this paper: Two correspondences of Local Affine Frames suffice to estimate epipolar geometry.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague3/16 Local Affine Frame correspondences Local Affine Frames (LAFs), Matas ICPR2002: Affine-covariantly detected local coordinate systems. One matching LAF pair provides three point-to-point correspondences.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague4/16 The correspondence problem Wide-baseline stereo framework (Matas ICPR2002): 1.Detect regions. MSERs are used here. 2.Build Local Affine Frames, define Measurement Region. 3.Geometric and photometric normalisation of MRs. 4.Establish correspondences by nearest neighbour search. 5.Verify correspondences, estimate model using RANSAC.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague5/16 How to find correct model of EG in presence of outliers? RANSAC - widely used robust estimator proposed by Fishler and Bolles Hypothesise & verify search: RANSAC Algorithm

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague6/16 Average number of RANSAC samples k : Example (   = 0.01): Chum DAGM2003, has shown that the theoretical number of samples can be achieved by introducing Local Optimisation step. RANSAC Algorithm – probability of finding a better model – proportion of inliers – size of the sample Proportion of inliers m50%45%40%35%30%25%20%15% … LAFs 3 LAFs 7 points

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague7/16 LO-RANSAC Algorithm Proposed by Chum DAGM2003, used with 3LAFs and seven point-to-point correspondences. Input: set of data points T, confidence  . Output: model M * of EG with the largest support S *. Repeat Draw a random sample of minimal size m from data points. Compute model parameters M i and its support S i. If new maximum was detected (i.e., |S i | > |S j | for (j < i): Apply local optimisation. Store the best model with support S *. Until the probability of finding model better than S * falls under  0.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague8/16 Six Point EG estimator Stewénius CVPR2005 has shown that EG can be estimated from six points under following assumptions on the cameras: Unit aspect ratio, zero skew, principal point in the middle of the image. Unknown but the same focal length. We take Stewénius’ assumptions. Is it possible to estimate EG from two Local Affine Frame correspondences?

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague9/16 Planar Degeneracy Chum CVPR2005 has that planar degeneracy causes suboptimal EG estimates. Six points on a plane do not provide enough constraints for estimation of the fundamental matrix. Planar degeneracy test - take sample points and check if they lie on a plane. When a plane is detected do plane and parallax search: Local optimisation that takes into account plane found. Sample another pair not consistent with the plane and compute fundamental matrix

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague10/16 2LAF-LO-RANSAC Algorithm Input: set of data points T, confidence  , sample size m = 2. Output: model M * of EG with largest support S *. Repeat Draw a random sample of minimal size m from data points. Compute model parameters M i and its support S i. If new maximum has occurred (i.e. |S i | > |S j | for (j < i): If a degenerated sample configuration is detected: Perform plane-and-parallax search for EG, otherwise Apply local optimisation. Store the best model with support S *. Until the probability of finding model better than S * falls under  . Performed at most log(k) times.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague11/16 Experiments: Scenes Corner  = 0.26 The China Wall  = 0.28 Wash  = 0.23

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague12/16 Experiment 1: Efficiency What is the “efficiency” of six point estimator? Is the planar degeneracy test necessary? Comparison of 2LAF, 3LAF with and without planar degeneracy test and PTS7 algorithm against reference EG. Number of good EG models generated from 1000 all-inlier samples (quality is measured by number of inliers, good – at least 90%). MethodCornerChina WallWash 7PTS LAF LAF ND LAF LAF ND ND - without degeneracy test

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague13/16 Experiment 2: Performance Average number of inliers and samples measured in hundred runs of 7PTS, 3LAF, 2LAF-LO-RANSAC algorithms. Small drop (two to four percent) in average number of inliers on difficult and noisy scenes for both 3LAF and 2LAF algorithm. Note the significant speedup (measured by number of samples) of 2LAF algorithm. Method CornerChina WallWash #inl#sam#inl#sam#inl#sam 2LAF LAF PTS

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague14/16 Experiment 3: Assumptions violation The six point solver assumes equal focal length in both images. The focal length ratio on the scene Corner is 1:3 and off-plane rotation about 30 degrees. No performance drop was observed.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague15/16 Experiment 3: Assumptions violation Robustness to change of pixel aspect ratio was measured on scene China Wall (#inliers in 100 runs). Width of pixel scaled: 0.5, 1.0, 2.0 times from top to bottom.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague16/16Conclusions Contribution Novel wide-baseline stereo algorithm using two LAF correspondences was proposed. 6 point (= 2 LAFs) EG estimator can be used in LO-RANSAC framework with the planar degeneracy test. Performance was experimentally tested and compared to 7PTS and 3LAF algorithms in wide-baseline stereo setup. Future work Detect other degenerated configurations. Compare the number of verifications instead of the number of samples. Thank you for your attention.