1. Example: line fitting

2. n=2

3. Model fitting

4. Measure distances

5. Count inliers c=3

6. Another trial c=3

7. The best model c=15

8. RANSAC

9. RANSAC for estimating homography RANSAC loop: 1.Select four feature pairs (at random) 2.Compute homography H (exact) 3.Compute inliers where SSD(p i ’, H p i )< ε 4.Record the largest set of inliers so far 5.Re-compute least-squares H estimate on the largest set of the inliers

10. RANSAC in general RANSAC = Random Sample Consensus an algorithm for robust fitting of models in the presence of many data outliers Compare to robust statistics Given N data points x i, assume that mjority of them are generated from a model with parameters , try to recover .

11. RANSAC algorithm Run k times: (1) draw n samples randomly (2) fit parameters  with these n samples (3) for each of other N-n points, calculate its distance to the fitted model, count the number of inlier points, c Output  with the largest c How many times? How big? Smaller is better How to define? Depends on the problem.

12. How to determine k n: number of samples drawn each iteration p : probability of real inliers P : probability of at least 1 success after k trials n samples are all inliers a failure failure after k trials npk 30.535 60.697 60.5293 for P=0.99

13. Applications

14. Automatic image stitching

19. Correspondence Results Chum & Matas 2005

20. Object Recognition Results Brown & Lowe 2005

21. Object Recognition Results Nister & Stewenius 2006

22. Object Classification Results Grauman & Darrell 2006, Dorko & Schmid 2004

23. Geometry Estimation Results Snavely, Seitz, & Szeliski 2006

24. Object Tracking Results Gordon & Lowe 2005

25. Robotics: Sony Aibo SIFT is used for  Recognizing charging station  Communicating with visual cards  Teaching object recognition  soccer

26. Image Alignment Feature Detection and Matching Cylinder: Translation 2 DoF Plane: Homography 8 DoF

27. Plane perspective mosaics –8-parameter generalization of affine motion works for pure rotation or planar surfaces –Limitations: local minima slow convergence

28. Revisit Homography x (X c,Y c,Z c ) xcxc f

29. Estimate f from H? x (X c,Y c,Z c ) xcxc f H {

30. The drifting problem Error accumulation –small errors accumulate over time

31. Bundle Adjustment Associate each image i with Each image i has features Trying to minimize total matching residuals

32. Rotations How do we represent rotation matrices? 1.Axis / angle (n,θ) R = I + sinθ [n]  + (1- cosθ) [n]  2 (Rodriguez Formula), with [n]  be the cross product matrix.

33. Incremental rotation update 1.Small angle approximation ΔR = I + sinθ [n]  + (1- cosθ) [n]  2 ≈ I +θ [n]  = I+[ω]  linear in ω= θn 2.Update original R matrix R ← R ΔR

34. Recognizing Panoramas [Brown & Lowe, ICCV’03]

35. Finding the panoramas

37. Algorithm overview

43. Finding the panoramas

45. Algorithm overview

48. Get you own copy! [Brown & Lowe, ICCV 2003] [Brown, Szeliski, Winder, CVPR’05]

49. How well does this work? Test on 100s of examples…

50. How well does this work? Test on 100s of examples… …still too many failures (5-10%) for consumer application

51. Matching Mistakes: False Positive

53. Matching Mistakes: False Negative Moving objects: large areas of disagreement

54. Matching Mistakes Accidental alignment –repeated / similar regions Failed alignments –moving objects / parallax –low overlap –“feature-less” regions No 100% reliable algorithm?

55. How can we fix these? Tune the feature detector Tune the feature matcher (cost metric) Tune the RANSAC stage (motion model) Tune the verification stage Use “higher-level” knowledge –e.g., typical camera motions Need a large training/test data set (panoramas)