1. Lecture 14: Feature matching and Transforms CS4670/5670: Computer Vision Kavita Bala

2. Announcements PA 2 out Artifact voting out: please vote HW 1 out Check piazza first before you post questions

3. Evaluating the results How can we measure the performance of a feature matcher? 50 75 200 feature distance

4. True/false positives The distance threshold affects performance – True positives = # of detected matches that are correct Suppose we want to maximize these—how to choose threshold? – False negatives = # of undetected matches Suppose we want to minimize these—how to choose threshold? 50 75 200 false match true match feature distance How can we measure the performance of a feature matcher?

5. "Precisionrecall" by Walber - Own work. Licensed under CC BY-SA 4.0 via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Precisionrecall.svg#mediaviewer/File:Precis ionrecall.svg

6. Confusion Matrix https://uberpython.wordpress.com/2012/01/01/precision-recall-sensitivity-and-specificity/

8. Precision vs. Recall Examples 1000 animals, 100 dogs Algorithm finds 50 (of which 40 are dogs, 10 are cats) Precision = Recall = Algorithm finds 10 (of which 10 are dogs) Precision = Recall = Algorithm returns 1000 (of which 100 are dogs) Precision = Recall =

9. https://uberpython.wordpress.com/2012/01/01/precision-recall-sensitivity-and-specificity/

10. 0.7 Evaluating the results 0 1 1 false positive rate # true positives # matching features (positives) 0.1 How can we measure the performance of a feature matcher? “recall” # false positives # unmatched features (negatives)

11. 0.7 Evaluating the results 0 1 1 false positive rate # true positives # matching features (positives) 0.1 # false positives # unmatched features (negatives) ROC curve (“Receiver Operator Characteristic”) How can we measure the performance of a feature matcher? “recall” AUC (area under curve)

12. Tradeoff Precision vs. recall F-measure:

13. More on feature detection/description

14. Lots of applications Features are used for: – Image alignment (e.g., mosaics) – 3D reconstruction – Motion tracking – Object recognition – Indexing and database retrieval – Robot navigation – … other

15. 3D Reconstruction Internet Photos (“Colosseum”) Reconstructed 3D cameras and points

16. Object recognition (David Lowe)

17. Sony Aibo SIFT usage: Recognize charging station Communicate with visual cards Teach object recognition

18. Available at a web site near you… For most local feature detectors, executables are available online: – http://www.robots.ox.ac.uk/~vgg/research/affine – http://www.cs.ubc.ca/~lowe/keypoints/ – http://www.vision.ee.ethz.ch/~surf K. Grauman, B. Leibe

19. CS4670/5760: Computer Vision Kavita Bala Transforms

20. Reading Szeliski: Chapter 6.1, 3.6

21. Image alignment

22. What is the geometric relationship between these two images? ? Answer: Similarity transformation (translation, rotation, uniform scale)

23. What is the geometric relationship between these two images? ?

24. Very important for creating mosaics!

25. Richard SzeliskiImage Stitching25 Image Warping image filtering: change range of image g(x) = h(f(x)) image warping: change domain of image g(x) = f(h(x)) f x h g x f x h g x

26. Richard SzeliskiImage Stitching26 Image Warping image filtering: change range of image g(x) = h(f(x)) image warping: change domain of image g(x) = f(h(x)) hh f f g g

27. Richard SzeliskiImage Stitching27 Parametric (global) warping Examples of parametric warps: translation rotation aspect

28. Parametric (global) warping Transformation T is a coordinate-changing machine: p’ = T(p) What does it mean that T is global? – Is the same for any point p – can be described by just a few numbers (parameters) Let’s consider linear xforms (can be represented by a 2D matrix): T p = (x,y)p’ = (x’,y’)

29. Common linear transformations Uniform scaling by s: (0,0) What is the inverse?

30. Common linear transformations Rotation by angle θ (about the origin) (0,0) What is the inverse? For rotations: θ

31. Cornell CS4620 Fall 2015 Lecture 8 Linear transformation gallery Rotation 31 © 2015 Kavita Bala w/ prior instructor Steve Marschner

32. Cornell CS4620 Fall 2015 Lecture 8 Linear transformation gallery Shear 32 © 2015 Kavita Bala w/ prior instructor Steve Marschner

33. 2x2 Matrices What types of transformations can be represented with a 2x2 matrix? 2D mirror about Y axis? 2D mirror across line y = x?

34. What types of transformations can be represented with a 2x2 matrix? 2D Translation? Translation is not a linear operation on 2D coordinates NO! 2x2 Matrices

35. Cornell CS4620 Fall 2015 Lecture 8 Composing transformations Linear transformations straightforward – Transforming first by M T then by M S is the same as transforming by M S M T 35 © 2015 Kavita Bala w/ prior instructor Steve Marschner

36. All 2D Linear Transformations Linear transformations are combinations of … – Scale, – Rotation, – Shear, and – Mirror Properties of linear transformations: – Origin maps to origin – Lines map to lines – Parallel lines remain parallel – Ratios are preserved – Closed under composition

37. What types of transformations can be represented with a 2x2 matrix? 2D Translation? Translation is not a linear operation on 2D coordinates NO! 2x2 Matrices

38. Cornell CS4620 Fall 2015 Lecture 8 Homogeneous coordinates A trick for representing translation elegantly Extra component w for vectors, extra row/column for matrices – for affine, always keep w = 1 Represent linear transformations with dummy extra row and column 38 © 2015 Kavita Bala w/ prior instructor Steve Marschner

39. Translation Represent translation using extra column

40. Affine transformations any transformation with last row [ 0 0 1 ] we call an affine transformation

41. Basic affine transformations Translate 2D in-plane rotationShear Scale

42. Affine Transformations Affine transformations are combinations of … – Linear transformations, and – Translations Properties of affine transformations: – Origin does not necessarily map to origin – Lines map to lines – Parallel lines remain parallel – Ratios are preserved – Closed under composition

43. Euclidean: translation, rotation, reflection Similarity: translation, rotation, uniform scale, reflection Affine: linear transformations + translation

44. 2D image transformations These transformations are a nested set of groups Closed under composition and inverse is a member