1. Lecture 12: Image alignment CS4670/5760: Computer Vision Kavita Bala http://www.wired.com/gadgetlab/2010/07/camera-software-lets-you-see-into-the-past/

2. Announcements Project 1 Artifact voting closes tomorrow. Please vote. Project 2 released – Due Monday, March 9 2015 (by 11:59pm) Prelim next Thu – Send me mail if you have a conflict – All material till end of this week – Closed book – Will release last year’s midterm. But it was a take home.

3. PA2: Feature detection and matching

4. Reading Appendix A.2, 6.1

5. Fitting: find the parameters of a model that best fit the data Alignment: find the parameters of the transformation that best align matched points Common problems in vision

6. Alignment Alignment: find parameters of model that maps one set of points to another Typically want to solve for a global transformation that accounts for *most* true correspondences Difficulties – Noise (typically 1-3 pixels) – Outliers (often 50%)

7. Computing transformations ?

8. Simple case: translations How do we solve for ?

9. Mean displacement = Simple case: translations Displacement of match i =

10. Another view System of linear equations – What are the knowns? Unknowns? – How many unknowns? How many equations (per match)?

11. Another view Problem: more equations than unknowns – “Overdetermined” system of equations – We will find the least squares solution

12. Least squares formulation For each point we define the residuals as

13. Least squares formulation Goal: minimize sum of squared residuals “Least squares” solution For translations, is equal to mean displacement

14. Least squares formulation Can also write as a matrix equation 2n x 22 x 12n x 1

15. Least squares Find t that minimizes

16. Least squares: find t to minimize To solve, form the normal equations Differentiate and equate to 0 to minimize

17. Affine transformations How many unknowns? How many equations per match? How many matches do we need?

18. Affine transformations Residuals: Cost function:

19. Affine transformations Matrix form 2n x 6 6 x 1 2n x 1

20. Homographies To unwarp (rectify) an image solve for homography H given p and p’ solve equations of the form: p’ = Hp p p’

21. Alternate formulation for homographies where the length of the vector [h 00 h 01 … h 22 ] is 1

22. Solving for homographies Linear or non-linear?

23. Solving for homographies

25. Defines a least squares problem: 2n × 9 9 2n Since is only defined up to scale, solve for unit vector

26. Homographies To unwarp (rectify) an image solve for homography H given p and p’ solve equations of the form: p’ = Hp –linear in unknowns: coefficients of H –H is defined up to an arbitrary scale factor –how many points are necessary to solve for H? p p’

27. Solving for homographies Defines a least squares problem: Since is only defined up to scale, solve for unit vector Solution: = eigenvector of with smallest eigenvalue Works with 4 or more points 2n × 9 9 2n

28. Recap: Two Common Optimization Problems Problem statementSolution Problem statementSolution (matlab)