1. CV: Matching in 2D
Matching 2D images to 2D images; Matching 2D images to 2D maps or 2D models; Matching 2D maps to 2D maps
MSU CSE 803 Stockman

2. 2D Matching Problem 1) Need to match images to maps or models
2) need to match images to images
Applications
1) land use inventory matches images to maps
2) object recognition matches images to models
3) comparing X-rays before and after surgery
MSU CSE 803 Stockman

3. Methods for study Recognition by alignment Pose clustering
Geometric hashing
Local focus feature
Relational matching
Interpretation tree
Discrete relaxation
MSU CSE 803 Stockman

4. Tools and methods Algebra of affine transformations
scaling, rotation, translation, shear
Least-squares fitting
Nonlinear warping
General algorithms
graph-matching, pose clustering,
discrete relaxation, interpretation tree
MSU CSE 803 Stockman

5. Alignment or registration
DEF: Image registration is the process by which points of two images from similar viewpoints of essentially the same scene are geometrically transformed so that corresponding features of the two images have the same coordinates
MSU CSE 803 Stockman

6. y v u x
MSU CSE 803 Stockman

7. 11 matching control points x, y, u, v below
= T [ u, v, 1] = T [x, y, 1] t
MSU CSE 803 Stockman

8. Least Squares in MATLAB
BigPic =
LilPic =
[ a11 a21
a12 a22 =
a13 a23 ]
MSU CSE 803 Stockman

9. Least squares in MATLAB 2
>> ERROR = LilPic - BigPic * AFFINE
ERROR =
>> AFFINE = BigPic \ LilPic
AFFINE =
Worst is 1.8 pixels
The solution is such that the 11D vector at the right has the smallest L2 norm
MSU CSE 803 Stockman

10. Least squares in MATLAB 3
X =
>> Y
Y =
>> T = X \ Y
T =
>> E = Y - X*T
E =
Solution from 4 points has smaller error on those points
MSU CSE 803 Stockman

11. Least squares in MATLAB 4
>> E2 = LilPic - BigPic * T
E2 =
When the affine
transformation obtained
from 4 matching points is
applied to all 11 points,
the error is much worse
than when the
transformation was
obtained from those 11
points.
MSU CSE 803 Stockman

12. Components of transformations
Scaling, rotation,
translation, shear
MSU CSE 803 Stockman

13. scaling transformation
MSU CSE 803 Stockman

14. Rotation transformation
MSU CSE 803 Stockman

15. Pure rotation
MSU CSE 803 Stockman

16. Orthogonal tranformations
DEF: set of vectors is orthogonal if all pairs are perpendicular
DEF: set of vectors is orthonormal if it is orthogonal and all vectors have unit length
Orthogonal transformations preserve angles
Orthonormal transformations preserve angles and distances
Rotations and translations are orthonormal
DEF: a rigid transformation is combined rotation and translation
MSU CSE 803 Stockman

17. Translation requires homogeneous coordinates
MSU CSE 803 Stockman

18. Rotation, scaling, translation
MSU CSE 803 Stockman

19. Model of shear
MSU CSE 803 Stockman

20. General affine transformation
MSU CSE 803 Stockman

21. Solving for an RST using control points
MSU CSE 803 Stockman

22. Extracting a subimage by subsampling
MSU CSE 803 Stockman

23. Subsampling transformation
MSU CSE 803 Stockman

24. subsampling
At MSU, even the pigs are smart.
MSU CSE 803 Stockman

25. recognition by alignment
Automatically match some salient points
Derive a transformation based on the matching points
Verify or refute the match using other feature points
If verified, then registration is done, else try another set of matching points
MSU CSE 803 Stockman

26. Recognition by alignment
MSU CSE 803 Stockman

27. Feature points and distances
MSU CSE 803 Stockman

28. Image features pts and distances
MSU CSE 803 Stockman

29. Point matches reflect distances
MSU CSE 803 Stockman

30. Once matching bases fixed
can find any other feature point in terms of the matching transformation
can go back into image to explore for the holes that were missed (C and D)
can determine grip points for a pick and place robot ( transform R and Q into the image coordinates)
MSU CSE 803 Stockman

31. Compute transformation
Once we have matching control points
(H2, A) and (H3, B) we can compute a rigid transform
MSU CSE 803 Stockman

32. Get rotation easily, then translation
MSU CSE 803 Stockman

33. Generalized Hough transform
Cluster evidence in transform parameter space
MSU CSE 803 Stockman

34. See plastic slides matching maps to images
MSU CSE 803 Stockman

35. “Best” affine transformation from overdetermined matches
MSU CSE 803 Stockman

36. “Best” affine transformaiton
Use as many matching pairs ((x,y)(u,v)) as possible
MSU CSE 803 Stockman

37. result for previous town match
MSU CSE 803 Stockman