1. CS 565 Computer Vision
Nazar Khan
Lecture 11

3. What is a Corner?
A corner may be interpreted as an intersection of two edges, or as a region where strong derivatives exist in more than one direction

4. Corner Detection
The gradient vector is sufficient to give the local structure information for an edge.
For corners, the local structure direction around the corner changes strongly.
This suggests that corner phenomenon requires looking in a neighbourhood.
We want to find the directions of largest/smallest grayvalue changes within a neighbourhood.
This requires integration of directional information over a neighbourhood.

5. Corner Detection Let ∇u be the gradient direction.
A vector n that is orthogonal to ∇u satisfies nT∇u=0.
Let us average the gradient ∇uN vectors in a certain neighbourhood N.
∑N ∇u
A vector n that is “most orthogonal” to the average gradient direction within a neighbourhood N
Is a minimiser of E(n)=∑N (nT∇u)2

6. Corner Detection
E(n) = ∑N (nT∇u)2 = ∑N nT∇u∇uTn = nT (∑N ∇u∇uT ) n = nT An where A = ∑N ∇u∇uT = [ux2 uxuy ; uxuy uy2].

7. Some Linear Algebra Let n be a unit vector.
nT An is projection of n onto An
This projection will be maximum when An has the same direction as n.
Vectors n for which An has the same direction are known as eigen-vectors of A.
An=λn where λ is the eigen-value corresponding to the eigen-vector n.
For the largest eigen-value of A, nT An = λ1.
For the smallest eigen-value of A, nT An = λ2.

8. Structure Tensor
The matrix of summations A = ∑N ∇u∇uT = [ ∑N ux2 ∑N uxuy ] [ ∑N uxuy ∑N uy2 ] is often replaced by the matrix of Gaussian averages Jρ = Kρ * (∇u∇uT) [ Kρ*ux2 Kρ*(uxuy) ] [ Kρ*(uxuy) Kρ*uy2 ] which is the so-called structure tensor. Captures changes in local structure around a point.

9. Structure Tensor The structure tensor Jρ is
Symmetric
Positive definite
Its orthonormal eigenvectors v1, v2 specify the directions of largest/smallest contrast within some integration scale ρ.
The corresponding eigenvalues λ1, λ2 describe the average contrast along these directions.
Let λ1>= λ2. The eigenvalues allow a useful analysis of the local image structure:
constant areas: λ1 = λ2 = 0
straight edges: λ1 >> λ2 = 0
corners: λ1 >= λ2 >> 0
measure of anisotropy: (λ1 - λ2)2

24. Harris Corner Detector
Approximation

32. Corner Detection – A Second Look
A corner is different from its surroundings.
A patch placed on a corner pixel is different from its surrounding patches.
Patch is similar to patches in all directions
Patch is similar to patches in vertical direction
Patch is similar to patches in horizontal direction
Patch is NOT similar to patches in all directions

33. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

34. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

35. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

36. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

37. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

38. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

39. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

40. Demo of a point + with well distinguished neighbourhood.-
> 0
Demo of a point + with well distinguished neighbourhood.

41. Patch Similarity
Similarity of two patches p and q can be computed via the sum-of-squared-differences (SSD).
SSD(p,q)=∑(pij-qij)2

42. Moravec Corner Detector
Among the early corner detection approaches.
Basic Idea: A corner should have low self-similarity.
Self-similarity: how similar a patch is to neighbouring, overlapping patches.
Corner Strength: min(SSD(p,qN ))
If p is different from its neighbours in all directions, then this min value will be large.
If corner strength at a pixel is locally maximal, then this pixel is a corner.

43. Moravec Corner Detector
Weakness: If an edge is not aligned with the directions we are looking in, then this edge will be falsely detected as a corner.
Directions that we look in
Patch is NOT similar to patches in all directions that we look in. So it will be detected as a corner instead of an edge pixel.

44. Harris Corner Detector
Solution:
Do not fix the directions that we are looking in.
Find the direction in which patches become most dissimilar.
That is, the direction that maximises the SSD from current patch.

45. Harris Corner Detector
H.W.: Prove that these two expressions are equivalent.

46. Harris Corner Detector

47. Harris Corner Detector

48. Harris Corner Detector
What is the difference between d* and gradient direction u?
Gradient u is for a pixel.
d* is for a patch. d*

49. Harris Corner Detector
For an edge, the direction orthogonal to d* will be the direction of least change in our weighted SSD.
From S(d)=dTAd=, the weighted SSD in direction d* is given by the corresponding eigenvalue.
Since the structure tensor A is symmetric, positive definite, its eigenvectors will be orthonormal.
So the direction of least change in weighted SSD is given by the 2nd eigenvector of A.
the weighted SSD in this direction is given by the corresponding eigenvalue.
If this eigenvalue is large, then we have a non-edge/corner.

50. Harris Corner Detector
If the smaller eigenvalue is large, then we have a non-edge/corner.
largesmall0  flat region
large>>small0  edge
large and small >> 0  corner

51. Harris Corner Detector
Author: Nazar Khan (2014)

52. Harris Corner DetectorV1 V2
Corner
Author: Nazar Khan (2014)

53. Harris Corner DetectorV1 V2
Corner
Author: Nazar Khan (2014)

54. Corner Detection
H.W: Write a detailed derivation/explanation of the Harris corner detector starting from the Moravec corner detector.
H.W: Implement a simple Harris corner detector using the rule small>T and test it on some sample real-world image.
Experiment with other corner measures:
Haris & Stpehens
Shi-Tomasi
Nobel