1. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Roadmap
Introduction to object detection
What kind of object do we want to detect?
Circle/ellipse detection
Least-Square fitting based detector
Line detection
Hough transform-based detector
Corner detection
Harris corner detector
Face/pedestrian/vehicle detection*
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

2. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Beyond Edge Detection
Edges are among the earliest primitives that have been studied in computer vision, but their definition remains fuzzy
There are many other primitives that can be more rigorously defined or at least conceptually easier to define
Corner, line, circle, ellipse, square, triangle, …
The ultimate vision tasks involve the detection of general objects
Face, pedestrian, vehicle, mouth, eyes, door
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

3. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Two Paradigms
Model-based approaches
Build an explicit model to characterize the objects we want to detect
Suitable for the class of simple objects for which good models are relatively easy to find
Examples: corner/line/circle detection
Data-driven (machine learning) approaches
Provide a training set (e.g., manually detected results) and detector works like a black box
Suitable for the class of complex objects for which good models are NOT easy to find
Examples: neural network, support vector machine
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

4. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Roadmap
Introduction to object detection
What kind of object do we want to detect?
Circle/ellipse detection
Least-Square fitting based detector
Line detection
Hough transform-based detector
Corner detection
Harris corner detector
Face/pedestrian/vehicle detection*
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

5. Recall: Edge Detection in Iris Image
>I=imread(‘iris.bmp’);
>c=edge(I,’canny’);
>[x,y]=find(c==1);
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

6. Line/Polynomial Fittingx x
y=ax+b
y=ax2+bx+c
MATLAB function: P = POLYFIT(X,Y,N)
Polynomial of degree N
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

7. Circle/Ellipse Fittingb r a
(xo,yo)
(xo,yo)
Min E=(x-xo)2+(y-y0)2-r2
Min E=(x-xo)2/a2+(y-y0)2/b2-1
MATLAB function: circle_detect.m
MATLAB function: fitellipse.m
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

8. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Image Examples
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

9. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Roadmap
Introduction to object detection
What kind of object do we want to detect?
Circle/ellipse detection
Least-Square fitting based detector
Line detection
Hough transform-based detector
Corner detection
Harris corner detector
Face/pedestrian/vehicle detection*
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

10. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Basic Idea Behind  y ?   x 
Question: Can we find a transform that maps a line to a point?
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

11. Radon/Hough Transform: Mathematics*
min max
min 
Sample of
two lines
 (x1y1) 
max
Two of an infinite number
of lines through point (x1,y1)
- Space
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

12. Radon/Hough Transform: Image Examples
Conclusion: Line detection can be implemented by point (peak)
detection in the Radom/Hough transform domain
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

13. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
MATLAB Functions
>> [H,theta,rho]=hough(f);
>> figure, imshow(theta,rho,H, [ ], 'notruesize')
>> axis on, axis normal;
>> xlabel('\theta'),ylabel('\rho'),
>> colorbar
>> colormap(jet)
>> [H,theta,rho]=hough(f);
>> figure, imshow(theta,rho,H, [ ], 'notruesize')
>> axis on, axis normal;
>> xlabel('\theta'),ylabel('\rho'),
>> colorbar
>> colormap(jet)
>>[r,c]=houghpeaks(H,5);
>> hold on
>> plot(theta(c), rho(r), 'linestyle', 'none','marker' , ...
's', 'color', 'w')
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

14. Line Detection from Real-world Images
Gray Scale Image Edge Image (Canny)
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

15. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Peak Finding
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

16. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Roadmap
Introduction to object detection
What kind of object do we want to detect?
Circle/ellipse detection
Least-Square fitting based detector
Line detection
Hough transform-based detector
Corner detection
Harris corner detector
Face/pedestrian/vehicle detection*
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

17. An introductory example:
Harris corner detector
C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

18. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
The Basic Idea
We should easily recognize the point by looking through a small window
Shifting a window in any direction should give a large change in intensity
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

19. Harris Detector: Basic Idea
“flat” region: no change in all directions
“edge”: no change along the edge direction
“corner”: significant change in all directions
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

20. Harris Detector: Mathematics
Change of intensity for the shift [u,v]:
Intensity
Window function
Shifted intensity or
Window function w(x,y) =
Gaussian
1 in window, 0 outside
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

21. Harris Detector: Mathematics*
For small shifts [u,v] we have a bilinear approximation:
where M is a 22 matrix computed from image derivatives:
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

22. Harris Detector: Eigenvalue Analysis
Intensity change in shifting window: eigenvalue analysis
1, 2 – eigenvalues of M
direction of the fastest change
direction of the slowest change
Ellipse E(u,v) = const
(max)-1/2
(min)-1/2
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

23. Eigenvalues Carry Clues2
Classification of image points using eigenvalues of M:
“Edge” 2 >> 1
“Corner” 1 and 2 are large, 1 ~ 2; E increases in all directions
1 and 2 are small; E is almost constant in all directions
“Edge” 1 >> 2
“Flat” region 1
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

24. Corner Response Measure
Measure of corner response:
(k – empirical constant, k = )
Note that other definition of corner response measure also exists
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

25. Harris Detector: Mathematics2
“Edge”
“Corner”
R depends only on eigenvalues of M
R is large for a corner
R is negative with large magnitude for an edge
|R| is small for a flat region
R < 0
R > 0
“Flat”
“Edge”
|R| small
R < 0 1
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

26. Harris Detector: Workflow
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

27. Harris Detector: Workflow
Compute corner response R
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

28. Harris Detector: Workflow
Find points with large corner response: R>threshold
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

29. Harris Detector: Workflow
Take only the points of local maxima of R
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

30. Harris Detector: Final Results
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

31. EE465: Introduction to Digital Image Processing Copyright Xin Li'2009
Roadmap
Introduction to object detection
What kind of object do we want to detect?
Circle/ellipse detection
Least-Square fitting based detector
Line detection
Hough transform-based detector
Corner detection
Harris corner detector
Face/pedestrian/vehicle detection*
EE465: Introduction to Digital Image Processing Copyright Xin Li'2009

32. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Face Detection
Face detection problem: do you see any human faces in an
image? (trivial for human eyes but difficult for computers)
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

33. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Challenges with FD
pose variation
lighting condition variation
facial expression variation
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

34. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Eigenfaces
Training data
Eigenface images ~
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

35. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Neural Network based FD
Cited from “Neural network based face detection”,
by Henry A. Rowley, Ph.D. thesis, CMU, May 1999
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

36. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
FD Examples
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

37. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Pedestrian Detection
Why do we need to detect pedestrians?
Security monitoring
Intelligent vehicles
Video database search
Challenges
Uncertainty with pedestrian profile, viewing distance and angle, deformation of human limb
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

38. Learning-based Pedestrian Detection
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

39. Thermal (Infrared) Imaging
Visible-spectrum image
Infrared image
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

40. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Image Examples
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

41. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Vehicle Detection
Intelligent vehicles aim at improving the driving safety by machine vision techniques
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

42. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003
Summary
Object detection remains one of the grand challenges in computer vision
The most intriguing question lies in how to define object (or not-an-object)
This is the task that human vision system (HVS) is good at; and the currently most effective machine vision is based on supervised learning principles.
EE465: Introduction to Digital Image Processing Copyright Xin Li'2003