EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Roadmap Introduction to object detection What kind of object do we want to detect? Circle/ellipse detection Least-Square fitting based detector Line detection Hough transform-based detector Corner detection Harris corner detector Face/pedestrian/vehicle detection* EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Beyond Edge Detection Edges are among the earliest primitives that have been studied in computer vision, but their definition remains fuzzy There are many other primitives that can be more rigorously defined or at least conceptually easier to define Corner, line, circle, ellipse, square, triangle, … The ultimate vision tasks involve the detection of general objects Face, pedestrian, vehicle, mouth, eyes, door EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Two Paradigms Model-based approaches Build an explicit model to characterize the objects we want to detect Suitable for the class of simple objects for which good models are relatively easy to find Examples: corner/line/circle detection Data-driven (machine learning) approaches Provide a training set (e.g., manually detected results) and detector works like a black box Suitable for the class of complex objects for which good models are NOT easy to find Examples: neural network, support vector machine EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Roadmap Introduction to object detection What kind of object do we want to detect? Circle/ellipse detection Least-Square fitting based detector Line detection Hough transform-based detector Corner detection Harris corner detector Face/pedestrian/vehicle detection* EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Recall: Edge Detection in Iris Image >I=imread(‘iris.bmp’); >c=edge(I,’canny’); >[x,y]=find(c==1); EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Line/Polynomial Fitting x x y=ax+b y=ax2+bx+c MATLAB function: P = POLYFIT(X,Y,N) Polynomial of degree N EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Circle/Ellipse Fitting b r a (xo,yo) (xo,yo) Min E=(x-xo)2+(y-y0)2-r2 Min E=(x-xo)2/a2+(y-y0)2/b2-1 MATLAB function: circle_detect.m MATLAB function: fitellipse.m EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Image Examples EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Roadmap Introduction to object detection What kind of object do we want to detect? Circle/ellipse detection Least-Square fitting based detector Line detection Hough transform-based detector Corner detection Harris corner detector Face/pedestrian/vehicle detection* EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Basic Idea Behind  y ?   x  Question: Can we find a transform that maps a line to a point? EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Radon/Hough Transform: Mathematics* min max min  Sample of two lines  (x1y1)  max Two of an infinite number of lines through point (x1,y1) - Space EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Radon/Hough Transform: Image Examples Conclusion: Line detection can be implemented by point (peak) detection in the Radom/Hough transform domain EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 MATLAB Functions >> [H,theta,rho]=hough(f); >> figure, imshow(theta,rho,H, [ ], 'notruesize') >> axis on, axis normal; >> xlabel('\theta'),ylabel('\rho'), >> colorbar >> colormap(jet) >> [H,theta,rho]=hough(f); >> figure, imshow(theta,rho,H, [ ], 'notruesize') >> axis on, axis normal; >> xlabel('\theta'),ylabel('\rho'), >> colorbar >> colormap(jet) >>[r,c]=houghpeaks(H,5); >> hold on >> plot(theta(c), rho(r), 'linestyle', 'none','marker' , ... 's', 'color', 'w') EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Line Detection from Real-world Images Gray Scale Image Edge Image (Canny) EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Peak Finding EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Roadmap Introduction to object detection What kind of object do we want to detect? Circle/ellipse detection Least-Square fitting based detector Line detection Hough transform-based detector Corner detection Harris corner detector Face/pedestrian/vehicle detection* EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

An introductory example: Harris corner detector C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988 EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 The Basic Idea We should easily recognize the point by looking through a small window Shifting a window in any direction should give a large change in intensity EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Basic Idea “flat” region: no change in all directions “edge”: no change along the edge direction “corner”: significant change in all directions EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Mathematics Change of intensity for the shift [u,v]: Intensity Window function Shifted intensity or Window function w(x,y) = Gaussian 1 in window, 0 outside EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Mathematics* For small shifts [u,v] we have a bilinear approximation: where M is a 22 matrix computed from image derivatives: EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Eigenvalue Analysis Intensity change in shifting window: eigenvalue analysis 1, 2 – eigenvalues of M direction of the fastest change direction of the slowest change Ellipse E(u,v) = const (max)-1/2 (min)-1/2 EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Eigenvalues Carry Clues 2 Classification of image points using eigenvalues of M: “Edge” 2 >> 1 “Corner” 1 and 2 are large, 1 ~ 2; E increases in all directions 1 and 2 are small; E is almost constant in all directions “Edge” 1 >> 2 “Flat” region 1 EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Corner Response Measure Measure of corner response: (k – empirical constant, k = 0.04-0.06) Note that other definition of corner response measure also exists EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Mathematics 2 “Edge” “Corner” R depends only on eigenvalues of M R is large for a corner R is negative with large magnitude for an edge |R| is small for a flat region R < 0 R > 0 “Flat” “Edge” |R| small R < 0 1 EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Workflow EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Workflow Compute corner response R EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Workflow Find points with large corner response: R>threshold EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Workflow Take only the points of local maxima of R EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Harris Detector: Final Results EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2009 Roadmap Introduction to object detection What kind of object do we want to detect? Circle/ellipse detection Least-Square fitting based detector Line detection Hough transform-based detector Corner detection Harris corner detector Face/pedestrian/vehicle detection* EE465: Introduction to Digital Image Processing Copyright Xin Li'2009

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Face Detection Face detection problem: do you see any human faces in an image? (trivial for human eyes but difficult for computers) EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Challenges with FD pose variation lighting condition variation facial expression variation EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Eigenfaces Training data Eigenface images ~ EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Neural Network based FD Cited from “Neural network based face detection”, by Henry A. Rowley, Ph.D. thesis, CMU, May 1999 EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 FD Examples EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Pedestrian Detection Why do we need to detect pedestrians? Security monitoring Intelligent vehicles Video database search Challenges Uncertainty with pedestrian profile, viewing distance and angle, deformation of human limb EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Learning-based Pedestrian Detection EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

Thermal (Infrared) Imaging Visible-spectrum image Infrared image EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Image Examples EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Vehicle Detection Intelligent vehicles aim at improving the driving safety by machine vision techniques http://www.mobileye.com/visionRange.shtml EE465: Introduction to Digital Image Processing Copyright Xin Li'2003

EE465: Introduction to Digital Image Processing Copyright Xin Li'2003 Summary Object detection remains one of the grand challenges in computer vision The most intriguing question lies in how to define object (or not-an-object) This is the task that human vision system (HVS) is good at; and the currently most effective machine vision is based on supervised learning principles. EE465: Introduction to Digital Image Processing Copyright Xin Li'2003