1. Lecture 5 Template matching
Slides by:
Clark F. Olson

2. Comparing images
The simplest form of image comparison is a pixel-by-pixel comparison of some rectangular region.
In template matching, all of a smaller image (the template) is compared to all possible subregions of a larger image.
This is a form of appearance-based matching.

3. Template matching
Can find objects by comparing pixels (and aggregating)
For example, sum of differences between corresponding pixels
Face recognition often uses a variation of this technique
In general, this requires an example image of the object you are looking for.
Sometimes called a template or exemplar
General strategy:
Consider all possible positions of template in search image
If match is close enough, save/output

4. Comparing pixels How should the images be compared?
Sum of absolute differences
Sum of squared differences
Correlation (similar to convolution, but kernel is not flipped)
Unfortunately, these will fail if there is significant difference in:
Illumination
Sensor
Object color
Viewpoint
Etc.

5. Face finding Finding faces is an important special case
The basic idea is very similar to filtering:
Filters tend to have strong responses at locations that look similar to them.
Use a face filter!
Good implementations must be able to handle:
different lighting
different sizes
different orientations

6. Face finding
H. Rowley, S. Baluja, T. Kanade, “Neural network-based face detection”, IEEE Conf. on Computer Vision and Pattern Recognition. © 1996 IEEE.

7. H. Rowley, S. Baluja, T. Kanade, “Neural network-based face detection”, IEEE Conf. on Computer Vision and Pattern Recognition. © 1996 IEEE. 7

8. Recognition by finding patterns
If we know exactly what something looks like then it is easy to find.
Objects look different under varying conditions:
Changes in lighting or color
Changes in viewing direction
Changes in size / shape
A single exemplar is unlikely to succeed reliably!
However, it is impossible to represent all appearances of an object.

9. Appearance-based matching
Appearance-based techniques use example images (templates or exemplars) of the objects to perform recognition (as opposed to extracted features). I include edge images in this definition, although this is considered feature-based by some.

10. Example ? = ? =
Frontal faces are fairly easy to find (and sometimes classify)
However, changes to lighting and background cause problems.

11. Edge matching
Changes in lighting and color usually don’t have much effect on image edges.

12. Edge matching Strategy: Detect edges in template and image
Compare edge images to find the template
Must consider range of possible template positions

13. Edge matching measures
What measure should we use to compare edge images?
Can count number of overlapping edges.
Not robust to changes in shape
Better: count number of template edge pixels with some distance of an edge in the search image.
Best:
Determine probability distribution of distance to nearest edge in search image (if template at correct position)
Estimate likelihood of each template position generating image

14. Matching gradients
One way to be robust to illumination changes, but not throw away as much information is to compare image gradients.
Matching is performed like matching greyscale images.
Simple alternative: use (normalized) correlation.

15. Different viewpoints What if we don’t know the viewpoint?
Non-frontal faces
Objects in arbitrary orientation
A partial solution: linear transformations model small changes in viewpoint.
A better solution: use templates that model all possible view directions.
Computationally expensive

16. Large modelbases
If we have many potential templates that we are looking can we search efficiently?
One approach is based on eigenvectors of the templates.
eigenfaces

17. Robust measures
If we want to be insensitive to changes in illumination, sensor, and object color, we must use a robust measure.
One option is to change the image to make them comparable:
Gradients
Entropy
Original image
Local entropy

18. Different sensors
infrared image
CCD image