1. Recognition using Nearest Neighbor (or kNN)
Modified from various sources including

2. Recall ---Object Recognition
Object detection and recognition is formulated as a classification problem.
The image is partitioned into a set of overlapping windows
… and a decision is taken at each window about if it contains a target object or not.
Decision boundary
Computer screen
Background
In some feature space
Where are the screens?
Bag of image patches
Object detection and recognition is formulated as a classification problem. The image is partitioned into a set of overlapping windows, and a decision is taken at each window about if it contains a target object or not.
Each window is represented by extracting a large number of features that encode information such as boundaries, textures, color, spatial structure.
The classification function, that maps an image window into a binary decision, is learnt using methods such as SVMs, boosting or neural networks.

3. Recognition Techniques…a few
Nearest neighbor
Neural networks
106 examples
LeCun, Bottou, Bengio, Haffner 1998
Rowley, Baluja, Kanade 1998 …
Shakhnarovich, Viola, Darrell 2003
Berg, Berg, Malik 2005 …
Support Vector Machines and Kernels
Conditional Random Fields
Many more references missing here. These are big fields on its own.
Big emphasis on the past on face detection and character recognition.
It provided a clear problem with a well defined visual category and many applications. It allowed making progress on efficient techniques.
Guyon, Vapnik
Heisele, Serre, Poggio, 2001 …
McCallum, Freitag, Pereira 2000
Kumar, Hebert 2003 …

4. Formulation Formulation: binary classification Classification function… x1 x2 x3 … xN
xN+1
xN+2 …
xN+M
Features x =
y = -1 +1 -1 -1
Labels ? ? ?
Training data: each image patch is labeled
as containing the object or background
Test data
In many cases, F(x) will give a real value and the classification is perform with a Threshold. If F(x)>0 then y =+1
Where belongs to some family of functions
Classification function
Minimize misclassification error
(Not that simple: we need some guarantees that there will be generalization)

5. Two ways to thing of “modeling an object” in its feature space
Parametric model
Here we fit one or more parametric equations to the feature space to model our “object”
Non-parametric model
We do not use an explicit parametric model, more sample oriented or statistical. z

6. Parametric models Subspace of monkeys Subspace of natural images
Nearest Neighbor IS NOT a Parametric model
Subspace of monkeys
Subspace of natural images
Space of all images
Parametric model of monkeys

7. Non-parametric Approach
!!! HIGH DIMENSIONAL !!!
!!! HIGH DIMENSIONAL !!!
Subspace of monkeys
Subspace of natural images z
Query image
Space of all images

8. Non-parametric Approach
!!! HIGH DIMENSIONAL !!!
!!! HIGH DIMENSIONAL !!!
Subspace of monkeys
Subspace of natural images
Query image
Space of all images

9. Non-parametric Classifier
Nearest-neighbors
For each query, obtain sibling set (neighbors)
3 different types of distance metric
Hand-designed, use whole image

10. Metric 1 - Dssd _ 2 Sum of squared differences (SSD)
To give invariance to illumination: Each image normalized to be zero mean, unit variance 2
Image 1
Image 2 _

11. Nearest Neighbors in 80 million images
105
Size of dataset
106
108

12. Differing density of images

13. Metric 2 - Dwarp _ 2 SSD but allow small transformations
Find min using gradient descent
Image 2
Translation:
Scalings:
Horizontal flip: _
Image 1
SSD
Transformations

14. Metric 3 - Dshift _ 2 As per Warping but also allow sub-window shifts
Start with warped version of image 2, as per Dwarp 2 _
Image 1
Image 2
Transformed

15. Metric 3 - Dshift _ 2 As per Warping but also allow sub-window shifts
Start with warped version of image 2, as per Dwarp 2 _
Transformed

16. Metric 3 - Dshift _ 2 As per Warping but also allow sub-window shifts
Start with warped version of image 2, as per Dwarp 2 _

17. Metric 3 - Dshift _ 2 As per Warping but also allow sub-window shifts
Local sub-window

18. Metric 3 - Dshift _ 2 As per Warping but also allow sub-window shifts
Quick since images are so small 2 _
Local sub-window

19. Metric 3 - Dshift
As per Warping but also allow sub-window shifts Tried various sizes of sub-window  1x1 (i.e. single pixel) worked best 2 _
Local sub-window

20. Comparison of metrics
SSD

21. Neighbors with Different Metrics

22. Examples
Solving vision by brute force
Normalized correlation scores

23. How Many Images Are There?
Note: D1=DSSD

24. How Does Dssd Relate to Semantic Distance?
Subject 1
Subject 2
Subject 3

25. How Does Dssd Relate to Semantic Distance?
Subject 1
Subject 2
Subject 3