1. Patch-Based Image Classification Using Image Epitomes
David Andrzejewski
Computer Sciences 766
Fall 2005

2. Problem Statement Given
Positive and negative example images for a certain classification (contains face, is outdoors, etc) Do
Develop classifier capable of classifying new images as positive or negative

3. Image Epitomes Consists of patches and mappings
Input image
A set of image patches
Image epitome
Consists of patches and mappings
Patches and mappings are learned with EM
Applications in vision (de-noising,segmentation,others)

4. Original image can then be reconstructed by mosaicing epitome patches
Image Reconstruction
Original image can then be reconstructed by mosaicing epitome patches

5. Recognition / Detection / Classification
The smiling point
Epitome of 295 face images
Images with the highest total posterior at the “smiling point”
Images with the lowest total posterior at the “smiling point”

6. Approach Construct collage of positive and negative examples
Learn the image epitome of the training collage
Find epitome patches that are preferentially mapped into the positive example images in the collage
Calculate P(patch(i)|pos/neg) for these patches (also use psuedo-counts)
Use these patches to classify new images by calculating odds ratio

7. Preliminary Results Training Collage Epitome Negative Test Images
Positive Test Images

8. Problems with Approach
Difficult to incorporate new examples
Would need to add to collage and re-learn epitome (is there a better way?)
“Bag of words” → Spatial information discarded
Not model-based
Pose/Illumination/Scale-variant
Only way to handle variation is to include training examples for various conditions

9. Potential Modifications
Cluster training images
Ex: Training images w/ low vs high illumination
Discriminative patches may map exclusively to one subset of positive images → take this into account
Change “winner take all” for P calculations
Consider relative probabilities of 'near matches'
Account for multiple mappings somehow

10. References
V. Cheung, B. J. Frey, and N. Jojic, Video epitomes, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005
N. Jojic, B. J. Frey, and A. Kannan, Epitomic analysis of appearance and shape, Proc. 9th Int. Conf. Computer Vision, 2003
R. Fergus, P. Perona, A. Zisserman, Object Class Recognition by Unsupervised Scale-Invariant Learning, Proc. of the IEEE Conf on Computer Vision and Pattern Recognition, 2003
Testing images from Google Images and Flickr