Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search University of Oxford 5 th December 2012 Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult, CVPR 2012

Attributes based image description 4-Legged Orange Striped Furry White Symmetric Ionic columns Classical Male Asian Beard Smiling Slide Courtesy: Neeraj Kumar

Attribute Classifiers Attribute and Simile Classifiers for Face Verification N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar ICCV 2009 FaceTracer: A Search Engine for Large Collections of Images with Faces N. Kumar, P. N. Belhumeur, and S. K. Nayar ICCV 2009

Attributes Fusion FaceTracer: “smiling asian men with glasses” Slide Courtesy: Neeraj Kumar

Score Normalization: Problem Necessary to prevent high confidence for one attribute from dominating the results. Ideal normalization technique should, 1)Normalize scores to a uniform range say, [0,1] 2)Assign perceptual quality to the scores. Positive and negative distributions of different classifiers do not necessarily follow same distribution. Fitting a Gaussian or any other distribution to scores satisfies condition 1 but doesn’t satisfy condition 2. Negative Scores DistributionsPositive Scores Distributions

Score Normalization: Solution Model distance between positive scores and the negative scores. If we knew distribution of negative scores, we could do a hypothesis test for each positive score using that distribution. Unfortunately, we don’t know anything about overall negative distribution. But, we know something about tail of the negative score distribution.

Extreme Value Theory Central Limit Theorem: The “mean” of a sufficiently large iid random variables will be distributed according to Normal distribution Extreme Value Theory: The maximum of a sufficiently large iid random variable will be distributed according to Gumbell, Frechet or Weibull distribution. If the values are bounded from above and below, the the values are distributed according to “Weibull” distribution.

Weibull Distribution PDF CDF k and λ are shape and location parameters respectively. PDFCDF

Extreme Value Theory: Application Tail Overall Negative Score Distribution Maximum values of random variables Tail of negative scores can be seen as a collection of maxima of some random variables. Hence it follows Weibull distribution according to Extreme Value Theory.

W-score normalization: Procedure For any classifier, Fix the decision boundary on the scores (Ideally this should be at score = 0 ) Select maximum N (tail size) samples from negative side of the boundary. Fit a Weibull Distribution to these tail scores. Renormalize scores using Cumulative Density Function (CDF) of this Weibull distribution.

Results: Dataset “Labeled Faces In The Wild” dataset. About 13,000 images of 5000 celebrities. 75 different attribute classification scores available from “ Attribute and Simile Classifiers for Face Verification”. Kumar et al. ICCV 09. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments.

Results

Multi Attribute Fusion: Joint score can be computed as multiplication of individual attribute probabilities. Attributes may not be independent. Low probability due to: bad classifier absence of images belonging to an attribute. Instead of product, authors propose use l1 norm of probabilities as a fusion score.

Results

Similarity Search: Given an image and a set of attributes, find nearest images. Perceived difference between images in different ranges might be similar. Distances between query attribute and its nearest neighbor needs to be normalized. Normalize query attribute scores on query image. Get nearest neighbor distances. Fit Weibull distribution to distances.

Summary Provides way of normalizing scores intuitively. Provides way for combining attributes. Relies on finding the right threshold and tail size. Requires fair bit of tuning.

Questions?