Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search

Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult, CVPR 2012 University of Oxford 5 th December 2012

Attributes based image description

4-Legged Orange Striped Furry White Symmetric Ionic columns Classical Male Asian Beard Smiling Slide Courtesy: Neeraj Kumar

Attribute Classifiers

Attribute and Simile Classifiers for Face Verification

N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar ICCV 2009

FaceTracer: A Search Engine for Large Collections of Images with Faces

N. Kumar, P. N. Belhumeur, and S. K. Nayar ICCV 2009

Attributes Fusion

FaceTracer: “ smiling asian men with glasses ” Slide Courtesy: Neeraj Kumar

Score Normalization: Problem

• Necessary to prevent high confidence for one attribute from dominating the results.

• Ideal normalization technique should, 1) Normalize scores to a uniform range say, [0,1] 2) Assign perceptual quality to the scores.

• Positive and negative distributions of different classifiers do not necessarily follow same distribution.

• Fitting a Gaussian or any other distribution to scores satisfies condition 1 but doesn’t satisfy condition 2.

Negative Scores Distributions Positive Scores Distributions

Score Normalization: Solution

• Model distance between positive scores and the negative scores .

• If we knew distribution of negative scores, we could do a hypothesis test for each positive score using that distribution.

• Unfortunately, we don’t know anything about overall negative distribution.

But, we know something about tail of the negative score distribution.

Extreme Value Theory

• Central Limit Theorem: • The “mean” of a sufficiently large iid random variables will be distributed according to Normal distribution • Extreme Value Theory: • The maximum of a sufficiently large iid random variable will be distributed according to Gumbell, Frechet or Weibull distribution.

• If the values are bounded from above and below, the the values are distributed according to “Weibull” distribution.

Weibull Distribution

• Weibull Distribution PDF CDF k and λ are shape and location parameters respectively.

PDF CDF

Extreme Value Theory: Application

Overall Negative Score Distribution Tail Maximum values of random variables • • Tail of negative scores can be seen as a collection of maxima of some random variables. Hence it follows Weibull distribution according to Extreme Value Theory.

W-score normalization: Procedure

For any classifier, • Fix the decision boundary on the scores (Ideally this should be at score = 0 ) • Select maximum N (tail size) samples from negative side of the boundary.

• Fit a Weibull Distribution to these tail scores.

• Renormalize scores using Cumulative Density Function (CDF) of this Weibull distribution.

Results: Dataset

• “Labeled Faces In The Wild” dataset.

• About 13,000 images of 5000 celebrities.

• 75 different attribute classification scores available from “ Attribute and Simile Classifiers for Face Verification”. Kumar et al. ICCV 09.

Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments.

Results

Multi Attribute Fusion:

• Joint score can be computed as multiplication of individual attribute probabilities.

• Attributes may not be independent.

• Low probability due to: • bad classifier • absence of images belonging to an attribute.

• Instead of product, authors propose use l1 norm of probabilities as a fusion score.

Results

Similarity Search:

• Given an image and a set of attributes, find nearest images.

• Perceived difference between images in different ranges might be similar.

• Distances between query attribute and its nearest neighbor needs to be normalized.

• • • Normalize query attribute scores on query image.

Get nearest neighbor distances.

Fit Weibull distribution to distances.

Summary

• Provides way of normalizing scores intuitively.

• Provides way for combining attributes. • Relies on finding the right threshold and tail size. Requires fair bit of tuning.

Questions?