An Invariant Large Margin Nearest Neighbour Classifier Results Matching Faces from TV Video Aim: To learn a distance metric for invariant nearest neighbour classification Large Margin NN (LMNN) Invariant LMNN (ILMNN) Drawbacks A Property of Polynomial Transformations x Lx xixi xjxj Same class points closer Different class points away Polynomial Transformations Euclidean Distance Current Fix Overfitting - O(D 2 ) parameters Use rank deficient L (non-convex) No invariance to transformations Add synthetically transformed data Inefficient Inaccurate Transformation TrajectoryFinite Transformed Data Nearest Neighbour Classifier (NN) Multiple classes Labelled training data Euclidean Transformation -5 o ≤  ≤ 5 o -3 ≤ t x ≤ 3 pixels-3 ≤ t y ≤ 3 pixels MethodExp1Exp2 kNN-E L 2 -LMNN D-LMNN DD-LMNN L 2 -ILMNN D-ILMNN DD-ILMNN M-SVM SVM-KNN Invariance to changes in position of features First Experiment (Exp1) Second Experiment (Exp2) True Positives Randomly permute data Train/Val/Test - 30/30/40 Suitable for NN No random permutation Train/Val/Test - 30/30/40 Not so suitable for NN 11 characters 24,244 faces min ∑ ij d ij = (x i -x j ) T L T L(x i -x j ) min ∑ ij (x i -x j ) T M(x i -x j ) M 0 (Positive Semidefinite) Semidefinite Program (SDP) (x i -x k ) T M(x i -x k )- (x i -x j ) T M(x i -x j ) ≥ 1 - e ijk min ∑ ijk e ijk, e ijk ≥ 0 EXP2EXP2 EXP1EXP1 2D Rotation Example a b cos θ sin θ -sin θ cos θ 1-θ 2 /2 -(θ-θ 3 /6) (θ-θ 3 /6) 1-θ 2 /2 a b a 1 θ b-a/2b/6 ba-b/2-a/6 θ2θ2 θ3θ3  = Univariate Transformation X Taylor’s Series Approximation X  Multivariate Polynomial Transformations - Euclidean, Similarity, Affine General Form: T(x,  ) = X  Distance between Polynomial Trajectories Sum of Squares Of Polynomials SD - Representability Lasserre, 2001 xixi xjxj D ij  P’ 0 SD - Representability of Segments M ij m ij x Lx xkxk Commonly used in Computer Vision Minimize Max Distance of same class trajectories Maximize Min Distance of different class trajectories Euclidean Distance Learnt Distance Non-convex Approximation:  ij d ij  ij = M ij m ij d ij min ∑ ij  ij d ij Convex SDP D ik (  1,  2 ) -  ij d ij ≥ 1 - e ijk min ∑ ijk e ijk, e ijk ≥ 0 P ijk 0 (SD-representability) M. Pawan Kumar Philip H.S. Torr Andrew Zisserman Find nearest neighbours, classify Typically, Euclidean distance used Our Contributions Adding Invariance using Polynomial Transformations Overcome above drawbacks of LMNN Regularization of parameters L or M Invariance to Polynomial Transformations Preserve Convexity P 0 Regularization Prevent overfitting Retain convexity L 2 -LMNN Minimize L 2 norm of parameter L min ∑ i M(i,i) D-LMNN Learn diagonal L  diagonal M M(i,j) = 0, i ≠ j DD-LMNN Learn a diagonally dominant M min ∑ i,j |M(i,j)|, i ≠ j 11 22 Weinberger, Blitzer, Saul - NIPS 2005 Globally Optimum M (and L) Euclidean Distance Learnt Distance Euclidean Distance Learnt Distance (θ 1,θ 2 ) xixi xjxj xkxk d ij Accuracy MethodTrainTest kNN-E-62.2 s L 2 -LMNN4 h62.2 s D-LMNN1 h53.2 s DD-LMNN2 h50.5 s L 2 -ILMNN24 h62.2 s D-ILMNN8 h48.2 s DD-ILMNN24 h51.9 s M-SVM300 s446.6 s SVM-KNN s Timings Precision-Recall Vision Group, Oxford Brookes University Visual Geometry Group, Oxford University R D -T ≤  ≤ T 11 22 D ik (  1,  2 )