An Invariant Large Margin Nearest Neighbour ClassifierAn Invariant Large Margin Nearest Neighbour Classifier

ResultsMatching Faces from TV Video

Aim: To learn a distance metric for invariant nearest neighbour classification

Large Margin NN (LMNN)

Invariant LMNN (ILMNN) Drawbacks

A Property of Polynomial Transformations

x Lx

xi xj

Same class points closer

Different class points away

Polynomial Transformations

Euclidean DistanceCurrent Fix

• Overfitting - O(D2) parameters

• Use rank deficient L (non-convex)

• No invariance to transformations

• Add synthetically transformed data• Inefficient • Inaccurate

Transformation Trajectory Finite Transformed Data

Nearest Neighbour Classifier (NN)• Multiple classes• Labelled training data

Euclidean Transformation-5o ≤ ≤ 5o -3 ≤ tx ≤ 3 pixels -3 ≤ ty ≤ 3 pixels

Method Exp1 Exp2

kNN-E 83.6 26.7

L2-LMNN 61.2 22.6

D-LMNN 85.6 24.3

DD-LMNN 84.4 24.5

L2-ILMNN 65.9 24.0

D-ILMNN 87.2 32.0

DD-ILMNN 86.6 29.8

M-SVM 62.3 30.0

SVM-KNN 75.5 28.1

Invariance to changes in position of features

First Experiment (Exp1)

Second Experiment (Exp2)

True Positives

• Randomly permute data• Train/Val/Test - 30/30/40• Suitable for NN

• No random permutation• Train/Val/Test - 30/30/40• Not so suitable for NN

11 characters

24,244 faces

min ∑ ij dij = (xi-xj)TLTL(xi-xj)

min ∑ ij (xi-xj)TM(xi-xj)

M 0 (Positive Semidefinite)

Semidefinite Program (SDP)

(xi-xk)TM(xi-xk)- (xi-xj)TM(xi-xj)

≥ 1 - eijk

min ∑ ijk eijk, eijk ≥ 0

EXP2

EXP1

2D Rotation Example

ab

cos θ sin θ

-sin θ

cos θ

1-θ2/2 -(θ-θ3/6)

(θ-θ3/6) 1-θ2/2

ab

a 1θ

b -a/2 b/6 b a -b/2 -a/6 θ2

θ3

=

Univariate Transformation

X Taylor’s Series Approximation X

• Multivariate Polynomial Transformations - Euclidean, Similarity, Affine

• General Form: T(x,) = X

Distance between Polynomial Trajectories

Sum of SquaresOf Polynomials

SD - Representability

Lasserre, 2001xi xj

Dij

P’ 0

SD - Representabilityof Segments

Mij mij

x Lx

xk

• Commonly used in Computer Vision

Minimize Max Distanceof same class trajectories

Maximize Min Distanceof different class trajectories

Euclidean Distance

Learnt Distance

Non-convex

Approximation: ijdij ij = Mij

mij

dij

min ∑ ij ijdij

Convex SDP

Dik(1, 2) - ijdij ≥ 1 - eijk

min ∑ ijk eijk, eijk ≥ 0

Pijk 0 (SD-representability)

M. Pawan Kumar Philip H.S. TorrM. Pawan Kumar Philip H.S. Torr Andrew ZissermanAndrew Zisserman

• Find nearest neighbours, classify• Typically, Euclidean distance used

Our Contributions

Adding Invariance using Polynomial Transformations

Overcome above drawbacks of LMNN• Regularization of parameters L or M

• Invariance to Polynomial TransformationsPreserve Convexity

P 0

Regularization • Prevent overfitting • Retain convexity• L2-LMNN Minimize L2 norm of parameter L min ∑ i M(i,i)

• D-LMNN Learn diagonal L diagonal M M(i,j) = 0, i ≠ j• DD-LMNN Learn a diagonally dominant M min ∑ i,j |M(i,j)|, i ≠ j

1

2

Weinberger, Blitzer, Saul - NIPS 2005

Globally Optimum M (and L)

Euclidean Distance

Learnt Distance

Euclidean Distance

Learnt Distance

(θ1 ,θ2)

xi xj

xk

dij

Accuracy

Method Train Test

kNN-E - 62.2 s

L2-LMNN 4 h 62.2 s

D-LMNN 1 h 53.2 s

DD-LMNN 2 h 50.5 s

L2-ILMNN 24 h 62.2 s

D-ILMNN 8 h 48.2 s

DD-ILMNN 24 h 51.9 s

M-SVM 300 s 446.6 s

SVM-KNN - 2114.2 s

TimingsPrecision-Recall

http://cms.brookes.ac.uk/research/visiongrouphttp://cms.brookes.ac.uk/research/visiongroup http://www.robots.ox.ac.uk/~vgghttp://www.robots.ox.ac.uk/~vggVision Group, Oxford Brookes UniversityVision Group, Oxford Brookes University Visual Geometry Group, Oxford UniversityVisual Geometry Group, Oxford University

RD RD

-T ≤ ≤ T

1

2

Dik(1, 2)