Face Recognition Committee Machine: Methodology, Experiments and A System Application

Face Recognition Committee Machine:Methodology, Experiments and A System Application

Oral Defense by Sunny Tang15 Aug 2003

OutlineIntroduction

Face RecognitionProblems and Objectives

Face Recognition Committee Machine

Committee MembersResult, Confidence and WeightStatic and Dynamic Structure

OutlineFace Recognition System

System ArchitectureFace Recognition ProcessDistributed Architecture

Experimental ResultsConclusionQ & A

Introduction: Face RecognitionDefinition

A recognition process that analyzes facial characteristics

Two modes of recognitionIdentification: “Who is this”Verification: “Is this person who she/he claim to be?”

Face Recognition ApplicationsSecurity

Access control systemLaw enforcement

Multimedia databaseVideo indexingHuman search engine

Problems & ObjectivesCurrent problems of existing algorithms

No objective comparisonAccuracy not satisfactoryCannot handle all kinds of variations

ObjectivesProvide thorough and objectively comparisonPropose a framework to integrate different algorithms for better performanceImplement a real-time face recognition system

Face Recognition Committee Machine (FRCM)

MotivationAchieve better accuracy by combining predictions of different experts

Two structures of FRCMStatic structure (SFRCM)Dynamic structure (DFRCM)

Static vs. DynamicStatic structure

Ignore input signalsFixed weights

Dynamic structureEmploy input signal to improve the classifiersVariable weights

Committee MembersTemplate matching approach

EigenfaceFisherfaceElastic Graph Matching (EGM)

Machine learning approachSupport Vector Machines (SVM)Neural Networks (NN)

Review: Eigenface & Fisherface

Feature spaceEigenface: Principal Component Analysis (PCA)Fisherface: Fisher’s Linear Discriminant (FLD)

Training & RecognitionProject images on feature spaceCompare Euclidean distance and choose the closest projection

Review: Elastic Graph MatchingBased on dynamic link architecture

Extract facial feature by Gabor wavelet transformFace is represented by a graph consists of nodes of jets

Compare graphs by cost functionEdge similarity Se and vertex similarity Sv

Cost function

Review: SVM & Neural NetworksSVM

Look for a separating hyperplane which separates the data with the largest margin

Neural NetworksAdjust neuron weights to minimize prediction error between the target and output

Result, Confidence & Weight Result

Result of expertConfidence

Confidence of expert on its resultWeight

Weight of expert’s result in ensemble

SFRCM Architecture

Result & Confidence (1)Eigenface, Fisherface & EGM

Result:• Identification:

• Verification:

Confidence:• Identification:

• Verification:

Result & Confidence (2)SVM

One-against-one approachResult:• Identification: SVM result • Verification: direct matching

Confidence:

Result & Confidence (3)Neural network

A binary vector of size J for target representationResult:

• Identification:

• Verification:

Confidence: output value oj

WeightDerived from performance of expert:

Amplify the difference of the performance:

Normalize in range [0, 1]:

Voting MachineAssemble result and confidenceScore of expert’s result:

Ensemble result:

SFRCM DrawbacksFixed weights under all situations

The weights of the experts are fixed no matter which images are given.

No update mechanismThe weights cannot be updated once the system is trained

DFRCM ArchitectureGating network is includedImage is involved in determination of weight

Gating NetworkKeep the performance of experts on different face databasesDetermine the database of input imageGive the corresponding weights of the experts for that database

Feedback Mechanism1. Initialize ni,j and ti,j to 02. Train each expert i on different database j3. While TESTING

a) Determine j for each test imageb) Recognize the image in each expert ic) If ti,j != 0 then Calculate pi,j

d) Else Set pi,j = 0e) Calculate wi,j

f) Determine ensemble resultg) If FEEDBACK then Update ni,j and ti,j

4. End while

Implementation: Face Recognition System

Real-time face recognition systemImplementation of FRCMFace processing

Face trackingFace detectionFace recognition

System Architecture

Face Recognition ProcessEnrollment

Collect face images to train the experts

RecognitionIdentificationVerification

System Snapshots

Identification Verification

Problems of FRCM on mobile device

Memory limitationLittle memory for mobile devicesRequirement for recognition

CPU power limitationTime and storage overhead of FRCM

Distributed ArchitectureClient

Capture imageEnsemble results

ServerRecognition

Distributed System: EvaluationImplementation

Desktop (1400MHz), notebook (300MHz)S: Startup, R: Recognition

Distinct servers:

Experimental ResultsDatabases used:

ORL from AT&T LaboratoriesYale from Yale UniversityAR from Computer Vision Center at U.A.B HRL from Harvard Robotics Laboratory

Cross validation testing

1. Apply median filter to reduce noise in background2. Apply Sobel filter for edge detection3. Covert to a binary image4. Apply horizontal and vertical projection5. Find face boundary 6. Obtain the center of the face region.7. Crop the face region and resize it

Preprocessing

ORL Result ORL Face database

400 images40 peopleVariations

• Position• Rotation• Scale• Expression

Yale ResultYale Face Database

165 images15 peopleVariations

• Expression• Lighting

AR ResultAR Face Database

1300 images130 peopleVariations

• Expression• Lighting• Occlusions

HRL ResultHRL Face Database

345 images5 peopleVariation

• Lighting

Average Running Time & ResultsAverage running time

Average experimental results

ConclusionMake a thorough comparison of five face recognition algorithmsPropose FRCM to integrate different face recognition algorithmsImplement a face recognition system for real-time applicationPropose a distributed architecture for mobile device

Question & Answer Section

Thanks