Face RecognitionFace Recognition Under Under Varying IlluminationVarying Illumination

Erald VUÇINI Vienna University of Technology

Muhittin GÖKMEN Istanbul Technical University

Eduard GRÖLLER Vienna University of Technology

Erald Vuçini - Vienna University of Technology 2

Face Recognition System

Image Capture

Face Identification

Face Detection

Database

Feature Projection

Erald Vuçini - Vienna University of Technology 3

Face Recognition Approaches

Principal Component Analysis (PCA)

Linear Discriminant Analysis (LDA)

Local Feature Analysis

Active Appearance Model

Hidden Markov Model

Support Vector Machine

…

Erald Vuçini - Vienna University of Technology 4

Face Recognition – Problems

The variations of the same face due toilluminationviewing direction

are almost always larger than image variations due to changes in the face identity

Erald Vuçini - Vienna University of Technology 5

Handling Variable Illumination

Extract illumination invariant features

Transform variable illumination to canonical representation

Model 2D illumination variations

Utilize 3D face models whose shapes and albedos are obtained in advance

Erald Vuçini - Vienna University of Technology 6

I. Dimensionality Reduction - LDA better than PCA regarding illumination

II. Image Synthesis - Solve the Small Sample Size (SSS) problem

III. Reconstruction – Restore frontal illumination

Outline of proposed approach

Erald Vuçini - Vienna University of Technology 7

I. Dimensionality Reduction - LDA better than PCA regarding illumination

II. Image Synthesis - Solve the Small Sample Size (SSS) problem

III. Reconstruction – Restore frontal illumination

Dimensionality Reduction

Erald Vuçini - Vienna University of Technology 8

Principal Component Analysis (PCA)

One of the most commonly used methods in Face Recognition

Maximizes the scattering of all projected samples

PCAx1

x2

x1

x2z1

z 2

Erald Vuçini - Vienna University of Technology 9

PCA under Varying Illumination

PCA fails with variant illumination

The scatter being maximized is due to

Between-class scatter

Within-class scatter

Discard 3 most significant principal components to reduce lighting variation

Erald Vuçini - Vienna University of Technology 10

LDA Interpretation

LDA is a class specific method

LDA

Erald Vuçini - Vienna University of Technology 11

LDA Problems

LDA maximizes the ratio of Between-class scatter and Within-class Scatter

Within-class Scatter singularity problem

Fisher LDA (FLDA) removes Null Space

FLDA handles best the variation in lighting, having lower error rate than PCA

Erald Vuçini - Vienna University of Technology 12

I. Dimensionality Reduction - LDA better than PCA regarding illumination

II. Image Synthesis - Solve the Small Sample Size (SSS) problem

III. Reconstruction – Restore frontal illumination

Image Synthesis

Erald Vuçini - Vienna University of Technology 13

Image Synthesis - Motivation

Face Recognition Systems

Performance related with training database

LDA require many samples per class

In many systems only one image per person is provided

Quotient Image makes possible the synthesis of the image space of a given input image

Erald Vuçini - Vienna University of Technology 14

Lambertian Objects - Faces

The image space lives in a 3D linear subspace

Three images are sufficient for generating the image space of the object

TImage n s

Albedo Surface Normal

Light Source Direction

Erald Vuçini - Vienna University of Technology 15

Quotient Image (Definitions)

Ideal class of facesSame shape

Different albedosT

i n s Synthesis Problem:

Given 3N images of N faces of the same class, illuminated under 3 lighting conditions

Synthesize image space of new input

Erald Vuçini - Vienna University of Technology 16

Quotient Image (Definitions)

Given objects y and a we define quotient image Q by the ratio of their albedos

Q is illumination invariant

Image space of y can be generated with

Quotient Q

3 images of a

Generalization: Use bootstrap of 3N images

Erald Vuçini - Vienna University of Technology 17

Quotient Image - Examples

Quotient

Quotient

Erald Vuçini - Vienna University of Technology 18

Quotient Image – Image Space Synthesis

Erald Vuçini - Vienna University of Technology 19

10 person Bootstrap

5 person Bootstrap

1 person Bootstrap

Erald Vuçini - Vienna University of Technology 20

I. Dimensionality Reduction - LDA better than PCA regarding illumination

II. Image Synthesis - Solve the Small Sample Size (SSS) problem

III. Reconstruction – Restore frontal illumination

Reconstruction

Erald Vuçini - Vienna University of Technology 21

YaleB Testing Database

Yaleb Database

450 images of 10 persons

Divided in 4 subsets

Subset1 up to 10˚

Subset2 up to 25˚

Subset3 up to 45˚

Subset4 up to 75˚

Normalized

Erald Vuçini - Vienna University of Technology 22

Histogram Equalization

Histogram equalization(HE) done as preprocessing increases the recognition rate

Adaptive HE(AHE) is used as a preprocessing step in the iterative face recognition approach

Results with the YaleB Database (PCA used)

No Preprocessing HE AHE

Recognition Rate(%) 43.4 74 81.5

Erald Vuçini - Vienna University of Technology 23

Illumination Restoration Approach

A face image with arbitrary illumination is restored to having frontal illumination. It has the following advantages:

No need to estimate face surface normals

No need to estimate light source directions and albedos

No need to perform image warping

Face images will be visually natural looking

Erald Vuçini - Vienna University of Technology 24

Algorithm Outline

Compute mean face image and eigenspace

Compute initial restored images

Create iteration by replacing Bro with blurred Hio

Continue iteration until stopping criteria satisfied

roio ik

ik

BH IB

Input Image Blurred Reference Image

Blurred Input Image

Restored Image

Erald Vuçini - Vienna University of Technology 25

Iteration Steps

Erald Vuçini - Vienna University of Technology 26

Experimental Results (Subset 3)

Restoration

Erald Vuçini - Vienna University of Technology 27

Experimental Results (Subset 4)

Restoration

Erald Vuçini - Vienna University of Technology 28

Results with YaleB Database

Subset1

Subset2

Subset3

Subset4

Overall

HE+PCA(%)

100 97.5 66.4 44.2 74

HE+New(%)

100 100 92.8 91.7 95.6

Subset1

Subset2

Subset3

Subset4

Overall

AHE+PCA(%)

100 100 83.57 50.0 81.6

AHE+New(%)

100 100 100 95.0 98.7

Erald Vuçini - Vienna University of Technology 29

Thank you for the attention!

Proposed Method

Dimension Reduction

Image Synthesis

Reconstruction