Face Recognition

• Summary– Single pose– Multiple pose– Principal components analysis– Model-based recognition– Neural Networks

Single Pose

• Standard head-and-shoulders view with uniform background

• Easy to find face within image

Aligning Images

Alignment– Faces in the training set must be aligned with

each other to remove the effects of translation, scale, rotation etc.

– It is easy to find the position of the eyes and mouth and then shift and resize images so that are aligned with each other

Nearest Neighbour

• Once the images have been aligned you can simply search for the member of the training set which is nearest to the test image.

• There are a number of measures of distance including Euclidean distance, and the cross-correlation

Principal Components

• PCA reduces the number of dimensions and so the memory requirement is much reduced.

• The search time is also reduced

Two ways to apply PCA (1)

• We could apply PCA to the whole training set.

• Then each face is represented by a point in the PC space

• We could then apply nearest neighbour to these points

Two ways to apply PCA (2)

• Alternatively we could apply PCA to the set of faces belonging to each person in the training set

• Each class (person) is then reprented by a different ellipsoid and Mahalanobis distance can be used to classify a new unknown face

• You need a lot of images of each person to do this

Problems with PCA

• The same person may sometimes appear differently due to– Beards, moustaches– Glasses,– Makeup

• These have to be represented by different ellipsoids

                                                                          

-------(2)--------------(3)--------------(4)-------

                                                                          

-------(5)--------------(6)--------------(7)-------

                                                                          

-------(8)--------------(9)--------------(10)-------

                                                                          

Problems with PCA

• Facial expressions– Differing facial expressions

• Opening and closing the mouth• Raised eyebrows• Widening the eyes• Smiling, frowing etc,

• These mean that the class is no longer ellipsoidal and must be represented by a manifold

Facial Expressions

• There are six types of facial expression• We could use PCA on the eyes and mouth – so we

could have eigeneyes and eigenmouths

Anger Fear Disgust Happy Sad Surprise

Multiple Poses

• Heads must now be aligned in 3D world space

• Classes now form trajectories in feature space

• It becomes difficult to recognise faces because the variation due to pose is greater than the variation between people

Model-based Recognition

• We can fit a model directly to the face image

• Model consists of a mesh which is matched to facial features such as the eyes, nose, mouth and edges of the face.

• We use PCA to describe the parameters of the model rather than the pixels.

Model-based Recognition

• The model copes better with multiple poses and changes in facial expression.