Facial RecognitionFacial Recognition

CSE 391CSE 391

Kris LordKris Lord

BackgroundBackground

Face recognition is one of the fundamental Face recognition is one of the fundamental problems in pattern analysisproblems in pattern analysis

Difficulties arise due to large variation in Difficulties arise due to large variation in facial appearance, head size, orientation facial appearance, head size, orientation and change in environmental conditionsand change in environmental conditions

Computerized face recognition system still Computerized face recognition system still cannot achieve a completely reliable cannot achieve a completely reliable performanceperformance

Main IssuesMain Issues

Often in practical situations, recognition must be Often in practical situations, recognition must be achieved in real-time so efficiency and speed are crucialachieved in real-time so efficiency and speed are crucialVariance in lighting, angles, and other environmental Variance in lighting, angles, and other environmental areas make recognition more of a problem to deal withareas make recognition more of a problem to deal withMay be hard to obtain a complete database of a May be hard to obtain a complete database of a population’s faces in optimal posture/lighting for population’s faces in optimal posture/lighting for processingprocessingFalse positives/inability to recognize a face still common False positives/inability to recognize a face still common in current state of algorithmsin current state of algorithmsStorage space a large issue, especially when dealing Storage space a large issue, especially when dealing with matrix-based algorithms (the more detailed the with matrix-based algorithms (the more detailed the picture, the larger the storage space needed)picture, the larger the storage space needed)

3 Main Steps3 Main Steps

Face detectionFace detection

- Facial area is singled out and removed - Facial area is singled out and removed

for processing within a noisy imagefor processing within a noisy image

Face normalizationFace normalization

- Facial image is processed to counteract posture issues such as tilt, - Facial image is processed to counteract posture issues such as tilt, angle, lighting, and other environmental noise angle, lighting, and other environmental noise

Face verification/recognitionFace verification/recognition

- Facial features are analyzed via a recognition algorithm to - Facial features are analyzed via a recognition algorithm to determine a match with an existing face in a databasedetermine a match with an existing face in a database

““Eigenfaces” ApproachEigenfaces” Approach

Patterns, in the domain of facial recognition could be the Patterns, in the domain of facial recognition could be the presence of some objects (eyes, nose, mouth) in a face presence of some objects (eyes, nose, mouth) in a face as well as relative distances between these objects. as well as relative distances between these objects. These characteristic features are called These characteristic features are called eigenfaces eigenfaces in the in the facial recognition domain (or facial recognition domain (or principal components principal components generally). They can be extracted out of original image generally). They can be extracted out of original image data by means of a mathematical tool called data by means of a mathematical tool called Principal Principal Component AnalysisComponent Analysis (PCA). (PCA).Each Each eigenfaceeigenface represents only certain features of the represents only certain features of the face. If the feature is present in the original image to a face. If the feature is present in the original image to a higher degree, the share of the corresponding higher degree, the share of the corresponding eigenfaceeigenface in the ”sum” of the in the ”sum” of the eigenfaceseigenfaces should be greater. should be greater.In order to cut down on large computational processing, In order to cut down on large computational processing, only only eigenfaceseigenfaces with the highest value (most with the highest value (most characteristic facial features) are kept for processing characteristic facial features) are kept for processing

Common “Eigenface” AlgorithmCommon “Eigenface” AlgorithmA set of training data (pictures of faces) are A set of training data (pictures of faces) are transformed into a set E of Eigenfacestransformed into a set E of EigenfacesAfterwards, the weights are calculated for each Afterwards, the weights are calculated for each image of the training set and stored in the set image of the training set and stored in the set WWUpon observing an unknown image X, the weights Upon observing an unknown image X, the weights are calculated for that particular image and stored are calculated for that particular image and stored in the vector WX. Afterwards, WX is compared in the vector WX. Afterwards, WX is compared with the weights of images, of which one knows with the weights of images, of which one knows for certain that they are faces (the weights of the for certain that they are faces (the weights of the training set W)training set W)If this average distance exceeds some threshold If this average distance exceeds some threshold value value , then the weight vector of the unknown , then the weight vector of the unknown image image WXWX lies too ”far apart” from the weights of lies too ”far apart” from the weights of the faces. In this case, the unknown the faces. In this case, the unknown X X is is considered to not a face. If it is considered to be a considered to not a face. If it is considered to be a face, its weight vector face, its weight vector WXWX is stored for later is stored for later classification, where it can be tested against classification, where it can be tested against specific images and their eigenfaces.specific images and their eigenfaces.

Success rate?Success rate?

Some algorithms are Some algorithms are much more successful much more successful than othersthan othersSuccess rate depends Success rate depends greatly on database of greatly on database of faces usedfaces usedRate can vary Rate can vary considerably if databases considerably if databases are combined are combined (“eigenface” success rate (“eigenface” success rate drops considerably, to drops considerably, to 66% with combined 66% with combined databases)databases)

Practical ApplicationsPractical Applications

Combat Terrorism/Airport SecurityCombat Terrorism/Airport SecurityLarge event (e.g. Superbowl) security – ability to Large event (e.g. Superbowl) security – ability to scan the crowd with a video camera and match scan the crowd with a video camera and match against a database of criminal recordsagainst a database of criminal recordsEliminate fake IDsEliminate fake IDsEliminate identity theft (ATMs)Eliminate identity theft (ATMs)Casino securityCasino securityTailored (personalized) advertisements of the Tailored (personalized) advertisements of the futurefutureOnline dating profilingOnline dating profiling

Current State of the ArtCurrent State of the Art

Neural Net algorithmsNeural Net algorithms

Elastic matching algorithmsElastic matching algorithms

NEC Developed 3D face recognition NEC Developed 3D face recognition algorithm with over 96.5% recognition rate algorithm with over 96.5% recognition rate under bad environmental conditions under bad environmental conditions