Pattern Recognition 39 (2006) 1542–1545www.elsevier.com/locate/patcog

Rapid and brief communication

Face recognition robust to left/right shadows; facial symmetry

Young-Jun Song, Young-Gil Kim∗, Un-Dong Chang, Heak Bong KwonDepartment of Computer and Communication Engineering, Chungbuk National University, 12 Gaesin-dong, Heungduk-gu Cheongju,

Chungbuk 361-763, Korea

Received 12 October 2005; accepted 15 February 2006

Abstract

This paper has proposed an efficient shaded-face pre-processing technique using front-face symmetry. The existing face recognitionPCA technique has a shortcoming of making illumination variation lower the recognition performance of a shaded face. The study hasaimed to improve the performance by using the symmetry of the left and right face.

In order to evaluate the performance of the proposed face recognition method, the study experimented with the Yale face databasewith left/right shadows. The experimental methods for this are as following: the existing PCA, PCA with first three eigenfaces excluded,histogram equalization and the proposed method. As the result, it was shown that the proposed method has a rather excellent recognitionperformance (98.9%).� 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

Keywords: Face recognition; PCA; Facial symmetry; Illumination

1. Introduction

Face recognition has been being studied as the ultimatecertification field of biometric recognition. Particularly, itcan be said that face recognition in security system is anessential technology for crime prevention and user certifi-cation. Also, high-qualified and low-priced computers andcameras are promoting popularization of the face recogni-tion system.

Of the existing face recognition methods, the PCA methodis obtaining eigenvalue and eigenvector by using the disper-sion of the whole training image, then eigenvalues arrangedin descending order corresponding eigenvectors [1]. First,Turk, etc. [2] extracted noncorrelational features betweenobjects by PCA, and applied the neighborhood algorithmclassification method to face recognition.

Recognition has a process of comparing the feature vec-tor of training image and the feature vector of test imagestored in database. However, great illumination variation can

∗ Corresponding author. Tel.: +82 43 261 2483; fax: +82 43 271 8085.E-mail addresses: songyjorg@dreamwiz.com (Y.-J. Song),

mmlover@dreamwiz.com (Y.-G. Kim), udchang@naver.com(U.-D. Chang), hbkwon@kimpo.ac.kr (H.B. Kwon).

0031-3203/$30.00 � 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.doi:10.1016/j.patcog.2006.02.018

not facilitate division of classes as feature vectors obtainedfrom face images. That is because illumination variation iscentered on first three eigenfaces with the most informationin time of comparison of eigenvalues. Therefore, except forthe first 3 eigenfaces, dividing the classes of feature vectorsbecomes a little easier [3]. And histogram equalization tech-nique, when luminance value distribution is not uniform,standardizes it by regulating values artificially.

Early work in illumination invariant face recognition fo-cused on image representations that are mostly insensitiveto changes in illumination. Shashua and Riklin-Raviv [4]proposed a different illumination invariant image represen-tation, the quotient image.

The study has tried to raise the PCA face recognition per-formance by mirror image made out of luminance differencebetween the left and right against the front face shaded by il-lumination. So as to evaluate the recognition performance ofthe proposed method, the study compared PCA, PCA withfirst 3 eigenfaces excluded, histogram equalization, and theproposed method by using the Yale face database.

The structure of the study is as following: Section 2explains the proposed method using symmetry; Section 3is on the face database used in the experiment, the

Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542–1545 1543

experimental method, and the results; finally, Section 4comes to a conclusion.

2. The proposed method

2.1. System architecture

The study has aimed to improve misrecognition of a faceimage shaded by illumination. In order to reduce the effect ofillumination, the pre-processing of face recognition producesmirror image from shaded test image. That is to say, based onthe face center of test image, first the luminance differenceshould be obtained between the right and left face image,then the brighter face of the two be mirrored, and its mirrorimage be produced. The mirror image produced by the pre-processing is used for the input image of face recognitionsystem which the existing PCA method has been applied to.

The whole architecture of the face recognition systemincluding the proposed pre-processing system is as inFig. 1. Above all, on the assumption that as for input imagethe front face has symmetry, the left and right face weredivided on the horizontal-axis center of input image.

Shade variation by illumination requires the block lumi-nance difference between the left and right face; when it is

Input image

Make left/right halfface by symmetry

Luminance differencebetween left and right face

> Threshold

No

PCA

Yes

Make mirrorimage

Euclidean distancemeasure

Face recognition

Fig. 1. System block diagram.

not bigger than threshold value (experimental value = 100),the PCA method must be applied to face recognition; how-ever, when it is bigger, the mirror image be produced andPCA be applied to it. Also, by using Euclidean distance andcomparing the similarities, face with the biggest similaritygets recognized as that of the same person.

2.2. Comparison of the left and right luminance value

The left and right face, in case of little variation, hassymmetry. So if illumination is uniformly applied to a frontface, the left and right, with the nose a center, has almostthe same luminance value. But when there is some posevariation, the luminance value difference of a specific partlike the eyes and the mouth can be wrongly recognized asthat of the whole face.

As a method overcoming some pose variation, the averageluminance value of a block (3 pixels × 3 pixels) can beused for comparison of the left and right face luminancevalue. When the luminance values of the left IL and the rightface part IR are compared, the binary face image IB , newlycomposed by giving 1 to the big part and 0 to the small partin case of more than 30 (experiment value) of differencebetween the average block values, is recomposed by Eq. (1).{

IB = 1, |IL − IR|�30,

IB = 30 otherwise.(1)

When, of the recomposed left and right face, more than100 is the number of image (about 38×31) 1 recomposed by3×3 blocks, that is regarded as an image with big luminancedifference in the left and right of face image. It means thecase when shade by illumination occupies about more than10% of the whole face. The shade comparison image (re-composed by 1178 pixels) are composed of 0 and 1: 0 meanslittle shade difference between the left and right; 1 meanssome shade difference. As the number of 1 can tell whatthe left–right shade difference is, about 10% of 1178 pixels(100) is used as threshold value producing mirror image.

Big luminance difference between the left and right facedeteriorates recognition performance when PCA is used.That is because, by using face symmetry, the bright face ismirrored on the part of the dark face and a new face imageis produced which compensates for the effect of shade.

3. Experimental results and analysis

The study made the Yale face database with much shadedifference into regularized gray image of 112 × 92 size;divided training image and test image by the hold-and-outmode. Whether to recognize training image from test im-age could measure similarity between feature vectors by us-ing Euclidean distance. Simulation experimented PCA, PCAwith first 3 eigenfaces excluded, histogram equalization, andthe proposed method. Fig. 2 shows the change of recognitionperformance according to increasing PCA dimension-levelin each of all the methods.

1544 Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542–1545

100

908070605040302010

01 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

PCA

PCA with 1st 3eigenfaces excludedHistogram equlization

proposed method

PCA (dimension)

Rec

ogni

tion

rate

(%

)

Fig. 2. The simulation result.

The result of application of PCA to original image keeps92.2% in the 10th dimension; the result of PCA with first3 eigenfaces excluded shows high recognition performanceof 95.6% in the 18th dimension; also, the result of PCA ap-plication after histogram equalization was recognition per-formance of 95.6% in the 17th dimension, the same as thatof PCA with first 3 eigenfaces excluded, in the 18th. How-ever, it showed low recognition performance under the 10thdimension. The result of PCA application to test image, re-composed of mirror image according to luminance differ-ence (the proposed method), has kept high recognition per-formance of 98.9% in and over the 10th dimension. Besides,it showed higher performance even under the 10th dimen-sion than any other method.

4. Conclusion

The study has proposed an efficient pre-processingmethod of a shaded face by the symmetry of the front face.In order to reduce the effect of illumination, the study usedthe luminance difference between the left and right face,with test image as an object and the center-line of a face asa boundary. The study judged the existence of shade fromthis; when there was much shade difference between thetwo, it produced mirror image and applied PCA to it.

When there was great shade by the effect of the left andright illumination, the proposed method had very efficientperformance; has about 4% higher performance than theexisting PCA, PCA with first 3 eigenfaces excluded, andhistogram equalization.

The future tasks are how much variation can be overcomeby applying the proposed method to image with pose varia-tion, and that the proposed method should be applied to an-other database (with partial shade) under illumination fromvarious angles.

5. Summary

This paper has proposed an efficient shaded-face pre-processing technique using front-face symmetry. The

existing face recognition PCA technique has a shortcom-ing of making illumination variation lower the recognitionperformance of a shaded face. The study has aimed to im-prove the performance by using the symmetry of the leftand right face. In order to reduce the effect of illumina-tion, the pre-processing of face recognition produces mir-ror image from shaded test image. That is to say, basedon the face center of test image, first the luminance dif-ference should be obtained between the right and left faceimage, then the brighter face of the two be mirrored, andits mirror image be produced. The mirror image producedby the pre-processing is used for the input image of facerecognition system which the existing PCA method has beenapplied to.

The proposed method is tested on the Yale face databasewith left/right shadows. The experimental methods for thisare as following: the existing PCA, PCA with first threeeigenfaces excluded, histogram equalization and the pro-posed method.

The result of application of PCA to original image keeps92.2% in the 10th dimension; the result of PCA with firstthree eigenfaces excluded shows high recognition perfor-mance of 95.6% in the 18th dimension; also, the result ofPCA application after histogram equalization was recogni-tion performance of 95.6% in the 17th dimension, the sameas that of PCA with first three eigenfaces excluded, in the18th. However, it showed low recognition performance un-der the 10th dimension.

The result of PCA application to test image, recom-posed of mirror image according to luminance difference(the proposed method), has kept high recognition perfor-mance of 98.9% in and over the 10th dimension. Besides,it showed higher performance even under the 10th dimen-sion than any other method. As the result, it was shownthat the proposed method has a rather excellent recognitionperformance.

Acknowledgments

This work was supported by the Regional ResearchCenters Program of the Ministry of Education & HumanResources Development in Korea.

References

[1] A.M. Martinez, A.C. Kak, PCA versus LDA, IEEE Trans. PatternAnal. Mach. Intell. 29 (2) (2001) 228–233.

[2] M. Turk, A. Pentland, Eigenfaces for recognition, J. CognitiveNeurosci. 3 (1) (1991) 71–86.

[3] P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs.fisherfaces: recognition using class specific linear projection, in:European Conference on Computer Vision, 1996, pp. 45–58.

[4] A. Shashua, T. Riklin-Raviv, The quotient image: class-based re-rendering and recognition with varying illumination conditions, IEEETrans. Pattern Anal. Mach. Intell. 23 (2) (2001).

Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542–1545 1545

About the Author—YOUNG-JUN SONG received the M.S. degree in computer and communication engineering from Chungbuk National University in1996. He received the Ph.D. degree on face and image recognition. He currently works at Chungbuk National University, South Korea. His researchinterests also include face recognition, computer vision, image processing.

About the Author—YOUNG-GIL KIM received the M.S. degree in computer and communication engineering from Chungbuk National University,Korea in 2001. He is currently working towards Ph.D. degree on face recognition and image segmentation. His research interests also include patternrecognition, computer vision, nonparametric analysis.

About the Author—UN-DONG CHANG received the M.S. degree in computer and communication engineering from Chungbuk National University,Korea in 2002. He is currently working towards Ph.D. degree on pattern recognition, face recognition.

About the Author—HEAK-BONG KWON received the M.S. degree in computer and communication engineering from Hoseo University, Korea in1992. He received the Ph.D. degree on face and image recognition from Chungbuk National University, Korea in 2001. He currently works at KimpoCollege, South Korea. His research interests also include image processing, computer vision, digital signal processing.