A Study on Face Recognition by Extracting CT-based Color Values

Jae-gu Song1,2, Sungmo Jung1, Yohwan So1, Seoksoo Kim1

1Department of Multimedia, 2 School of Computing & Information System 1Hannam University, 2Tasmania University

1133 Ojeong-dong, Daedeok-gu, Daejeon-city, 2Churchill Avenue Sandy Bay TAS 7005. 1KOREA, 2AUSTRALIA

bhas9@paran.com, sungmoj@gmail.com, arto@hnu.kr, sskim0123@naver.com

Abstract: - This is basic research for extracting face by analyzing color values, and the purpose of this study is to secure expression data with more clear color values extracted by the light sources. For this purpose, how to apply CT algorithm that processes color values efficiently by the light to the existing color value analysis algorithm (RGB. YcbCr, HIS), was suggested. While designing the order of color-based facial expression recognition tracking in 6 steps and applying this to the program, this study assured the efficiency of the detection area. The result shows that the facial region becomes relatively distinct when illumination is focused exclusively on the facial region, than under the global illumination condition. Key-Words: - Face recognition, Color recognition, Face detection, Visual effect, Analysis algorithm, Video tracking

1 Introduction As the face recognition has proven to be useful in the fields from individual identification in security system to graphics, different types of related researches are being carried out for applications. The application researches for face recognition requires technologies to ensure a constant recognition rate even if environment components and conditions change for its commercialization. Most of the researches that have been done, improved the recognition rate by deleting additional information not required for face recognition such as illumination value, or on the contrary, by detecting feature point only shown on the face to trace particular values including face color [1][2]. These researches have improved recognition rate by extracting feature points of the user’s face. On the other hand, this study conducted basic research for facial expression recognition, while adding color value extraction algorithm to bottom-up feature-based characteristics, which enables more detailed analysis of facial expression by perceiving location values centering around a particular point and analyzing characteristics of the area using color values. This study investigates the face recognition technique in Chapter 2 Basic Research, and explains how to analyze color values in Chapter 3. Chapter 4 explains facial expression recognition technique using CT-based color value analysis and Chapter 5 discusses the conclusion and future researches.

2 Basic Research – Face Detection Technique The face is recognized more accurately when its frontal image is secured and 2D image is analyzed [4,5]. However in practical applications, the environment applied to the face recognition includes various information which interrupt the recognition, such as background image information, side image analysis and the light influence. Especially, background image, illumination and shadow and other additional information in the photographing environment hamper accurate face recognition task. Face detection is on of the visual tasks which humans can do effortlessly. However, in computer vision terms, this task is not easy. A general statement of the problem can be defined as follows: Given a still of video image, detect and localize an unknown number of faces. The solution to the problem involves segmentation, extraction, and verification of faces and possibly facial features form an uncontrolled back ground. As a visual frontend processor, a face detection system should also be able to achieve the task regardless of illumination, orientation, and camera distance[3]. Typical face recognition methods that have been used are as followings; 2.1 Knowledge-based Top-Down Method As a method of developing face detection algorithm according to the knowledge-based rule, a candidate face image is detected from the target image by any pre-defined rule. The detection accuracy might be lowered by the standard of the rules.

Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing

ISBN: 978-960-474-262-2 132

2.2 Bottom-Up Feature-Based Method As a method of detecting face using not changing features of the face, it detects specific feature points in spite of the changes of angle and neighboring background. Detection using features including eye, nose and mouth; detection using particular texture of the face and; detection using face color are typical examples of the method. 2.3 Template Matching Method This is a method of attempting to detect by measuring the correlation degree with the face image input using the standard pattern information calculated by a specific function. Its implementation is easy but it does not efficiently adapt to the environmental change. 2.4 Appearance-Based The appearance-based face detection method is a method of detecting face through a statistical analysis of the fact image features learned from different other face images. Typical researches include Eigenfaces, LDA, Neural networks, support vector machine and hidden Markov model. Current development in this area is moving toward more generalized vision applications such as face recognition and video coding techniques[3]. Many of the current face recognition techniques assume the availability of frontal faces of similar sizers[4,5] Each research has different features in its approach, standard to be identified and improvement method.

3 Basic Research – A Study of Color Value Recognition Color analysis based methods have been used and proven to be an effective feature in the application of face detection and tracking. Although different people have different skin color, several studies have that the major difference lays largely between their intensity rather than their chrominance[6].

3.1 Color centroids segmentaton model[6] Different methods are introduced that apply widely-known methods to recognize color values in image processing. The approach of this study is based on "Face detectionand tracking in clolor images using color centroids segmentation" (Zhang, Q.) technique. The procedure of color centroids segmentaton model is as followings; Color triangle is created by following steps: Step 1 : create a standard 2-D polar coordinate system; Step 2 : create three color vectors to reflect R, G and B, the range of them is [0, 255] and alternation reciprocally as follows:

    90°,    ∈ 0, 255

    90°,    ∈ 0, 255  

    90°,    ∈ 0, 255

Step 3 : connect the three apexes. After above processes, color triangle is created by follwing steps:

Fig. 1 Color triangle model[6] 3.2 Face Detection Technique using RGB Color [7] In general, an algorithm that can recognize the most simple skin color can be created by analyzing RGB values by pixel, which is excellent in sensing various colors by the light. The pixels for skin region can be detected using a normalized color histogram, and can be further normalized for changes in intensity on dividing by luminance[7]. 3.3 Face Detection Technique using YCbCr Color YCbCr Color space algorithm detects pixel values of a limited region by extracting Cb and Cr values and analyzing their similarity [8]. YcbCr Color space can detect any skin color of a person. The thresholds be chose as C , and s C , ,a pixel is classified to have skin tone if the values C , fall within the thresholds. The skin color distribution gives the face portion in the color image[9]. 3.4 Face Detection Technique using HSI Color HIS color technique is very similar to YCbCr Color technique, but it uses hue(H) value and saturation (S) value. The threshold be chose as H , and H , , and a pixel is classified to have skin tone if the values

Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing

ISBN: 978-960-474-262-2 133

H, S fall with the threshold and this distribution gives the localised face image. 3.5 Color Value Recognition Technique using the Light Colors in the image are affected by the light. This means that values from the illumination and reflection characteristics should be considered and if color values affected by the light can be calculated, color will be classified more clearly. This study was conducted on the basis of the thesis, "Face detection and tracking in video sequences using the modified census transformation" written by Christian Ku¨blbeck. Generally in an image, brightness, illumination, reflection characteristic and each location value are defiend as I X , L X , R X , and x x, y respectively. Camera influence can be modeled by a gain factor and a bias term , which are assumed to be constant on the image plane. Thus a simple image formation model is

I x  gL x R x  b (1) In the formula (1), R X value can be derived based on the illumination value, L X [10]. Besides, on the assumption that the illumination value is fixed, local structure influences the reflection characteristics, which means reflection characteristic only influences the image without affected by the illumination value. If fluorescent color which is sensitive to the light reflection is applied to the face using the reflection characteristic of the light, the texture of the face can be recognized more specifically without affected by the illumination value. This requires a local calculation of the influence of the light, and the brightness values of pixels in a certain area can be compared based on the center pixel, by means of the reserach on Census Transform [11]. It is defined as an ordered set of comparisons of pixel intensities in a local neighborhood representing which pixels have lesser intensity than the center. In general the size of the local neighborhood is not restricted, but in this work we always assume a 3x3 surrounding as motivated in the last section. The formula defined by CT is as below;

⊗∈

, (2)

I(X) : Brightness of center fixel of the window I(Y): Brightness of neighorhood pixel I(X) < I(Y) means 0 or 1 ⊗ : A concatenation operator that connects structure values of neighborhood pixels

However, CT has a limitation that it cannot express 511, the number of all cases expected to occur within 3x3 pixel, but only expresses 256 practically. Thus, MCT was suggested to make up for the limitation of CT

Γ  ⨂ ∈ , (3)

: Mean brightness value of pixel within the window

MCT can express all 511 cases that CT could not express. In this study, CT algorithm was applied to secure facial expression data.

4 A Study on Facial Expression Recognition using CT-based Colors This chapter designs the order of facial expression recognition using fluorescent color. Step 1 : Video Tracking Step 2 : RGB Color Analysis Step 3 : YcbCr Color Analysis Step 4 : HSI Color Analysis Step 5 : Census Transformation (CT) Step 6 : Expression Recognition

Fig. 2 Color-based Facial Expression Recognition Tracking Flowchart The program used in this study was constructed using the following procedures. By applying each algorithm obtained through the basic research in consecutive order, facial region is extracted and data of facial surface can be secured. Especially, since CT reduces effects of

Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing

ISBN: 978-960-474-262-2 134

variain rofaciaimaganalycoloattemillumregiothe l

Fig.

Fig. Colowas with

ations causedobustness toal surface coge obtained fyses. The re

or values are mpting to dminated by loon color distlocal facial re

3 Extraction

4 Extraction

or of the surchanged us

h the calcul

d by camera’ outliers nea

olor is morefrom RGB, Yesults of imaas in the foldetect face ocal light soutinct from thegion.

n using face c

n using face c

face shown sing color clated results

s gain and biar depth-disc

e precisely dYcbCr Colorage extractiollowing figur

color out urces, and Fie light sourc

color on the g

color on the

to be part ofclassification applied, th

ias and increacontinuities,

distinct from r and HSI Con by analyzres. Fig 3 sh

of the img 4 shows fa

ces that focus

global light f

local light fi

f a facial regn algorithm he image c

ases the the

olor zing ows

mage acial s on

field

eld

gion and olor

extextis dwhthe the

Figsou

Figsou Whthe regpre

5 Thithe anait astudreglighligh

racted was raction resuldisplayed as ere the facialimage detecface detectio

g. 5 Histogramurces

g. 6 Histograurces

hen features color area

gion, the histecisely than th

Conclusiis study intro

image accoalysis procedapplied CT ady result sh

gion improveht, than the ht.

converted inlts using face

Fig 5. The l region can bcted on the loon area is the

m of the ima

am of the ima

of the face cwhich does

togram in Fighe histogram

ion oduced algorording to R

dures and extralgorithm tohowed that tes more if thcase the wh

nto a histoge color on thedotted-line bbe detected. Oocal light fiele dotted-line

age illuminate

age illumina

color are ananot corresp

g 6 detects tm in Fig 5.

rithms that clRGB, YcbCrract face colo control lighthe ability t

he face only hole image i

gram graph. e global light box is a partOn the other ld is as Fig 6

e box.

ed by global

ated by local

alyzed, excluond to the fthe features

lassify regionr and HSI or values. Beht influence.to classify fis exposed tis exposed to

The t field tition hand,

6, and

light

light

uding facial more

ns on color

esides, . The facial o the o the

Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing

ISBN: 978-960-474-262-2 135

This study will be used as a basic research for securing data of facial expression for image photographing project, and is expected to be extended to the researches for securing facial expression information by the further introduction of advanced camera and multi-camera. Acknowledgement This research is supported by Ministry of Culture, Sports and Tourism(MCST) and Korea Creative Content Agency(KOCCA) in the Culture Technology(CT) Research & Development Program 2010. Corresponding author Seoksoo Kim(sskim0123@naver.com) References: [1] HSU, R.-L, ABDEL-MOTTALEB, M., AND JAIN,

A. K. 2002. Face detection in color images. IEEE Trans. Pattern Analysis and Machine Intelligence 24, 5, pp.696–706, 2002.

[2] W. Chen, M. J. Er, and S. Wu, Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain, IEEE Trans. on System, Man and Cybernetics-Part B: Cybernetics, vol.36, no.2, pp.458-466, Apr. 2006.

[3] B.K.L. Erik Hjelmas, Face Detection: A Survey, Computer Vision and Image Understanding, vol.3, no.3, pp.236-274, Sept. 2001.

[4] A Samal and P. A. Iyengar, Automatic recognition and analysis of human faces and facial expressions: a survey, Pattern Recog. 25, pp.65–77, 1992.

[5] R. Chellappa, C. L.Wilson, and S. Sirohey. Human and machine recognition of faces: A survey, Proc. IEEE 83, 5, 1995.

[6] Qieshi Zhang and Jun Zhang, RGB Color Analysis for Face Detection, Advances in Computer Science and IT, pp.119-126, December 2009

[7] Crowley, J. L. and Coutaz, J., Vision for Man Machine Interaction, Robotics and Autonomous Systems, Vol.19, pp.347-358, 1997.

[8] Cahi, D. and Ngan, K. N., “Face Segmentation Using Skin-Color Map in Videophone Applications,” IEEE Transaction on Circuit and Systems for Video Technology, Vol.9, pp. 551-564 1999.

[9] Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh, A Robust Skin Color Based Face Detection Algorithm. Tamkang Journal of Science and Engineering, Vol.6, No.4, pp.227-234, 2003.

[10] C. K¨ublbeck and A. Ernst. Face detection and tracking in video sequences using the modified census transformation. In Image and Vision Computing, 24 (6):564–572, 2006

[11] Ramin Zabih, John Woodfill, A non-parametric approach to visual correspondence, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996.

Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing

ISBN: 978-960-474-262-2 136