1 rnnnrr11 0000086932

AGE INVARIANT FACE RECOGNITION SYSTEM

USING AUTO MA TED VORONOI DIAGRAM SEGMENTATION

NIK NURUL AIN BINTI NIK SUKI

A project report submitted in partial fulfillment of the

requirements for the award of the degree of

Master of Computer Science (Information Security)

Advanced lnformaticSchool

Faculty of Computer Science and Information Systems

Universiti Teknologi Malaysia

JANUARY 2013 U1\iVERSITI MALAYSIA PAHANG

No. Perolehan No. Panggllan

086832

111

ABSTRACT

One of the challenges in automatic face recognition is to achieve sequential

face invariant. This is a challenging task because the human face undergoes many

changes as a person grows older. In this study we will be focusing on age invariant

features of a human face. The goal of this study is to investigate the face age invariant

features that can be used for face matching, secondly is to come out with a prototype

of matching scheme that is robust to the changes of facial aging and finally to

evaluate the proposed prototype with the other similar prototype. The proposed

approach is based on automated image segmentation using Voronoi Diagram (VD)

and Delaunay Triangulations (DT). Later from the detected face region, the eyes will

be detected using template matching together with DT. The outcomes, which are list

of five coordinates, will be used to calculate interest distance in human faces. Later

ratios between those distances are formulated. Difference vector will be use in the

proposed method in order to perform face recognition steps. Datasets used for this

research is selected images from FG-NET Aging Database and BioID Face Database,

which is widely being used for image based face aging analysis; consist of 15 sample

images taken from 5 different person. The selection is based on the project scopes

and difference ages. The result shows that 11 images are successfully recognized. It

shows an increase to 73.34% compared to other recent methods.

lV

ABSTRAK

Salah satu cabaran pemadanan muka secara automatik adalah untuk mencapai

turutan muka yang tidak berubah mengikut masa. Faktor utama yang menyumbang

kepada permasalahan ini adalah kerana wajah manusia mengalami banyak perubahan

sepanjang proses peningkatan usia. Kajian ini akan member tumpuan kepada ciri-ciri

pada wajah manusia yang tidak berubah mengikut umur. Objektif kajian ini adalah

untuk menyiasat ciri-ciri pada wajah yang tidak berubah mengikut umur dan boleh

digunakan untuk memadankan muka, objektif kedua adalah untuk membina prototaip

system pemadanan wajah yang teguh kepada perubahan penuaan wajah dan akhir

sekali adalah untuk menilai prototaip yang dicadangkan dengan prototaip lain yang

hampir serupa. Kajian yang dicadangkan mengunakan segmentasi imej secara

automatik menggunakan Voronoi Diagram (VD) dan Delaunay Triangulations (DT) .

Kemudian daripada imej wajah yang dikesan, mata akan dikesan menggunakan

Template Matching bersama-sama dengan DT. Hasilnya, iaitu lima koordinat, akan

digunakan untuk mengira jarak cirri-ciri penting dalam muka manusia. Kemudian

antara jarak-jarak ini akan dikira. Difference Vector akan digunakan dalam kaedah

yang dicadangkan untuk melaksanakan langkah-langkah pengiktirafan muka. Dataset

yang digunakan di dalam kajian ini merupakan imej-imej yang dipilih dari Pangkalan

Data Penuaan FG-NET dan Pangkalan Data Wajah BioID, yang secara meluas

diterima pakai bagi analisis imej muka berasaskan penuaan. 15 imej sampel yang

diambil daripada 5 orang yang berbeza. Pemilihan adalah berdasarkan skop projek

dan perbezaan umur. Hasilnya menunjukkan bahawa 11 imej berjaya dicam. Ia

menunjukkan peningkatan kepada 73,34% berbanding dengan kaedah lain baru-baru

llll.

v

TABLE OF CONTENTS

CHAPTER TITLE PAGE

DECLARATION I

DEDICATION 11

ABSTRACT lll

ABSTRAK IV

TABLE OF CONTENTS v

LIST OF TABLES Vlll

LIST OF FIGURES IX

LIST OF ABBREVIATIONS XI

1 INTRODUCTION

1.1 Background Of The Problem 2

1.2 Problem Statement 4

1.3 Project Objectives 4

1.4 Research Questions 5

1.5 Project Aims 5

1.6 Scopes Of The Study 6

2 LITERATURE REVIEW 7

2.1 Face Variations By Age 8

2.2 Image Segmentation 13

Vl

2.2.1 Based on RGB Information 14

2.2.2 Based on Boundary 18

2.2.3 Based on Thresholding 19

2.2.4 Based on Eigen Faces 20

2.2.5 Based on Voronoi Diagram 22

2.3 Features Extraction 23

2.3.1 Chain Codes 23

2.3.2 Template Matching 26

2.4 Distance Metrics 27

2.4.1 Types of Distance 27

2.5 Face Recognition System 30

2.6 Summary 36

3 METHODOLGY 37

3.1 Project Deliverables 38

3.2 Research Operational Framework 39

3.3 Information Gathering and Problem Formulation 41

3.4 Requirements Analysis 41

3.5 Software Design 42

3.5.l Pre-Processing 42

3.5.2 Features Extraction 44

3.5.3 Computation 45

3.5.4 Testing 47

3.6 Implementation 48

3.7 Report Writing 49

3.8 Summary 50

4 DESIGN AND IMPLEMENTATION 51

4.1 Proposed Prototype 52

4.2 Pre-Processing Phase 53

4.2.1 Image Segmentation Using Voronoi

Tessellations

4.2.2 Selecting Of Probable Face Region

4.3 Features Extraction

4.3.1 Detecting Eyes

4.3.2 Detecting Nose And Mouth

4.4 Face Recognition

4.5 Testing

5 RESULT AND ANALYSIS

5.1 Experimental Result

5.1.l Image Segmentation

5.1.2 Data Calculation

5.1.3 Face Recognition

5.2 Comparison With Other Similar Methods

6 CONCLUSION AND CONTRIBUTION

6.1

6.2

6.3

REFERENCES

APPENDICES

Summary Of The Study

Contribution

Future Works

55

58

61

62

64

67

70

71

72

72

72

76

81

82

83

83

85

86

90

vu

Vlll

LIST OF TABLES

TABLE NO. TITLE PAGE

2.1 Descriptions on human face variations across the age 8

(MAB, 2005)

2.2 Descriptions on four ( 4) key factors that compromise 35

face recognition accuracy.

3.1 Project deliverables throughout the entire process 38

3.2 List of primary global features on human face. 45

3.3 List of the hardware requirements for the system 48

3.4 List of the software requirements for the system 49

4.1 Extracted features with its coordinates 68

5.1 The coordinates (x, y) of Features Extractions 72

5.2 Findings on Distance, Ratios and Standard Deviation 75

5.3 Differences between all tested Images 76

5.4 BiolD _ 0019.jpg test result 78

5.5 Comparison of the proposed method with other 81

methods.

6.1 Project Contributions 84

IX

LIST OF FIGURES

FIGURE NO. TITLE PAGE

1.1 Typical images with age differences. 3

2.1 Perception of wrinkles at different resolution (in pixels) 11

of a face image at age 50.

2.2 Simple Scenes for Image Segmentations 13

2.3 Illustrations of the HSI Color Space 15

2.4 Segmentation to a colored photo image 16

2.5 Segmentation to the HSI model 17

2.6 Canny edge detected image 18

2.7 Example of Global Thresholding Segmentation 20

2.8 Some example of eigenfaces computed from the ORL 21

dataset

2.9 Example ofDelaunay Triangulation 22

2.10 Direction numbers of a Chain Code 24

2.11 Chain Code Representation 25

2.12 Eyes Template 26

2.13 Manhattan geometry versus Euclidean distance 28

2.14 The reduction in error rate for state-of-the-art face 33

recognition algorithms

2.15 Face images of one subject are shown under various 34

circumstances

3.1 Research Operational Framework 40

x

3.2 The proposed block diagram for the face recognition 42

system.

3.4 Illustrations (marked in red color) of the expected 44

outcome from the Features Extraction Step.

3.5 The four ratios used 46

4.1 The main interface for the proposed prototype 52

4.2 The block diagram of the Pre-Processing phase 54

4.3 Delaunay Triangulation of I 0 dot patterns 56

4.4 Comparison of image segmentation using the K-Means 57

and Cheddad A. et al. (2008) method.

4.5 Applying Distance Transformation to separate the face 59

when it is merged with the background

4.6 Face repairing. 60

4.7 The Features Extraction phase 61

4.8 The pseudo code in detecting the eyes 63

4.9 Clustering concept based on Delaunay Triangulations 64

4.10 The proposed ellipse model 65

4.11 The eclipse model 66

4.12 The block diagram of the recognition phase 67

5.1 Extracted Features in BioID _0000.jpg 73

5.2 Difference Vector of tested images 77

5.3 Face Recognition Result 77

5.4 Correct/ False Detection 80

2D

3D

DT

EU

FEBFRGAC

FERET

FRVT 2002

FRVT 2006

GB

GOP

GWT

HDD

HP

HSI

ICAO

JPG/JPEG

LDA

MBE 2010

MFDA

MLBP

NIST

LIST OF ABBREVIATIONS

Two dimensional

Three dimensional

Delaunay Triangulations

Euclidean

Extraction Based Face Recognition, Gender And Age

Classification

The Facial Recognition Technology Database

Face Recognition Vendor Test 2002

Face Recognition Vendor Test 2006

Giga Byte

Gradient Oriented Pyramid

Gabor Wavelet Transform

Hardisk

Hewlett Packard

Hue Saturation Intensity value

International Civil Aviation Organization

Joint Photographic Experts Group

Linear Discriminant Analysis

Multiple Biometric Evaluation

Multi Features DiscriminatAnalysis

Multi scale local binary pattern

National Institute of Standards and Technology

XI

PCA

RFID

RGB

SFS

SIFT

SVM

UDP

VD

WLBP

Principle Component Analysis

Radio Frequency Identification

Red, Green and Blue

Shape From Shading

Scale Invariant Feature Transfer

Support Vector Machine

Unsupervised Discriminant Projection

Voronoi Diagram

Walsh-Hadamard transform encoded local binary

patterns

xii

CHAPTER I

INTRODUCTION

Personal verification and identification is an actively growing area of research

nowadays. Face, voice, lip movements, hand geometry, odor, gait, iris, retina,

fingerprint are the most commonly used authentication methods. These physiological

and behavioral characteristics are known as biometrics, contributed significant

advantages compared to the traditional authentication techniques (Ramesha et al.,

2010). The high demand of the progress in this field is due to the growing role of the

technology in the modem society. As a result, numbers of application appeared in

daily tasks such as online electronic commerce, automated factory machines and

building security system.

Face recognition is one of the biometrics approaches to identify individuals by

the features of the face. Research in this area has been conducted for more than 30

years but only become famous for the past 10 years (Li et al., 2011); therefore,

currently the face recognition technology is well advanced. Wide range of

applications has been using this approach such as security and surveillance system,

human computer interaction, image retrieval, criminal identification, etc. In general,

the approach will be using a pre-stored image database where later the face

2

recognition system will be able to identify or verify the person from the captured

image or another source of images to the one in the database. The age and gender of a

person are grouped by visual observation of still images where it is difficult for the

computer vision. Hence, there comes the need to discover the age and gender related

area for further studies. However, in this project, we will be focusing only on age

related issues.

1.1 BACKGROUND OF THE PROBLEM

Face recognition across the ages is a signi.fl~ant problem and has many

applications such as passport photo verification, image retrieval, surveillance (Ling et

al. , 2007), etc. This is a challenging task because the human face undergoes many

changes as a person grows older. Each person has different facial features in many

aspects, including facial texture (e.g. wrinkles), shape (e.g. weight gain), facial hair,

presence of glasses, and exposure to sunlight (Ling et al., 2010), that make us

different to each other. In addition, the image acquisition environments also have an

effect on the images captured.

The performance of face recognition systems cannot challenge with the

sequential changes over a period of time. Law enforcement agencies such as crime

and record bureau regularly require matching a probe image with the individuals in

the missing person database (Singh et al. , 2007), or a customs need to compare new

photo to the previous photo during the passport renewal process. In such applications,

there are many significant differences between the facial features of probe and gallery

image due to age variation. For example, ifthe age of the probe image is 15 years old

and the gallery image of the same person is 5 years old, an existing matching system

3

is ineffective and might not produce the expected result. Figure 1 shows several

typical images with different age gaps.

Figure 1.1 Typical images with age differences. Source images are from the FG-

NET Aging Database, http://www.cut.ac.cy/members/fgnet.

One way to overcome this challenge is to regularly update the database with

recent images or templates (Singh et al., 2007). However, this method is not suitable

for applications such as border control, homeland security and missing person

identification. To address this issue, researchers have proposed several techniques

which later we will discuss in Chapter 2.

4

1.2 PROBLEMSTATEMENT

Determining age-invariant features of a human face is a difficult task since

human face facing significant changes across time. Still this task is necessary in many

applications, especially those that require checking whether the same person have

issued multiple government documents (e.g. passport, identification card and driving

license) that include facial images (Li et al., 2011 ).

In this project, we will study on standard features or also being referred as

age-invariant features of a human face which later can be used as target points for

face recognition process. To recognize the face, we will come out with a prototype to

determine whether the images come from the same person, in whom an individual is

identified from a large gallery of individuals; in this project we will be using FG-NET

Aging and BioID face database as our dataset.

1.3 PROJECT OBJECTIVES

The objectives of the project are:

1. To investigate age-invariant features of a human face;

2. To develop a prototype for face recognition system based on the selected

features; and

3. To test and evaluate the accuracy of the proposed prototype by companng

with other similar face recognition prototypes.

5

1.4 RESEARCH QUESTIONS

The main questions this research motivates to answer are as follows:

1. What are the features of human face that are not changing across age?

2. How to develop a prototype of the face recognition system?

3. What is the effectiveness of the proposed prototype with other similar face

recognition prototype?

1.5 PROJECT AIMS

The aim of this study is to investigate the age-invariant features of a human

face and later develop a prototype of face recognition systems implementing the

selected features. The prototype will be tested using FG-NET Aging and BioID face

database as datasets in our experiments.

1.6 SCOPES OF THE STUDY

The scopes for this study focus on:

I. Only three (3) age-invariant features will be implemented in the prototype;

2. Image format use for this project will be in Joint Photographic Expert Group

(.jpg and .jpeg) only;

3. Images sample will be a face image with frontal faces only; and

4. The prototype can dealt with color and grayscale images;

5. Sample face images will be taken from FG-NET Aging and BioID face

databases;

6. Age group of the sample images is 18 years old and above.

6

CHAPTER2

LITERATURE REVIEW

Human faces are varied with the age group; baby, children, adult and old age.

In this chapter, we will be discussing on the invariant features of human faces. The

need of automated face recognition has been important yet challenging problem. This

challenge can be due to range of variations such as pose, illumination, expression,

and aging. Among these variations, aging factors is now raised attention in the face

recognition community. Designing an age invariant face recognition are crucial in

many applications, especially those that require checking on the person verification

(e.g. passports, driver license) that include facial images. Despite numerous research

efforts has been conducted to study in this area for the past 30 years, most of the

works focus on age estimation and age simulation from the face variations. However

for the past 10 years, the face recognition based on age barrier seems to take

researchers' interest which later will be discussed in this chapter.

8

2.1 FACE VARIATIONS BY AGE

According to Marquardt Beauty Analysis, California, the most common

variations of human faces can be grouped into three groups which are age, sex and

geographical (ethnicity). In this project we will be focusing only on the age factor.

The analysis also stated that the periods of ages can be randomly grouped into four

main categories which are babies, children, adults and old age. Table 2.1 explains

further on the variations of the face across the age. Just like every other aspect of

human existence the face also varies with age. To state that the face changes with age

is barely news.

Table 2.1: Descriptions on human face variations across the age (MAB, 2005)

Group Description

Major personal visual attractiveness.

Age group: I - 24 months.

Bigger eyes in proportion to its face.

Rounder face.

Plumper cheeks.

Larger head in portion to its face.

Baby Short, flat eyebrows.

Short, relatively small, turned up nose.

Table 2.1: Descriptions on human face variations across the age (MAB, 2005)

(continue)

Group

Less attractive.

Age group: 3 - 11 years

Freckles or acne.

Description

Flat eyebrows, fuller than the child.

Ears that appears oversized.

Nose generally small, short and wide.

Children Relatively flatter and less defined cheeks

Adult

Attractive peak.

Age group: 14 to 24 years.

Static facial form.

9

10

Table 2.1: Descriptions on hwnan face variations across the age (MAB, 2005)

(continue)

Group Description

Age group: 25 years and above

The face begins to gradually matures/ aging

The cheeks sag inferiorly resulting the appearance of jowls

The comers of the mouth move inferiorly resulting in a slight frown look

The tissue around the eyes sag inferiorly

The eyelids, upper and lower, themselves sag inferiorly

The tissue of the forehead drifts inferiorly, creating wrinkles and dropping the

eyebrows downward and giving them a flatter appearance

Old Age The nose may lengthen and move the tip inferiorly

The nose may develop a small to pronounced dorsal hwnp

The tip. of the nose may enlarge and become bulbous

Generalized wrinkling of the face may occur

11

Human face undergoes significant changes as the person grows older. The

facial features of human are different compared to each other. Singh et al. in their

study divided the human age into three age groups for testing purposes which are 1-

18 years old, 19-40 years old and beyond 41 years old. They strongly believe that

face development depends on the ages of a person. For example, development in

muscles and bone structure caused major changes in the face at the age of 1-18 years

old. From 19-40 years old, the growth rate is comparatively lower, whereas after 40

years old, wrinkles and skin loosening cause major change in facial features and

appearance (Singh et al., 2007).

(a) (b) (c) (d)

Figure 2.1 Perception of wrinkles at different resolution (Ling et al., 2007)

Figure 2.1 illustrate the perception of wrinkles at different resolution (in

pixels) of a face image at age 50.- Image (a)is the original image with height = 347,

image (b) with height = 207, image (c) with height = 160, image (d) with height= 96,

and image (e) with height = 72 pixels. From Figure 2.1, we can clearly see the

wrinkles seem to be disappearing when the resolutions decreases The effects of

wrinkles to the face normalcy are reduced by the image resolution and the natural

12

distribution of the wrinkles itself. Kwon (1999), believe that wrinkles are hardly

perceptible for images with low resolution. The distribution of wrinkles on human

faces is known to be inidentical. For example, Boissieux et al. (2007), concluded

eight wrinkles maps from the qualitative data from L'Oreal. From the study, wrinkles

appear frequently on the forehead and cheeks, which are not important for face

recognition. This means that most regions with heavy wrinkles can be ignored for

face recognition.

Earlier study of facial growth conducted by Pittenger and Shaw (1975), has

shown that face profile (e.g. boundary shapes, eye locations) are relatively stable at

the age of 18 years old onward. S. Akazaki et al. (2003), stated that wrinkles are one

of the most important factors for the visual perception of age whereas the width and

depth of wrinkles grow roughly linearly with age.

Considering that entire face with the high structural complexity easily changes

across time, Xu et al. (2011) decided to only use periocular region as one of the age

invariant features for face recognition. This is because the periocular region changes

practically smaller over time due to the shape and location of the eyes remain largely

unchanged compared to mouth, nose, chin, cheek, etc., which are more vulnerable to

changes given by a loosened skin (Masoro and Austad, 2010). These clarify why

most of the researchers select eyes location as a reference point for face recognition.

13

2.2 IMAGE SEGMENTATION

Image segmentation can be described as a process of partitioning a digital

image into multiple segments. The goal of image segmentation is to simplify and/ or

change the representation of an image into something that is more meaningful and

easier to analyze (Shapiro G. and Stockman C., 2001). Image segmentation is

normally used to locate objects and boundaries (e.g. lines, curves, etc) in images. The

result from this process is a set of segments extracted from the entire image which

later can be used for object recognition, boundary estimation, image editing, image

database lookup etc (Jepson A. D. and Fleet D. J., 2007). While in this research,

image segmentation is mainly implemented to detect the face region for the ''face

localization" step.

Figure 2.2

Original Image Segmentation (by SMC)

Simple Scenes for Image Segmentations (Jepson A. D. and Fleet D. J. ,

2007)

Figure 2.2 shows a simple example of image segmentation using Spectral

Embedding and Min-Cut (SMC) algorithm. It clearly describes that segmentations of