70
A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL by Dmitri Warren De Klerk A mini-thesis submitted in partial fulfillment of the requirements for the degree of Bachelor of Science (Honours) in Computer Science University of the Western Cape Supervisor: Mr. J. Connan November 2009

A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL by

Embed Size (px)

Citation preview

A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL

by

Dmitri Warren De Klerk

A mini-thesis submitted in partial fulfillment of the requirements for the degree of

Bachelor of Science (Honours) in Computer Science

University of the Western Cape

Supervisor Mr J Connan

November 2009

ii

University of Western Cape

Abstract

A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL

by Dmitri Warren De Klerk

Supervisor Mr J Connan Department of Computer Science

In this project we develop a face detection system which is used for access control The face detection system will accurately determine the locations and sizes of all possible human faces The faces are then scaled to a recognizable size and passed to a face recognition system implemented by Desmond Eustin van Wyk [1] that can accurately determine the identity of a person and decide whether or not to grant them access to a facility

iii

TABLE OF CONTENTS

Table of Contents iii

declaration v

keywords vi

Acknowledgments vii

Glossary viii

CHAPTER 1 10 11 - Users view of the problem 10 12 - Problem domain 11 13 - Complete description of the problem 11 14 - Expectations from a system 12

CHAPTER 2 13 21 - Users requirements interpretation 13 22 - Existing solutions 14 23 - Suggested system 14

CHAPTER 3 16 31 ndash Complete user interface 16 311 - How the user interface behaves 19 312 - The Add User Dialog 20 313 - The Confirm User Delete dialog 22 314 - The File menu 23 315 - The Face Detection menu 24 316 ndash Face Detection Settings dialog 25 317 ndash The Help menu 26 318 ndash The User Guide dialog 27 319 ndash The About dialog 28

CHAPTER 4 30 Object Orientated Analysis (OOA) 30

41 - Data dictionary 30 42 - Class diagrams 33 43 - Relationship between objects 40

CHAPTER 5 41 Object Orientated Analysis (OOD) 41

51 ndash Inner details of class attributes and methods 41

CHAPTER 6 48

CHAPTER 7 49

iv

CHAPTER 8 51 81 - System requirements 51 82 - The Face Recognition System project directory 52 83 - Running the Access Control system 54 84 ndash Complete user interface 54 841 - How the user interface behaves 57 841 - The Add User Dialog 58 842 - The Confirm User Delete dialog 61 843 - The File menu 62 844 - The Face Detection menu 63 845 ndash Face Detection Settings dialog 64 846 ndash The Help menu 65 847 ndash The User Guide dialog 66 848 ndash The About dialog 67

Chapter 9 69 CONCLUSION 69

v

DECLARATION

I declare that A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL is

my own work that it has not been submitted for any degree or examination in any other

university and that all sources I have used or quoted have been indicated and

acknowledged by complete references

Dmitri Warren De Klerk November 2009

Signature helliphelliphelliphelliphelliphelliphelliphelliphelliphelliphellip

vi

KEYWORDS

Haar Features

Integral Image

Cascade

AdaBoost

Boosting

Classifier

Weak Classifier

Strong Classifier

vii

ACKNOWLEDGMENTS

I thank the Almighty God for itrsquos through his strength I can do all things through Christ

who strengthens me Thanks to my supervisor Mr J Connan for his patience support

and guidance and his input in this project Thanks to my family and girlfriend who has

supported and motivated me

Thanks to the staff members of the Department of Computer Science and my classmates

for their kind assistance during the year

I also would like to acknowledge the work done by Desmond Eustin van Wyk His

documentation proved extremely useful in documenting my work

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

ii

University of Western Cape

Abstract

A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL

by Dmitri Warren De Klerk

Supervisor Mr J Connan Department of Computer Science

In this project we develop a face detection system which is used for access control The face detection system will accurately determine the locations and sizes of all possible human faces The faces are then scaled to a recognizable size and passed to a face recognition system implemented by Desmond Eustin van Wyk [1] that can accurately determine the identity of a person and decide whether or not to grant them access to a facility

iii

TABLE OF CONTENTS

Table of Contents iii

declaration v

keywords vi

Acknowledgments vii

Glossary viii

CHAPTER 1 10 11 - Users view of the problem 10 12 - Problem domain 11 13 - Complete description of the problem 11 14 - Expectations from a system 12

CHAPTER 2 13 21 - Users requirements interpretation 13 22 - Existing solutions 14 23 - Suggested system 14

CHAPTER 3 16 31 ndash Complete user interface 16 311 - How the user interface behaves 19 312 - The Add User Dialog 20 313 - The Confirm User Delete dialog 22 314 - The File menu 23 315 - The Face Detection menu 24 316 ndash Face Detection Settings dialog 25 317 ndash The Help menu 26 318 ndash The User Guide dialog 27 319 ndash The About dialog 28

CHAPTER 4 30 Object Orientated Analysis (OOA) 30

41 - Data dictionary 30 42 - Class diagrams 33 43 - Relationship between objects 40

CHAPTER 5 41 Object Orientated Analysis (OOD) 41

51 ndash Inner details of class attributes and methods 41

CHAPTER 6 48

CHAPTER 7 49

iv

CHAPTER 8 51 81 - System requirements 51 82 - The Face Recognition System project directory 52 83 - Running the Access Control system 54 84 ndash Complete user interface 54 841 - How the user interface behaves 57 841 - The Add User Dialog 58 842 - The Confirm User Delete dialog 61 843 - The File menu 62 844 - The Face Detection menu 63 845 ndash Face Detection Settings dialog 64 846 ndash The Help menu 65 847 ndash The User Guide dialog 66 848 ndash The About dialog 67

Chapter 9 69 CONCLUSION 69

v

DECLARATION

I declare that A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL is

my own work that it has not been submitted for any degree or examination in any other

university and that all sources I have used or quoted have been indicated and

acknowledged by complete references

Dmitri Warren De Klerk November 2009

Signature helliphelliphelliphelliphelliphelliphelliphelliphelliphelliphellip

vi

KEYWORDS

Haar Features

Integral Image

Cascade

AdaBoost

Boosting

Classifier

Weak Classifier

Strong Classifier

vii

ACKNOWLEDGMENTS

I thank the Almighty God for itrsquos through his strength I can do all things through Christ

who strengthens me Thanks to my supervisor Mr J Connan for his patience support

and guidance and his input in this project Thanks to my family and girlfriend who has

supported and motivated me

Thanks to the staff members of the Department of Computer Science and my classmates

for their kind assistance during the year

I also would like to acknowledge the work done by Desmond Eustin van Wyk His

documentation proved extremely useful in documenting my work

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

iii

TABLE OF CONTENTS

Table of Contents iii

declaration v

keywords vi

Acknowledgments vii

Glossary viii

CHAPTER 1 10 11 - Users view of the problem 10 12 - Problem domain 11 13 - Complete description of the problem 11 14 - Expectations from a system 12

CHAPTER 2 13 21 - Users requirements interpretation 13 22 - Existing solutions 14 23 - Suggested system 14

CHAPTER 3 16 31 ndash Complete user interface 16 311 - How the user interface behaves 19 312 - The Add User Dialog 20 313 - The Confirm User Delete dialog 22 314 - The File menu 23 315 - The Face Detection menu 24 316 ndash Face Detection Settings dialog 25 317 ndash The Help menu 26 318 ndash The User Guide dialog 27 319 ndash The About dialog 28

CHAPTER 4 30 Object Orientated Analysis (OOA) 30

41 - Data dictionary 30 42 - Class diagrams 33 43 - Relationship between objects 40

CHAPTER 5 41 Object Orientated Analysis (OOD) 41

51 ndash Inner details of class attributes and methods 41

CHAPTER 6 48

CHAPTER 7 49

iv

CHAPTER 8 51 81 - System requirements 51 82 - The Face Recognition System project directory 52 83 - Running the Access Control system 54 84 ndash Complete user interface 54 841 - How the user interface behaves 57 841 - The Add User Dialog 58 842 - The Confirm User Delete dialog 61 843 - The File menu 62 844 - The Face Detection menu 63 845 ndash Face Detection Settings dialog 64 846 ndash The Help menu 65 847 ndash The User Guide dialog 66 848 ndash The About dialog 67

Chapter 9 69 CONCLUSION 69

v

DECLARATION

I declare that A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL is

my own work that it has not been submitted for any degree or examination in any other

university and that all sources I have used or quoted have been indicated and

acknowledged by complete references

Dmitri Warren De Klerk November 2009

Signature helliphelliphelliphelliphelliphelliphelliphelliphelliphelliphellip

vi

KEYWORDS

Haar Features

Integral Image

Cascade

AdaBoost

Boosting

Classifier

Weak Classifier

Strong Classifier

vii

ACKNOWLEDGMENTS

I thank the Almighty God for itrsquos through his strength I can do all things through Christ

who strengthens me Thanks to my supervisor Mr J Connan for his patience support

and guidance and his input in this project Thanks to my family and girlfriend who has

supported and motivated me

Thanks to the staff members of the Department of Computer Science and my classmates

for their kind assistance during the year

I also would like to acknowledge the work done by Desmond Eustin van Wyk His

documentation proved extremely useful in documenting my work

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

iv

CHAPTER 8 51 81 - System requirements 51 82 - The Face Recognition System project directory 52 83 - Running the Access Control system 54 84 ndash Complete user interface 54 841 - How the user interface behaves 57 841 - The Add User Dialog 58 842 - The Confirm User Delete dialog 61 843 - The File menu 62 844 - The Face Detection menu 63 845 ndash Face Detection Settings dialog 64 846 ndash The Help menu 65 847 ndash The User Guide dialog 66 848 ndash The About dialog 67

Chapter 9 69 CONCLUSION 69

v

DECLARATION

I declare that A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL is

my own work that it has not been submitted for any degree or examination in any other

university and that all sources I have used or quoted have been indicated and

acknowledged by complete references

Dmitri Warren De Klerk November 2009

Signature helliphelliphelliphelliphelliphelliphelliphelliphelliphelliphellip

vi

KEYWORDS

Haar Features

Integral Image

Cascade

AdaBoost

Boosting

Classifier

Weak Classifier

Strong Classifier

vii

ACKNOWLEDGMENTS

I thank the Almighty God for itrsquos through his strength I can do all things through Christ

who strengthens me Thanks to my supervisor Mr J Connan for his patience support

and guidance and his input in this project Thanks to my family and girlfriend who has

supported and motivated me

Thanks to the staff members of the Department of Computer Science and my classmates

for their kind assistance during the year

I also would like to acknowledge the work done by Desmond Eustin van Wyk His

documentation proved extremely useful in documenting my work

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

v

DECLARATION

I declare that A FACE DETECTION SYSTEM USED FOR ACCESS CONTROL is

my own work that it has not been submitted for any degree or examination in any other

university and that all sources I have used or quoted have been indicated and

acknowledged by complete references

Dmitri Warren De Klerk November 2009

Signature helliphelliphelliphelliphelliphelliphelliphelliphelliphelliphellip

vi

KEYWORDS

Haar Features

Integral Image

Cascade

AdaBoost

Boosting

Classifier

Weak Classifier

Strong Classifier

vii

ACKNOWLEDGMENTS

I thank the Almighty God for itrsquos through his strength I can do all things through Christ

who strengthens me Thanks to my supervisor Mr J Connan for his patience support

and guidance and his input in this project Thanks to my family and girlfriend who has

supported and motivated me

Thanks to the staff members of the Department of Computer Science and my classmates

for their kind assistance during the year

I also would like to acknowledge the work done by Desmond Eustin van Wyk His

documentation proved extremely useful in documenting my work

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

vi

KEYWORDS

Haar Features

Integral Image

Cascade

AdaBoost

Boosting

Classifier

Weak Classifier

Strong Classifier

vii

ACKNOWLEDGMENTS

I thank the Almighty God for itrsquos through his strength I can do all things through Christ

who strengthens me Thanks to my supervisor Mr J Connan for his patience support

and guidance and his input in this project Thanks to my family and girlfriend who has

supported and motivated me

Thanks to the staff members of the Department of Computer Science and my classmates

for their kind assistance during the year

I also would like to acknowledge the work done by Desmond Eustin van Wyk His

documentation proved extremely useful in documenting my work

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

vii

ACKNOWLEDGMENTS

I thank the Almighty God for itrsquos through his strength I can do all things through Christ

who strengthens me Thanks to my supervisor Mr J Connan for his patience support

and guidance and his input in this project Thanks to my family and girlfriend who has

supported and motivated me

Thanks to the staff members of the Department of Computer Science and my classmates

for their kind assistance during the year

I also would like to acknowledge the work done by Desmond Eustin van Wyk His

documentation proved extremely useful in documenting my work

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

viii

GLOSSARY

AdaBoost (Adaptive Boosting) AdaBoost is an efficient machine learning boosting

algorithm which combines weak classifiers while reducing significantly not only the

training error but also the more elusive generalization error

Boosting A method of producing a very accurate prediction rule by combining rough

and moderately inaccurate rules of thumb

Base Resolution The resolution at which the detector starts to detect faces

Weak Classifier The classifier is called weak because they only need to classify correctly

the examples in more of 50 of the cases

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

9

INTRODUCTION

In this project a face detection system is implemented and integrated into an Access

Control system Face detection systems locate the size and scale of human faces in images

and video sequence if present Face detection is the first step for Face localization Face

Tracking Facial Expression recognition and Face Recognition

Face detection in itself is a challenging problem The difficulty resides in the face that faces

are non rigid objects Face appearance may vary between two photographs of the same

person depending on the emotional stage lighting conditions or pose[4] This is why so

many methods have been developed in the past years

The goal is to detect very quickly faces in cluttered backgrounds This situation can be

found in many applications as surveillance of public places common Access Control

conditions Thus far learning-based approaches have been the most effective and have

therefore attracted a lot of attention the last years Viola and Jones [6][7] introduced an

impressive face detection system capable of detection frontal-view faces in real time This

is attributed to the AdaBoost learning algorithm presented by Freund and Schapire [8] and

the response of simple

features used by Viola and Jones [6] Hundreds of features can quickly be calculated by

introducing a new image representation called the ldquoIntegral ImagerdquoThe Adaboost

algorithm sequentially contructs a classifier as a linear combination of ldquoweakrdquo classifier

The classifiers are combined in a ldquocascaderdquo which allows background regions to be quickly

discarded while spending more computation on more promising object like reqions

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

10

C H A P T E R 1

USERS REQUIREMENTS DOCUMENT (URD)

This document describes the problem from the users point of view It briefly describes

the problem domain of face detection Then the document delivers a simple and exact

problem description wherein the user states exactly what heshe would like the face

detection system to do We focus on the tasks to be solved rather than the interface

required solving them

11 - Users view of the problem

The user requires a face detection system given an image the goal of the face detection

system is to determine whether or not there are any faces in the image and if present

return the face location The system should operate at real-time to make for a passive and

fully automatic Access Control system A user would only be required to stand in front of

the camera in order to be recognized The main motivation for a face detection system is

that the user wouldnrsquot be required to position hisher face into a fixed size box in order to

be recognized by the face recognition system

Other reasons for a face detection system are that it is the first step

Face localization seeks to determine the position of a single face within an image the

detection problem is simplified since the input image contains only one face

Facial feature detection seeks to detect the presence and location of features such as the

mouth nose eyes lips ears etc the detection problem is simplified since the input image

contains only one face

Facial expression recognition identifies the emotional states of humans eg happy sad

anger

Face tracking methods estimates the location and possibly the orientation of a face in an

image on a continuous basis within real time

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

11

As proven above face detection is the first step in any fully automated system which solves

the above problems therefore a robust and accurate face detector system is critical

12 - Problem domain

Given an image the goal of a face detection system is to identify all face regions regardless

of its position and scale [1] This problem is challenging as faces are non rigid objects Face

appearance may vary between two different persons and also between two images of the

same person depending on the lighting conditions emotional state and pose of the

subject [4]

Face detection difficulties

The face global attributes Some common face attributes from every face A face can

globally be estimated by a kind of ellipse but humans have thin faces round faces etc

Skin color is also different from one person to another

The facial expression Face appearance is highly depends on emotional state of people

Face features of a smiling face is far from those of an indifferent temperament or a sad

face

Presence or absence of structural components Face detection included objects that

can be found on a face glasses which change one of the main characteristics of the faces

the darkness of the eyes Natural facial features beards mustaches or can occult part of

the face

13 - Complete description of the problem

Determine whether or not there are any faces in the camera output and if present return

the face locations of the images The biggest face detected being the user closest to the

camera are scaled to a recognizable scale This detection window is then passed to the face

recognition system implemented by Desmond Eustin van Wyk [3] for recognition

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

12

14 - Expectations from a system

The first and foremost expectation for a face recognition system is that it must have a high

degree of accuracy when recognizing people The next highest expectation from a system

is that people should be indicated who they are when the system recognized them It must

also be possible to add and remove people from the system that should be recognized

The system is to accurately identify and locate human faces under the following conditions

and circumstances

upright frontal faces

minor variations in lighting conditions and

minor variations in facial expression

minor variation in illumination

big enough scale in order to perform face recognition

any position

15 - Not expected from a system

A solution system is not expected to detect a human face under the following conditions

and circumstances

non frontal face pose

rotated faces

extreme lighting conditions

darkness

too much light

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

13

C H A P T E R 2

REQUIREMENTS ANALYSIS DOCUMENT (RAD)

This document takes the user requirements as a starting point and looks at face detection

from the designers view The analysis focuses on the system and software needed to

implement the user requirements We take the userrsquos requirements and clearly identify all

of the details and mitigating factors that will affect the solution the user wants The RAD

then identifies the software system and paradigm that will best fit the user requirements

21 - Users requirements interpretation

Access control systems are considered to be mission-critical and real-time systems and

thus must operate correctly under many different situations and circumstances For a fully

automatic face recognition system face detection and face localization are very important

and the very first steps to developing such a system[3] The background composition is

one of the main factors for explaining the difficulties in face detection [4] Face detection

in access control systems need to detect faces in any background meaning the background

can be textured and with great variability [4] The single most dominant problem with a

face recognition system or other biometric systems is accuracy and they do not perform

well under the many different situations and circumstances that are encountered in day-to-

day life [3]

The two most important characteristics for a face detector are its detection and error rate

The detection rate of a face detector is defined as the ratio between the number of

correctly detected and the number of actual faces The error can be broken down into two

types of error namely

False positives ndash an image sub-region is declared to be a face but is not

False negatives ndash an image sub-region is not declared as a face when it is a face

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

14

22 - Existing solutions

Face detection is the first step to any fully face recognition face localization face

expression recognition system etc Thus face detection has been highly researched in the

past years there are many different techniques and algorithms for performing face

detection All these techniques fall under the following main methods

Knowledge-based methods

Based on what constitutes a typical face eg the relationship between facial

features

Feature invariant approaches

Finds structural features of a face that exist even when the viewpoint lighting or

pose vary

Template matching methods

Uses several standard patterns to describe the face as a whole or the facial features

separately

Appearance based methods (classifierslearning-based)

The models are learned from a set of training images that capture the

representative variability of facial appearance

23 - Suggested system

Detecting faces in black and white still images with unconstrained complex backgrounds

is a complicated task [1] Thus far learning-based approaches have been most effective

and have therefore attracted a lot of attention over the last years In 2001 Viola and Jones

[6][7] published an impressive face detector system capable of detecting frontal-view faces

in real time The properties of the detector are partly attributed to the AdaBoost learning

algorithm

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

15

AdaBoost (Adaptive Boosting) was rapidly made popular in the machine learning

community when it was presented by Freund and Schapire [8]

As we want to detect faces in various backgrounds found in Access Control systems it

would be improper to use purely geometrical methods In fact the main advantage to these

geometrical methods is the geometric invariant properties We are not interested by them

since were staying in the context of frontal face detection So it is quite naturally that we

have oriented our choice towards learning algorithms Boosting is a powerful iterative

procedure that builds efficient classifiers by selecting and combining very simple classifiers

The suggested system uses Boosting and Haar features (features)

The first step is to compute an image representation called an integral image that allows

for very fast feature evaluation In order to compute these features very rapidly at many

scales we introduce the integral image representation for images The integral image can be

computed from an image using a few operations per pixel Once computed any one of

these Haar-like features can be computed at any scale or location in constant time[7]

The second contribution of this paper is a method for constructing a classifier by selecting

a small number of important features using AdaBoost [6] AdaBoost is used to both select

and train the classifier AdaBoost then boost the performance of weak classifier

The third major contribution of this paper is a method for combining successively more

complex classifiers in a cascade structure which dramatically increases the speed of the

detector by focusing attention on promising regions of the image [7]

24 - Testing the suggested system

The system can be tested on the MIT+CMU frontal face test set [6] This set consists of

130 images with 507 labeled frontal faces

The image dataset is used by the CMU Face Detection Project and is provided for

evaluating algorithms for detecting frontal views of human faces

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

16

C H A P T E R 3

USERS INTERFACE SPECIFICATION

This chapter describes exactly what the user interface is going to do what it looks like and

how the user interacts with the program The UIS however does not describe how the

interface is implemented Nor does it describe what the program does behind the

interface Rather the UIS focuses in detail specifically on the user interface itself

31 ndash Complete user interface

Figure 31 below displays the main graphical user interface frame for our system titled

Access Control

Figure 31 Complete user interface

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

17

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 35 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 38 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

18

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

19

311 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 33 below

Figure 33 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 34 below

Figure 34 How the user interface behaves ndash with detection and recognition

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

20

312 - The Add User Dialog

The Add User Dialog is displayed in Figure 35 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 31

Figure 35 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 36

is displayed If no images was captured the

error dialog in Figure 37 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 31 The Add User dialog interface components described

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

21

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 36 below is displayed

Figure 36 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 37 below is displayed

Figure 37 Error dialog displayed when no image was captured

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

22

313 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 38 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 38 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 32 The Confirm User Delete dialog interface components described

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

23

314 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 39 below

Each menu item of the File menu with its associated action is described in Table 33

Figure 39 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 33 The File menu items described

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

24

315 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

310 below Each menu item of the Face Detection menu with its associated action is

described in Table 34

Figure 310 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 311

Table 34 The File menu items described

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

25

316 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 311 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in a 320x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 311 The File Detection Setting dialog

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

26

317 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 312 Each

menu item of the Help menu with its associated action is described in Table 35

Figure 312 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 313

About Open the About dialog displayed in

Figure 314

Table 35 The Help menu items described

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

27

318 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 313 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 313 The Users Guide dialog

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

28

319 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 313 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 314 The About dialog

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

29

Figure 315 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 315 The webcam that Users interact with and that is also used to capture face images

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

30

C H A P T E R 4

OBJECT ORIENTATED ANALYSIS (OOA)

In this chapter we apply an object-oriented view to the face detection system We begin by

providing a detailed description of the objects in the form of a data directory In addition

we provide detailed class diagrams identifying class attributes and methods Finally we

present the relationships between objects

41 - Data dictionary

The data dictionary describes the Face detection system objects in detail each object is

documented We provide a clean understanding of each object in terms of the functions

they perform in the form of a detailed description

Class Description

adaboostTrain This class trains a weak classifier using the AdaBoost machine learning algorithm presented by Freund amp Schapire [8] Once trained the weak classifier is saved to the database of weak classifiers

buildCascadeClassifier This class builds a cascade classifier which achieves increased performance while radically improving computation time The key insight is smaller more efficient classifiers are constructed which reject many of the negative sub windows while detecting all positive image instances Simple classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to decrease the false positive [6][7] The stages in the cascade classifier are constructed by training classifiers using AdaBoost and then adjusting the threshold to minimize the false negatives [6][7]

cascadeClassifier This class represents a complete cascade classifier with all its stages The cascade classifier is a chainarray of cascade stages Recall that each stage in the cascade

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

31

classifier is a smaller more efficient boosted classifier The form of the cascade classifier is that of a degenerate decision tree A positive result from the first classifier triggers the evaluation of a second classifier and so on A negative result at any point leads to immediate rejection of the sub-window[6][7]

cascadeStage This class represents a cascade classifier stage used in the cascade classifier class The cascade classifier stage is an AdaBoost strong classifier with a fairly small number of weak classifiers This is such that the detector can quickly distinguish if an image sub region is a Face or a Non-face Each stage in the cascade classifier is trained by adding weak classifiers until the target detection and false positive rates are met The stage threshold is adjusted to accept all face example in the training set while minimizing the false negatives

feature The class represents simple features our detector classifies images based on the value of simple features These features which have also been used by Viola and Jones are also known as Haar features or Haar-like features We use 5 kinds of features Given the base resolution of the detector is 19x19 our detector has an exhaustive set of 67209 features

integralImage

This class determines the integral image of a given input PGMImage The integral image can be considered as a means to quickly compute the rectangle features This intermediate representation for the input PGMImage at location (x y) is the sum of all the pixels above and to the left of (x y) Making use of the integral image any rectangular sum can be computed in four array references

integrateMultipleDetections This class represents a final detection since the final detection is insensitive to small changes in translation and scale multiple detections will normally occur around each face in a scanned image This class returns in one final face detection per face by combining clusters of overlapping detections into a single detection In addition this class returns the biggest face detection used by the face recognition system

scanDetector The final detector is scanned across the 320x240 image sequences at multiple scales and locations The detector itself is scaled rather than the image The detector is

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

32

scanned across locations by shifting the detector by two pixels horizontally and vertically The choice of this shifting affects both speed and accuracy of the detector The two pixels shifting promised good results in experiments In addition this class passes the biggest detection to the face recognition system

trainImage This class represents a training image for the AdaBoost machine learning algorithm Each training image has an integral image that represents the object of interest The type being a positive or negative image The weight of the training image is used for training the face detector The weight is used by AdaBoost the weight is a measure of how important the training image is

weakClassifier

This class represents a weak classifier the classifier is called ldquoweakrdquo because the classifier is only expected to classify 50 of the training set images correctly The features are used to build the weak classifiers A weak classifier is a feature with the extend of a classifier error ndash how bad the classifier is tested on a validation set and a classifier weight ndash how good the classifier is tested on the positive training set

Table 4-1 Data Dictionary - the objects combined with a brief

description of each

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

33

42 - Class diagrams

The class diagrams contain the name of the class itrsquos attributes as well as the associated

methods of the class The type of the class attributes the return types of the methods as

well as the class method parameters

Figure 41 adaboostTrain Class ndash trains a weak classifier using the AdaBoost algorithm

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

34

Figure 42 buildCascadeClassifier Class ndash trains a cascade classifier used by the face detector for face detection

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

35

Figure 43 cascadeClassifier Class ndash The trained cascade classifier used by the face detector for face detection

Figure 44 cascadeStage Class ndash A smaller more efficient boosted classifier used by the cascade classifier

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

36

Figure 45 feature Class ndash Our detector classifies images based on these features

Figure 46 integralImage Class ndash determines the integral image in order to quickly compute feature response values

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

37

Figure 47 integrateMultipleDetections Class ndash This class combines multiple detection as well as return the biggest

detection

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

38

Figure 48 scanDetector Class ndash scans the detector at multiple scales and locations and passes the biggest detection to the face

recognition system

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

39

Figure 49 trainImage Class ndash training image used to train the face detector

Figure 410 weakClassifier Class ndash classifier used by the

cascade classifier

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

40

43 - Relationship between objects

The following Figure 411 represents the relationship between the objects indicates how

the objects interact with each other or how they are related to each other

Figure 411 Relationship between classes

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

41

C H A P T E R 5

OBJECT ORIENTATED ANALYSIS (OOD)

The document is as close to coding as you can get without actually coding This document

takes the classes in the Object Oriented Analysis deeper into the realm of pseudo-code

51 ndash Inner details of class methods

The inner details of class method in detail each method is documented to give a more

detailed description of the class object

Class Method Description

adaboostTrain determineNumPosAndNeg() This method determines the number of negative and positive images the training set consists of

getTrainingSet() This method gets the Adaboost training set

getWeakClassifierCounter() This method returns the number of weak classifiers trained thus far in the weak classifier database

initializeWeights() This function initializes the weight uniformly over the training data the sum of the weights of all images in the trainingSet equals 1

normalizeWeights(javautilVectorlttrainImagegt training

Set) This method normalize the weights of the training set such that the weights is a probability distribution Sum(all weights) = 1

printWeights(javautilVectorlttrainImagegt trainingSet) This method is used for debugging

setupTrainingSet() This function sets the positive and negative images of the training set

setWeakClassifierCounter(int weakClassifierCounter) This method sets the Adaboost training set

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

42

trainClassifierWithAdaBoost() This method trains a weak classifier using AdaBoost and writes the weak classifier to the database of weak classifiers

updateNegativeTrainingImages(javautilVectorlttrainIma

gegt newNegatives) This method updates the negative training images by adding the input vector of negatives to the training set

updateWeight(weakClassifier weakClassifiers

javautilVectorlttrainImagegt trainingSet) This method updates the weight of the training set

baseLearner

allFeatures(int scale) This method calculates all possible features that can fit into a given image of width and height specified

baseLearner(javautilVectorlttrainImagegt trainingSet) This method returns the weak classfier with the lowest training error on the training set

calculateOptimalThresholdValues(javautilVectorlttrain

Imagegt trainingSet) This method determines the optimal thresold values for all the features

evaluateError(int featureOptThreshold

double lowestError) This method evaluates the weighted error of a feature over the training set such that we can choose the weak classifier with the lowest error

getAllFeatures() This method gets the features used to choose a weak classifier from

getFeatureOptimalThreshold() This method sets the feature optimal threshold values for all the features

initializeValues() This method initialize the features response values as well as the feature optimal threshold values

optimalThreshold() This method is used to determine the optimal threshold over the training set

removeFeatures(int featureNum) This method removes a feature from being used

setAllFeatures(javautilVectorltfeaturegt allFeatures) This method sets the features used to choose a weak classifier from

setFeatureOptimalThreshold(javautilVectorltjavalang

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

43

Integergt featureOptimalThreshold) This method sets the feature optimal threshold values for all the features

totalHScales(int imageWidth feature evaluateFeature) This method determines the total number of scales at which the input feature can be applied within a given imageWidth

buildCascadeClassifier

buildCascadeClassifier() This method build a cascade classifier using the AdaBoost machine learning algorithm

calculateTotalWeightForStage() This method calculates the total weight for the current stage from where the stage start to where the stage end

evaluateD() This method calculates the values D the detection rate of the cascade classifier on a the positive training images

evaluateF(boolean getThresholdValues) This method calculates the value F - the false positive rate of the cascade classifier on a validation set

initializeThresholdValues() This method initialize the thresholdValues array

smartTrainWrite(javautilVectorlttrainImagegt trainingS

et javalangString filename)

stageThreshold() The initial AdaBoost threshold is designed to yield a low error rate on the training data

cascadeClassifier addStage(cascadeStage stage) This method adds a stage to the cascade classifier

getStage(int stageNum) This method returns the cascade classifier stage at the given stageNum

readCascade(javalangString filename) This method reads the cascade classifier from the file system

stageEdit(int stageNum cascadeStage newStage)

toFile() This method is used by the writeCascade method

toString() This method prints the cascade classifier to a string

totalStages() This method returns the total number of stages in the cascade classifier

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

44

writeCascade(cascadeClassifier cascade

javalangString filename) This method saves the cascade classifier to the file system

cascadeStage getThreshold() This method gets the threshold of the stage

getTotalClassifiers() This method gets the total number of weak classifiers there are in this stage

getweakClassifierStart() This method gets the start of the weak classifiers in the weak classifier database for this stage

increaseTotalClassifiers() This method increase the total number of weak classifiers there are in this stage

setThreshold(double threshold) This method sets the threshold of the stage

setTotalClassifiers(int weakClassifierTotal) This method sets the total number of weak classifiers there are in this stage

setweakClassifierStart(int weakClassifierStart) This method sets the start of the weak classifiers in the weak classifier database for this stage

feature calculateFeature(int[][] integralimage

int initialScale int currentScale int x int y) This method calculates the feature response value of this feature on the input integral image

getHeight() This method gets the height of the feature

getHeightScale() This method gets the heightScale of the feature

getOptimalThreshold() This method gets the optimal threshold of the feature

getWidth() This method gets the width of the feature

getWidthScale() This method gets the widthScale of the feature

getX() This method gets the x(topleft) column location of the feature

getY() This method gets the y(topleft) row location of the feature

I(int[][] integralImage int xCoordinate

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

45

int yCoordinate) This method returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

setHeight(int height) This method sets the height of the feature

setHeightScale(int heightScale) This method sets the heightScale of the feature

setOptimalThreshold(int optimalThreshold) This method sets the optimal threshold of the feature

setWidth(int width) This method sets the width of the feature

setWidthScale(int widthScale) This method sets the widthScale of the feature

setX(int xCoordinate) This method gets the x(topleft) column location of the feature

setY(int yCoordinate) This method gets the y(topleft) row location of the feature

toString() This method prints the feature to a string

integralImage

I(int x int y) This function returns the integral image values in case x = -1 or y = -1 for the following condition I(-1 y) = I(x -1) = I(-1-1) = 0 else it just returns the integral image value at x and y respectively

integralImage(PGMImage pgm) This function takes as input a pgm image and determines the corresponding integral image of the image

printIntegralImage(int[][] integralImage) This method is used for debugging

integrateMultipleDetections

biggestDetection() This method returns the biggest detection of all the detections

clusterizeDetections() This method puts the detections into clusters - a cluster is defined as detections which overlaps

integratedDetections(boolean print) Each partition yields a single final detectionThe corners of the final bounding region are the average of the corners of all detections in the set

integrateMultipleDetections(javautilVectorltdetection

Windowgt multipleDetections byte[] outData int cy

int cY int cx int cX int lineStride

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

46

int pixelStride) This method takes as input a Vector of detections of faces

printDetection(detectionWindow detectWindow) Takes as input a detection and prints a white border around the detected face

scanDetector loadCascade() This method loads in the cascade classifier from the file system

scanDetector(PGMImage inputImage byte[] outData

int cy int cY int cx int cX int lineStride

int pixelStride) This method runs the detector(cascade classifier) over the camera output images

trainImage getIntegralImage() This method gets the integral image for this trainImage

getType() This method gets the type of an image if its a positive or a negative image

getWeight() This method gets the weight of an image in the training database over the training set

setIntegralImage(int[][] image) This method sets the integral image for this trainImage

setType(int type) This method sets the type of an image if its a positive or negative image

setWeight(double weight) This method sets the weight of an image in the training database over the training set

weakClassifier

determineAndSetClassifierWeight(double error) This method determines and sets the weight of the classifier since hypothesis weight = 12 ln (1-error)error

getClassifierError() This method gets the error of this classifier

getClassifierFeature() This method gets the feature of a weak classifier

getClassifierWeight() This method gets the weight of the classifier

readWeakClassifier(javalangString filename) This function reads the weakClassifier from a file

setClassifierError(double classifierError) This method sets the error for this classifier

setClassifierFeature(feature classifierFeature)

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

47

This method sets the feature of a weak classifier

weakClassifiertoString() This function prints the weak classifier to a string

writeWeakClassifier(weakClassifier weakclassifier

javalangString filename) This function write the weakClassifier to a file

Table 5-1 Inner details of the system classes

52 ndash Pseudo Code

521 ndash Scan Detector

Recall that the detector gets scanned at all scales and location across the image rather than

the image itself This pseudo code scans the image at the base (initial) resolution of 19x19

at all locations in the image it then scans the 320x240 resolution image The scale of the

detector is increased and the process in repeated until itrsquos scanned the images at all scales

windowWidth = 320 heightWidth = 240 width = 19 height = 19 for ( all possible scales a widthheight sub-window can fit into windowWidthheightWidth) scanDetectorWithSubwindow(width height ) width ++ height ++ function scanDetectorWithSubwindow(width height ) for (h = 0 to h lt windowHeight ) for (w=0 to w lt windowWidth ) if (w + width lt windowWidth AND h + height lt windowHeight ) classifySubWindow(w+width h+height ) endIf endFor endFor

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

48

C H A P T E R 6

CODE DOCUMENTATION

The full code documentation of our code is not contained in this project due to the

number of pages it covers The code documentation can however be found on the

accompanying compack disk(CD)

In the code documentation

Every class and class method are described using in-line comments or a brief

detailed description of the algorithm or its workings The same applies for

methods The javadoc web pages makes for easily browsing the code

documentation Where applicable we note any caveats ndash things that could go

wrong or things that the code doesnrsquot address

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

49

C H A P T E R 7

TESTING DOCUMENT

This chapter describes how we tested our system The system has been tested on the

MIT+CMU frontal face test set The results of the face detector are shown below

MIT+CMU frontal face test set

Images collected at CMU and MIT

275 Correctly detected out of 472 images

58 Detection rate

The following Figure 71 displays a false positive the system detects the following sub-

window as a face when itrsquos not

Figure 71 ndash False positive detection

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

50

The following Figure 72 displays a positive detection the system detects the face in this

sub-window and the detection isnrsquot a fully cropped face This is because of the way the

system integrates multiple detections The system is invariant to small changes in

detection therersquos multiple detections around the face The best detection is the one

displayed

Figure 72 ndash Correct detection

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

51

C H A P T E R 8

USERS GUIDE

This document tells a user how to use the Access Control system This document may

also be used by a programmer as a guide to improve or edit the system This document

describes the system requirements to use the Access Control system the system project

directory structure and how to run the Access Control system

81 - System requirements

All the requirements to setup run and edit the Access Control system are contained in

Table 81

System requirements

Hardware Software

A personal computer (PC) that can satisfy

the software requirements and webcamrsquos

requirements

Microsoft Windows based operating system

available from Microsoft Corporation

A webcam Installed Sun Java Runtime Environment

(JRE) and Java Development Kit (JDK) ndash

6u2 (Update 2) available from Sun

Microsystems

Installed Java Media Framework 211e

available from Sun Microsystems

Installed NetBeans IDE 65 available from

wwwnetbeansorg

Table 81 The system requirements a table to setup run and edit the Access Control system

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

52

82 - The Face Recognition System project directory

The Access Control System project directory is displayed in Figure 81 below Table 82

explains the directory and its contents

Figure 81 Access Control Systemrsquos project directory structure

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

53

Directory ContentsDescription

Access_Control_System The main project directory which contains all

project directories The directory contains two

files

AccessControljar ndash an executable jar file that

runs the Access Control system

AccessControlJavadocshtml ndash a link to the

javadocs indexhtml file which documents the

source code

Access_Control_SystemUserGuide This directory contains the Html User Guide

file used by the Access Control system

Access_Control_Systemlog This directory logs any Access Control activity

The directory contains a directory namely

YEAR-MONTH-DAY this directory contains

the log files for that day The log files are

named per hour and minute the system was

terminated eg HOUR-MINUTElog

Access_Control_Systemtrain This directory contains the training images used

for training the face detector as well as the

weak classifiers and cascade classifier the

detector uses for face detection

Access_Control_Systemdistjavadoc This directory contains the generated javadoc

files

Access_Control_Systemsrc The directory that contains all the source code

used by the Access Control system

Access_Control_SystemsrcaccessControl

Systemresources

The directory that contains all the Access

Control resources

Access_Control_SystemsrcfaceDetection The source code directory for the faceDetection

package

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

54

Table 82 The Access Control system project directory and contents explained

83 - Running the Access Control system

Table 83 below describes the steps necessary to run the Access Control system

Running the Access Control system

Step 1 Make sure that the system requirements in Table 81 are met

Step 2 Make sure that a video capture device is connected to the PC

Step 3 Double click the Executable Jar File named Access Controljar to run the

Access Control System

Table 83 Running the Access Control system

84 ndash Complete user interface

Figure 82 below displays the main graphical user interface frame for our system titled

Access Control

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

55

Figure 82 Complete user interface

User Image panel

The User Image panel displays the user image of the user currently selected in the User list

panel

User List panel

The User List panel lists the authorized users the Access Control system can recognize

Search field

The search field can be used to search for a user in the face recognition system This

feature can be very useful when the face recognition system have hundreds of users

The search function searches for the substring of the search text in the username and

lists all the matched usernames in alphabetical order displaying the first matched usersrsquo

image in the user image panel

Camera output

The camera output displays the camerarsquos output Used for capturing images when adding

users to the face recognition system and for monitor current user login activity

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

56

Add user button

The add user button adds an authorized user to the Access Control system On execution

of the Add User button the Add User dialog as shown in Figure 86 is displayed

Remove user button

The remove button is only enabled when there are users in the system The remove button

removes a user from the Access Control system On execution of the Remove User

button the Confirm User Delete dialog in Figure 89 is displayed

Recognition output

The Access Control system displays the output of the recognized user below camera

output When a user is not recognized by the Access Control system the system displays

Who are you otherwise the system displays the user ID of the recognized user and the

rate of the face recognition

Log panel

When a user is detected by the face detection system and recognized by the face

recognition system the Access Control system logs the userrsquos information to the log panel

The log panel logs the user and the time at which the user logged in the log file gets saves

with the current date every time the user terminates the system

Acceptance threshold

The face recognition system has a threshold at which users should be recognized With 0

being the lowest threshold value and with 1 being the highest threshold value With 0

being the most lenient and 1 being the most strict at which the recognition operates The

Acceptance threshold spinner is used to adjust the acceptance threshold value used with

which the ANNs output is compared [1]

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

57

841 - How the user interface behaves

When the face detection system does not detect any faces or detects a false positive or the

face recognition system does not recognize the user Then the system displays Who are

you As displayed in Figure 84 below

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

58

Figure 84 How the user interface behaves ndash with detection and no recognition

When the users face is detected and recognized the system will display the user ID and

the recognition rate as displayed in Figure 85 below

Figure 85 How the user interface behaves ndash with detection and recognition

841 - The Add User Dialog

The Add User Dialog is displayed in Figure 85 below this frame is displayed when the

Add User button in the Access Control GUI is pressed or when the Add User menu item

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

59

from the File menu is selected The location of the Add User Dialog is such that the

cameras output can be clearly viewed Each interface component in the Add User Dialog

with its purpose or action is described in Table 81

Figure 86 The Add User dialog

Add User dialog

Interface component PurposeAction

User Image panel Displays the captured image

Capture Image button Captures an image

ID label and text field The text field where a user identifier should

be entered

Ok button Add the user with the specified identifier

that should be recognized If the ID text

field is empty the error dialog in Figure 87

is displayed If no images was captured the

error dialog in Figure 88 is displayed

Cancel button Do not add the user discard the captured

image and close the Add User dialog

Table 81 The Add User dialog interface components described

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

60

From the add user dialog if a user clicks Ok without entering a user ID or capturing an

image The following dialog in Figure 87 below is displayed

Figure 87 Error dialog displayed for empty ID text field or no captured image

From the add user dialog if a user clicks Ok without capturing an image The following

dialog in Figure 88 below is displayed

Figure 88 Error dialog displayed when no image was captured

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

61

842 - The Confirm User Delete dialog

The Confirm User dialog is displayed in Figure 89 below this dialog is displayed when

selecting a user in the User List panel and clicking the Remove User button When

removing a user from the system the default option is set to no such that the

administrator doesnrsquot blindly remove a user from the Access Control system

Figure 89 Confirm User Delete dialog displayed when clicking the remove user button

Confirm User Delete dialog

Interface component PurposeAction

Yes button Completely removes the user from the

Access Control system

No button Do not remove the user close the

confirmation dialog

Table 82 The Confirm User Delete dialog interface components described

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

62

843 - The File menu

The complete File menu with the menu items it contains is displayed in Figure 810 below

Each menu item of the File menu with its associated action is described in Table 88

Figure 810 The File menu

The File menu

Menu item Action

Add User Adds a user to the Access Control system

Exit Exits the Access Control system

Table 88 The File menu items described

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

63

844 - The Face Detection menu

The complete Face Detection menu with the menu items it contains is displayed in Figure

811 below Each menu item of the Face Detection menu with its associated action is

described in Table 84

Figure 811 The File Detection menu

The Face Detection menu

Menu item Action

Settings Opens the Face Detection Settings dialog as

displayed in Figure 812

Table 84 The File menu items described

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

64

845 ndash Face Detection Settings dialog

The Face Detection Settings dialog is displayed in Figure 812 below this dialog is

displayed when the Settings menu item from the Face Detection menu is selected

In this dialog the administrator can set the scales in which the face detection should

operate The face detection system will try and locate faces at the starting scale and

increasing in step size until it reaches the final scale The default settings has a starting

scale of 19 thus the face detection system will try and locate all 19x19 faces in the camera

output Then the system will increase this resolution by its step size which are 5 by

default The system then tries and locates all 24x24 faces in the camera output The system

will continue increasing in step size until it reaches the final scale which are set to 240x240

by default

Choosing a big starting scale will dramatically improve the performance of the face

detection system and thus the Access Control system This is because there are more small

scales in an 820x240 window than big scales

Important

These settings highly affect the performance of the Access Control system The more

scale the face detections system has to cover the slower the system will run

The starting scale and final scale is also the distance as to how close a user should stand

in front of the camera in order to be detected by the face detection system so much care

should be taken into playing with these settings

Figure 812 The File Detection Setting dialog

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

65

846 ndash The Help menu

The complete Help menu with the menu items it contains is displayed in Figure 813 Each

menu item of the Help menu with its associated action is described in Table 85

Figure 813 The Help menu

The Help menu

Menu item Action

User Guide Open the Users Guide dialog displayed

in Figure 814

About Open the About dialog displayed in

Figure 815

Table 85 The Help menu items described

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

66

847 ndash The User Guide dialog

The Users Guide dialog is a modal dialog that is displayed in Figure 814 This dialog

contains the Users Guide for the Administrator It is opened when the Users Guide menu

item from the Help menu is selected The Users Guide dialog can be closed by either the

close button at the top right corner or the Ok button at the bottom of the dialog [1]

Figure 814 The Users Guide dialog

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

67

848 ndash The About dialog

The About dialog is a modal dialog that is displayed in Figure 815 This dialog contains a

message about the Access Control system The About dialog can be closed by either the

close button at the top right corner or the OK button at the bottom of the dialog

Figure 815 The About dialog

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

68

Figure 816 below displays the webcam that Users interact with and that is also used to

capture face images

Figure 816 The webcam that Users interact with and that is also used to capture face images

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

69

C h a p t e r 9

CONCLUSION

In this mini-theses We discussed the implementation of a face detection system to be

used for access control The focus was thus to implement a face detection system good

enough to be used for access control Access control systems normally use video cameras

that deliver image data of poor quality and that also contain much noise [3] The focus was

also on developing a real-time face detection system The Viola and Jones have been most

suited for our requirements thus we implemented a detector strongly based on the Viola

and Jones detector

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995

70

BIBLIOGRAPHY

[1] A Jorgensen AdaBoost and Histograms for Fast Face Detection 2006

[2] BKL Erik Hjelmas ldquoFace Detection A Surveyrdquo Computer Vision and Image

Understanding vol 3 no 3 pp 236-274 Sept 2001

[3] D van Wyk httpwwwcsuwcaczaindexphpHonours-2006Desmond-Van-

Wykhtml[online] November 2006

[4] J Meynet Fast Face Detection Using AdaBoost July 2003

[5] R Lienhart and J Maydt An extended set of Haar-like features for rapid object detection In IEEE ICIP 2002 Vol1 pp 900-903 [6] P Viola and M Jones Rapid object detection using a boosted cascade of

simple features In Proc IEEE Conference on Computer Vision and Pattern Recognition pages

511ndash518 Dec 2001

[7] P Viola and M Jones Robust real-time object detection IEEE ICCV Workshop

Statistical and Computational Theories of Vision July 2001

[8] Y Freund and RE Schapire A decision-theoretic generalization of on-line learning

and an application to boosting In Proceedings of the Second Europen Conference on Computational

Learning Theory pages 23ndash37 Springer-Verlag 1995