10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010)

CLASSIFICATION OF HUMAN GAIT FEATURES WITH DIFFERENT

APPAREL AND WALKING SPEED

Hu Ng #, Hau-Lee Tong # , Waai-Haw Tan *, Junaidi Abdullah #

# Faculty ofInformation Technology, Multimedia University lalan Multimedia, 63100 Cyberjaya, Se1angor, Malaysia

* Faculty of Engineering, Multimedia University 1 alan Multimedia, 63100 Cyberj aya, Selangor, Malaysia

ABSTRACT

In this paper, we proposed a new approach for the classification of human gait features with different apparel and various walking speed. The approach consists of two parts: extraction of human gait features from enhanced human silhouette and classification of the extracted human gait features using fuzzy k-nearest neighbours (KNN). The joint angles together with the height, width and crotch height of the human silhouette are collected and used for gait analysis. The training and the testing sets are separable without overlapping. Both sets involve nine different apparel and three walking speed. From the experiment conducted, it can be observed that the proposed system is feasible as satisfactory results have been achieved.

Keywords: Human identification, Gait analysis, Fuzzy k­

nearest neighbour

1. INTRODUCTION

Human identification based on biometrics refers to the automatic recognition of the individuals based on their physical and/or behavioural characteristics such as face, fingerprint, gait and spoken voice. Biometrics are getting important and widely acceptable nowadays because they are unique and one will not lose or forget them over time.

Since every individual has hislher own walking pattern, gait is an unique feature. Human walking is a complex locomotion which involves synchronized movements of body parts, joints and the interaction among them [1]. Gait is a new motion based biometric technology, which offers the ability to identify people at a distance when other biometrics are obscured. Furthermore, there is no point of contact with any feature capturing device and is henceforth unobtrusive.

Basically, gait analysis can be divided into two major categories, namely model-based method and model-free method [2]. Model-based method generally models the human body structure or motion and extracts features to match them to the model components. It incorporates knowledge of the human shape and dynamics of human gait into an extraction process. This implies that the gait dynamics are extracted directly by determining joint positions from model components, rather than inferring dynamics from other measures, thus, reducing the effect

978-1-4244-7167-6/10/$26.00 ©201 0 IEEE 662

of background noise (such as movement of other objects). For instance, Johnson used activity-specific static body parameters for gait recognition without directly analyzing gait dynamics [3]. Cunado used thigh joint trajectories as the gait features [4]. The advantages of this method are the ability to derive gait signatures directly from model parameters and free from the effect of different clothing or viewpoint. However, it is time consuming and the computational cost is high due to the complex matching and searching process.

Conversely, model-free method generally differentiates the whole motion pattern of the human body by a concise representation without considering the underlying structure. The advantages of this method are low computational cost and less time consuming. For instance, BenAbdelkader et al. proposed an eigengait method using image self-similarity plots [5]. Collins et al. established a method based on template matching of body silhouettes in key frames during a human's walking cycle [6]. Philips et al. characterized the spatial-temporal distribution generated by gait motion in its continuum [7].

This paper presents a model-free silhouette based technique to extract the human gait features by dividing human silhouette into six body segments and applying Hough transform to obtain the joint angles. This concept of joint angle calculation is found faster in process and less complicated than the model-based method like linear regression approach by Y 00 et al. [8] and temporal accumulation approach by Wagg et al. [9].

2. OVERVIEW OF THE SYSTEM

First, morphological opening is applied to reduce background noise on the raw human silhouette images. Each of the human silhouette is then measured for its width and height. Next, each of the enhanced human silhouette is divided into six body segments based on the anatomical knowledge [10]. Morphological skeleton is later applied to obtain the skeleton of each body segment. The joint angles are obtained after applying Hough transform on the skeletons. Step-size, which is the distance between the bottom of both legs, is measured from the skeletons of the lower legs. Crotch height, which is the distance between the subject's crotch and the floor, is also determined. The dimension of the human silhouette, step-size, crotch height and six joint angles

from body segments are then used as the gait features for classification. Figure I summarises the process flow of the proposed system.

Part 1: Extraction of human gait features

Original image Measurement of Human silhouette enhancement wi dth and height segmentation

1 Measurement of Joint angles .- Skeletonization

step-size and f4- extraction Q.( body segments

crotch height

----- ----------------------------------

Part 2: Classification of extracted human gait features

Computation of Determination of Classification of the similarities t----i the fuzzy k- f---e the unlabeled

nearest neighbor subjects

Figure 1. Flow chart of the proposed system

3. EXTRACTING THE HUMAN GAIT FEATURES

The original human silhouette images are obtained from the SaTaN covariate database [11]. They used static cameras to capture eleven subjects walking along the indoor track in four different angles. Video data was first preprocessed using Gaussian averaging filter for noise suppression, followed by Sobel edge detection and background subtraction technique to create the human silhouette images. The database was used in this study to evaluate the recognition rate of the proposed gait features extraction technique for different covariate factors.

In most of the human silhouette images, shadow was found especially near to the feet due to poor lighting condition. It appeared as part of the subject body in the binary human silhouette image as shown in Figure 2. The presence of the artifact affects the gait feature extraction and the measurement of joint angles. The problem can be reduced by applying Morphological opening with a 7 x 7 diamond shape structuring element, as denoted by

Ao B = (A8B) EEl B) (1)

where A is the image, B is the structuring element, 8

represents morphological erosion and EEl represents morphological dilation. The opening first performs erosion, followed by dilation. Figure 2 shows the original and enhanced images.

After that, the width and height of the enhanced human silhouette are measured.

(a) Original video image (b) Original silhouette image

(c) Enhanced silhouette image

Figure 2. Original and enhanced images after morphological

opening

663

Next, the enhanced human silhouette is divided into six body segments based on anatomical knowledge [10]. Figure 3 shows the six segments of the body, where a

represent head and neck, b represents torso, c represents right hip and thigh, d represents right lower leg and foot, e represents left hip and thigh and f represent left lower leg and foot.

Figure 3. Six body segments

To enhance the segments structure, morphological skeleton is used to construct the skeleton for all the body segments. Skeletonization involves consecutive erosions and opening operations on the image until the set differences between the two operations is zero.

Erosion Opening Set differences

A8kB (A8kB) 0 B (A8kB) � ((A8kB)) 0 B (2) where A is an image, B is the structuring element and k is from zero to infinity.

To extract the joint angle for each body segment, Hough transform is applied on the skeleton. For the details of the steps and figures, please refer to [12].

To obtain the step-size of each walking sequence, the Euclidian distance between the bottom ends of lower right leg and lower left leg are measured. To obtain the crotch height, the distance between the subject's crotch and the floor is measured. Figure 4 shows all the gait features extracted from a human silhouette, where Angle 7 is the thigh angle, calculated as

Angle 7 = Angle 6 � Angle 4 (3)

� __ 1-oo1 .. ------'..:.:='-'------�· 1

Figure 4. All the extracted gait features

4. CLASSIFICATION TECHNIQUE

For the classification, the supervised fuzzy K-Nearest Neighbour (KNN) algorithm is applied, as there is sufficient data to be used for training and testing. Basically, KNN is a classifier to distinguish the different subjects based on the nearest training data in the feature space. In other words, subjects are classified according to the majority of nearest neighbours. In extension to KNN, Keller et al. [13] has integrated the fuzzy relation with the KNN. According to the Keller's concept, the unlabeled subject's membership function of class i is given by:

I U;(X)[ 1 2 J

xEKNN II X -X Ilm-l

�W= �

I [ 1 2 J

xEKNN II X -X Ilm-l

where X , x and ulx) represent the unlabeled subjects, labelled subjects and x's membership of class i respectively. Equation (4) will compute the membership value of unlabeled subject by the membership value of labelled subject and distance between the unlabeled subject and KNN labelled subjects.

5. EXPERIMENTAL RESULTS AND DISCUSSION

The experiment was carried out for nine subjects with nine different apparel and three different walking speed to give a total of twelve conditions. Each subject was filmed wearing a variety of footwear (flip flops, socks, boots, own shoes and trainers), clothes (normal or with rain coat) and carrying various bags (barrel bag slung over shoulder or carried by hand, and rucksack). They were also filmed walking at different speeds (slow, fast and normal speed). For each subject, there are approximately twenty sets of walking data in normal track (walking parallel to the static camera). Figure 5 shows the examples of the subject with different apparel.

(a) Flip flop ...,.-=-...,

( c) Rucksack (d) Rain coat

664

(e) Trainer shoes (f) Barrel bag carried by hand

Figure 5. Examples of subject with different apparel. Left:

original image. Right: silhouette image.

The experiments are carried out to study whether the collected data provide any degree of significance for the recognition of different subject via the supervised classification techniques.

In order to obtain the optimized results, five gait features were adopted for the classification. First,

maximum thigh angle, (f'ax was determined from all the

thigh angles collected during a walking sequence. When (f'ax was located, the corresponding values for the step­

size, S and width, wand height, h are determined as well. From the graph plotted in Figure 6, it can be observed that the changes in width, crotch height and thigh angle over time depict sinusoidal patterns during a walking sequence. Therefore, additional features observed are the average of the local maxima detected for width (A w) ,

crotch height (AH) and thigh angle (AT).

200 r--�-�-�-�-�-�----'

Figure 6. Changes in width, crotch height and thigh angle over

time

As a result, seven features are channelled into the classification process and the distance is defined as:

D(x;'Xj) = (Oinax _O;ax)2+(W; _ W)2 +(h; _ h)2 +(S; _S)2+(A;W - A7i

+ (A;H _A;)2 +(A/ -AJf

Since supervised classification algorithm is adopted, the classification process is divided into training and testing parts. For the training part, eight walking data for each subject in each condition were utilised. The rest of the data will be used in the testing part. As such, in total there are 864 training and 1368 testing data respectively.

The experiments have been conducted for the different values of k neighbours, where k = 2, 4, 5, 6, 7 and 8. The

(5)

maximum of k is eight as the training data of each class is eight. The results obtained are depicted in Table 1.

TABLE 1.

THE OVERALL ACCURACY FOR DIFFERENT VALUES OF k

k Accuracy (%)

2 75.29

3 75.73

4 75.87

5 76.82

6 77.05

7 77.63

8 77.41

From Table 1, it can be concluded that the changes of value k do not have a significant impact on the accuracy of the classification. However, the result obtained when k

= 7 is slightly better than other values. This is also better than the recognition rate of 73.4% reported by Bouchrika et al. [14].

In addition to the evaluation for each condition, classification results for each subject were evaluated as well. This is to determine which unlabelled subjects are well identified and vice versa for all the conditions. Since k = 7 provided the best result for all twelve conditions, the experiment was carried out using k = 7 for subject evaluation. The results obtained for the respective subjects are shown in Table 2.

TABLE 2.

THE ACCURACY OF RECOGNITION FOR RESPECTIVE SUBJECTS WHEN k=7

Accuracy (%)

Subject 1 92.59

Subject 2 86.36

Subject 3 67.32

Subject 4 69.72

Subject 5 96.05

Subject 6 89.54

Subject 7 65.66

Subject 8 55.27

Subject 9 79.60

From Table 2, subject 5 produces the most satisfactory classification results for all the twelve conditions. This is attributed to the features in subject 5 which are highly distinctive from other subjects. However, subject 8 only achieved an accuracy of 55.27%. This is due to the large number of misclassifications of subject 8 to other subjects.

6. CONCLUSION

We have described a new approach for extracting the gait features from enhanced human silhouette image. The gait features are extracted from human silhouette by determining the skeleton from body segment. The joint angles are obtained after applying Hough transform on the skeleton. The results show that the proposed method is robust and can perform well regardless of walking speed and apparel. The future plan is to apply more gait

665

features in order to achieve higher accuracy in classification.

ACKNOWLEDGMENT

The authors would like to thank Prof Mark Nixon, School of Electronics and Computer Science, University of Southampton, United Kingdoms for providing the database for use in this work.

REFERENCES

[I] C. BenAbdelkader, R. Culter, H. Nanda and L. Davis,

"EigenGait: motion-based recognition of people using image self-similarity," in Proc. of International Conference Audio and Video-Based Person Authentication, pp. 284-294, 2001.

[2] M. S. Nixon, T. Tan and R. Chellappa, Human Identification Based On Gait. 2nd ed., Berlin, Germany:

Springer, 2006.

[3] Bobick and A. Johnson, "Gait recognition using static,

activity-specific prameters," in Proc. of IEEE Computer Vision and Pattern Recognition, I, pp.423-430, 2001.

[4] D. Cunado, M. S. Nixon and 1. Carter, "Automatic extraction and description of human gait models for recognition

purposes," Computer and Vision Image Understanding, vol. 90, no 1, pp. 1 � 41,2003. [5] BenAbdelkader, R. Cutler and L. Davis, "Motion-based

recognition of people in EigenGait space," in Proc. of Fifth IEEE International Conference, pp. 267 � 272, May 2002.

[6] R. Collin, R. Gross and J. Shi, "Silhouette-based human

identification from body shape and gait," in Proc. of Fifth IEEE International Conference, pp 366-371, May 2002.

[7] P.J. Phillips, S. Sarkar, 1. Robledo, P. Grother and K.

Bowyer. "The gait identification challenge problem: Dataset and baseline algorithm," in Proc. of 16th International Conference Pattern Recognition, vol I, pp. 385-389, 2002.

[8] Jang-Hee Yoo, M. S. Nixon and C. 1. Harris, "Extracting

human gait signatures by body segment properties," in Fifth IEEE Southwest Symposium on Image Analysis and Interpretation, . pp. 35-39,2002.

[9] K. Wagg and M. S. Nixon "On Automated Model-Based

Extraction and Analysis of Gait," in Proc. of 6th IEEE International Conference on Automatic face and Gesture Recognition, pp.II-16, 2004.

[10] W. T. Dempster and G. R. L. Gaughran, "Properties of

body segments based on size and weigh,t, American Journal of Anatomy, vol. 120, pp. 33-54, 1967.

[11] J. D. Shutler, M. G. Grant, M. S. Nixon, and J. N. Carter, "On a large sequence-based human gait database," in Proc. of lh International Conference on Recent Advances in Soft Computing, Nottingham (UK), pp. 66-71, 2002.

[12] H. Ng, W. H. Tan, H. L. Tong, J. Adullah and R. Komiya,

"Extraction of Human Gait Features from Enhanced

Silhouettes," in Proc. of of rt International Conference on Signal and Image Processing Application, Nov. 2009.

[13] 1. Keller, M. Gray and J. Givens, "A fuzzy K-nearest

neighbour algorithm," IEEE Trans. Systems, Man, Cybem. vol.

15,p� 580-585, 1985. [14] 1. Bouchrika and M. S. Nixon, "Exploratory Factor

Analysis of Gait Recognition," in 8th IEEE International Conference on Automatic Face and Gesture Recognition, 2008.