Embed Size (px)
Citation preview
A Dual-Biometric-Modality Identification System Based on Fingerprint and EEG
Fei Su Liwen Xia Anni Cai School of Information and Communication Engineering
Beijing University of Posts and Telecommunications Beijing, China
[email protected], [email protected], [email protected]
Junshui Ma
Merck Research Laboratories Merck & Co, Inc. Rahway, NJ, USA
Abstract—This paper proposes a dual-biometric-modality personal identification system, which has both outstanding identification performance and effective anti-spoofing property. This dual-modality system represents the first effort to fuse the conventional fingerprint with a novel biometric modality-- Electroencephalogram (EEG). The fusion of these two modalities is achieved at the matching score level. Experimental results show that the highest identification performance is obtained in the dual-modality system, when its performance is compared with those of the systems using these two modalities separately. Furthermore, it is demonstrated that EEG is an appealing complementary modality to enhance the anti-spoofing capability of conventional biometrics-based identification systems.
Keywords- Biometrics; Electroencephalogram; EEG; fingerprint verification; multimodal; fusion
I. INTRODUCTION
In recent years, biometrics verification systems are used more commonly and expected to be more reliable. However, the conventional biometrics, such as face, fingerprint, palm-print, voice, iris, and signature, share a common shortcoming. That is, they can be produced by a person under coercion [1]. There are increasing discussions about how to improve the security and privacy of biometric technology. One solution can be liveness detection, which aims to determine if the captured biometrics are actually from the claimed person who is alive when conducting the identification [2][3]. This liveness detection should be easy to implement in the form of either a hardware or software system, and should own the properties that are difficult to imitate [4].
Since each biometric modality has its advantages and disadvantages, a biometric authentication system based upon two or more biometric technologies can reinforce the strengths, meanwhile alleviate the drawbacks, of individual modality. Multimodal biometric systems integrate data from different biometric sources, which can sometimes lead to better recognition accuracy compared with the unibiometric systems [30].
Since only alive persons can generate human brain wave, which can be recorded in the form of EEG signals, EEG is a natural candidate for liveness detection, and is an attractive
complement modality to the existing unibiometric systems [2]. However, whether EEG can be a practical biometric modality has been widely questioned. This is partially because almost all the published studies on EEG-based biometrics used medical EEG recording equipment. The conventional medical EEG recording systems are generally very expensive. Also, using these systems remains a time-consuming process. For example, they frequently require properly preparing human scalp before attaching the electrodes, and the electrodes in these medial EEG systems generally need to apply conductive gel between electrodes and the human scalp to record signal with acceptable quality. These equipment restrictions hinder the applications of EEG-based personal identification system in practice. Fortunately, the development of EEG recording technology brought out portable EEG recording systems with a much simpler recording procedure in the past decade. In [5], it was demonstrated that the single channel EEG recorded by one model of the portable equipment has the potential to be used for personal identification.
In this paper, we proposed a dual-biometric-modality personal identification system based on the fingerprint and EEG. Since a subject's fingerprint and EEG are information sources without any evident correlation, we expect integrating these two biometric modalities can outperform either one. Secondly, fingerprint modality is widely used for its relatively high verification accuracy, but is also more and more subject to sophisticated spoofing attempts due to its increasing popularity. Meanwhile, EEG is hard to reproduce under coercion, and is ideal for liveness detection. Therefore, using EEG as a complementary modality to the existing fingerprint identification systems can be a natural solution to an enhanced fingerprint identification system with the anti-spoofing capability.
The rest of the paper is organized as follows. Section 2 and Section 3 describe the fingerprint-based and the EEG-based identification systems respectively. The fusion framework is presented in Section 4, while the experimental results are provided in Section 5, which is followed by the conclusions in Section 6.
978-1-4244-7580-3/10/$26.00 ©2010 IEEE
II. FINGERPRINT-BASED IDENTIFICATION SYSTEM
Fingerprints are the pattern of ridges and valleys on the surface of human fingers. Lifetime “immutability” and “uniqueness” are fingerprint’s distinctive characteristics that have determined the use of fingerprints as one of the most reliable techniques for personal identification [6]. Significant progress has been made in designing automatic fingerprint identification systems over the past years. Fingerprint matching methods can be coarsely classified into five families: correlation-based matching, minutiae-based matching, ridge-based matching, texture-based matching, and mixture feature-base matching. In correlation-based matching, two fingerprint images are superimposed and the correlation between corresponding pixels is computed for different alignments [7], [8]. Minutiae-based fingerprint matching is actually a point-pattern matching problem and widely used in fingerprint matching [6], [9], and [10]. Most of fingerprint matching algorithms that provide real-time processing and high confidence are so far based on minutiae-based matching. Ridge-based matching uses the ridge image to match the query and template fingerprints [11], [12]. Some texture-based fingerprint matching methods can be found in literature [13], [14], mixture feature-base matching methods [15], [16] use the mixture feature like minutiae with ridge, minutiae with texture, etc..
A generic fingerprint authentication system consists of two parts: enrollment and verification. In enrollment, the collected raw fingerprint image is preprocessed, and the features are extracted and stored. In verification, the similarity between the enrolled fingerprint features and the features computed from the input fingerprint is examined.
A. Data Collection
Forty healthy volunteers’ fingerprint images were collected (29 male and 11 female) using optical fingerprint sensor "U.are.U 4000" by Digital Persona. The image size is 328×356 with 500 dpi. Forefingers of right hand of each volunteer were acquired, and seven impressions were acquired of the forefinger for each volunteer. We did not make any efforts to control image quality and the sensor platens were not systematically cleaned. Some fingerprint samples are shown in Fig. 1.
Figure 1. Samples of the collected fingerprint images.
B. Fingerprint Preprocessing
In our algorithm, the fingerprint preprocessing method is based on our previous paper [15], and the included stages are
shown in Fig.2. An example of the preprocessing result in each stage is given in Fig.3.
Figure 2. The flowchart of the fingerprint preprocess.
After the orientation field is computed, the fingerprint image is segmented into the foreground and background regions (shown in Fig. 3 (b)), and the following processing focuses on the foreground area to save the processing time. The directional filtering aims to remove noises and some distortion introduced in collecting the fingerprint image (shown in Fig. 3 (c)). Then the enhanced image is binarized and thinned to extract the minutiae. Due to the case of different noises (e.g. blurred prints smeared with under-inking or over-inking, uneven pressure, worm prints, etc.), the thinned image may contain a large number of false minutiae which may highly decrease the matching performance of the system. Therefore, the post-processing is used to purify the extracted minutiae (shown in Fig. 3 (d)). Here, each minutia is stored as (x, y, θ), where (x, y) is the minutia location and θ is the local ridge orientation of the minutia.
(a) Original image (b) Segmented fingerprint
(c) Enhanced image (d) Extracted minutiae
Figure 3. Fingerprint image preprocessing results.
C. Minutiae-based Fingerprint Matching
The matching procedure is divided into three stages: coarse matching, fusing and fine matching as described in [15]. Coarse matching is first performed from a number of
seeds. The results are then fused to obtain a constrained corresponding relationship between the query and template minutiae sets considering the similarity of the recorded minutiae along the associated ridges. Through using the distortion-tolerant support degree of the elements in the constrained relations, the one-to-one correspondence is finally determined by comparing the local structures’ similarity which is measure by SVM. An example of the matched fingerprint images is given in Fig.4, and the corresponding matched minutiae are signed as n and n’ (n=1, 2, …, 12).
Figure 4. An example of matched corresponding minutiae.
III. EEG-BASED VERIFICATION SYSTEM
In the past ten years, several studies have been proposed using brain waves, e.g. EEG as a biometric modality [2]–[5], [17]–[29]. Related research could be classified in two categories including EEG-based and Visual Evoked Potential (VEP)-based [5]. EEG-based methods use restful EEG signals recorded in case the subject opens/closes his/her eyes in relaxation [2]–[5], [17], [18], the other is to use the response when some mental tasks or stimuli are given to the subject [19]–[24]. In practical personal identification, we think EEG-based method is more convenient than VEP-based system where additional display terminal is necessary, and some mental tasks or stimuli should be arranged. Therefore, we
focused on EEG-based biometrics in [5], where single channel portable EEG equipment was applied to study personal identification based on a well-designed EEG recording experiment. Results of these studies provided supporting evidence that EEG-based personal identification from proof-of-concept to system implementation is promising.
The generic framework of EEG-based identification system is the same as other biometric identification system including enrollment and matching. In enrollment, the EEG features are extracted and stored as a template in database. In matching, the similarities between testing EEG and the stored EEGs are estimated. The details will be addressed as follows.
A. Data Collection
All EEG signals used in this paper were recorded using HXD-I portable equipment1 (see Fig. 5), which collects signals from FP1 electrode without requiring any skin preparation or conductive pastes. The sampling rate is 200Hz, and reference sensors are placed at both earlobes. Therefore, it is quite unobtrusive, fast and easy to place, which makes it possible to be used in personal identification.
Figure 5. Portable EEG recording equipment (left) and the recording scene
(right)
The EEG data collection procedure strictly followed a well-documented Standard Operation Procedure (SOP). Forty healthy volunteers’ EEG signals were collected (29 male and 11 female). All volunteers who were as the same in fingerprint collection were screened to ensure that they were healthy and not under any medication. The recording environment is quiet, normal temperature and daylight. Volunteer sits on a comfortable sofa. After a standard instruction was read, a segment of five-minute restful EEG signal was recorded when the volunteer kept his/her eyes close (see Fig. 5). In each recording event, the FP1 electrode was placed on the standard FP1 position with a little random skewing.
The data collection follows a two-period crossover procedure. Each volunteer’s EEG was recorded in two separate days, which was considered as two periods. In one period, a cup of pure water was drunk. In the other period, the same amount of coffee was drunk at the same time of the day. Here we use coffee to represent a diet that potentially has an impact on EEG. In each period, 6 EEG sessions were recorded at different time points, including a pre-dose session and five post-dose sessions. Immediately after the pre-dose EEG recording session was finished, the volunteer was instructed to drink a cup of coffee or water, i.e. conducting the drinking
1 http://www.easymonitor.com.cn
event. Then, five post-dose sessions were recorded at 0.5 hour, 1 hour, 1.5 hours, 2 hours, and 2.5 hours after the drinking event. Each five-minute recording is an EEG event here. There are totally 480 EEG events (i.e. 1 event per EEG session, 6 sessions per period, 2 periods per subject, and 40 subjects) in our dataset.
B. Feature Set
The feature set we used is composed of AR model parameters and Burg Power Spectrum Density (PSD). In our previous experiment, the performance of an AR model with an order ranging from 10 to 50 was tested, and the optimal order was found to be 19 [5]. Furthermore, a PSD with full frequency range was also used. However, because the PSD below 4Hz is frequently contaminated by ocular artifacts, and the PSD above 33Hz is affected by a system specific notch filter, we finally decided to incorporate the PSD at a frequency range from 5 to 32Hz into our feature set and each frequency bin was presented by four points. Fig. 6 shows the PSDs of two randomly chosen volunteers with blue and red lines respectively. Each volunteer has 12 EEG events. It is clear to see the discriminant properties of different persons.
Figure 6. PSDs of all samples from two volunteers .
C. Naive Bayes Classifier
Though KNN is demonstrated to be a classifier of extremely good classification performance on the proposed feature set [5], we did not adopt it in the identification system in this paper for it has several inherent drawbacks on practical applications. KNN classifier calculates all distances between testing sample and the stored templates which leads to the extremely computational intensive and requires large number of memories; KNN is likely to classify the testing sample to the class with more training samples. But the uniform distribution of training samples in different classes is hard to guarantee in fact. Furthermore, KNN is suitable for classification instead of verification, in that it cannot deny the subject which is not included in the database, because the testing sample will be assigned to the nearest neighborhood (the class with minimum distance). Therefore, the KNN classifier is replaced by a naive Bayes classifier in this paper.
In our experiment, six recording sets were randomly selected for each subject as the stored templates, and the others were remained as the testing sets. A naive Bayes model was trained by fitting the templates. The similarity measure is based on the posterior probability.
For genuine match, each of the selected 6 EEG events is compared with the stored template, where the number of matches is 6×40 = 240. For impostor match, the random selected one EEG event of one volunteer is compared with the stored templates of other volunteers, where the number of matches is 2
40 780C = . In order to guarantee the robustness of the evaluation, the templates and testing sets are generated randomly 100 times. The result is the summary statistics of 100 splits.
Considering that the diet and personal circadian are crucial factors affecting the EEG signals compared to other factors, we addressed how much the diet and times of day affected a person’s EEG recordings [34]. It has shown that recording factors have some effects on the system performance, and if more diversity samples were included in modeling, the higher accuracy would be obtained.
IV. MULTIMODAL IDENTIFICATION SYSTEM
Multimodal biometric systems are expected to be more reliable due to the presence of multiple, independent pieces of evidence, and could provide more difficult to spoof the multiple biometric modalities of a genuine user simultaneously. Some literatures deal with the multimodal biometric systems in recent years. In [29], authors give the overview of the multi-biometrics. In [30], it showed that the assumption of independence between matchers has a significant negative impact on the performance of the likelihood ratio fusion scheme only when (i) the dependence characteristics among genuine match scores is different from that of the impostor scores and (ii) the individual matchers are not very accurate. In [31], likelihood ratio-based score fusion, which was originally designed for verification systems, can be extended for fusion in the identification scenario under certain assumptions. In [32], the author described a multi-biometric system that uses the face and palm-print traits of an individual for identification. A multi-biometric system based on the geometrical dimensions of a human hand, such as hand geometry, finger stripe geometry or palm geometry is proposed [33] for human verification.
The flowchart of our proposed dual-biometric-modality identification system is shown in Fig. 7. It is divided into three parts: fingerprint identification, EEG identification and a dual-modal identification using fingerprint and EEG based on the fusion at the matching score level.
The information used in multi-biometric system can be consolidated at various levels, including fusion at the feature extraction level, at the matching score level, and at the decision level. Here, we took the strategy of fusion at the matching score level, because it assumes that the component matchers are statistically independent to simplify the design of the fusion algorithm, and offers the best trade-off in terms of the information content and the ease in fusion.
Figure 7. A dual-biometric-modality identification system framework.
In our scheme, the matching scores of the fingerprint and EEG were both normalized into [0, 1] using a min-max normalization method. The sum-rule based fusion scores were calculated with equal weights assigned to each modality.
In fingerprint verification, our collected database consists of 280 fingerprint images from 40 fingers (seven impressions per finger). Because we adapt the matching score level fusion, the fingerprint-based test should match the EEG-based test. Therefore, there are the same 6×40=240 genuine matching. For impostor match, as the same, the first impression of each finger is compared with the first impression of other fingers, and the total number is 2
40 780C = .
The equal error rates (EER) are list in Table 1, and Receiver Operating Characteristic (ROC) curves are shown in Fig. 8.
TABLE I
COMPARISON OF THE MATCHING PERFORMANCES
Fingerprint EEG Fusion
EER of the proposed method (%) 2.71 4.16 1.12
Figure 8. ROC curves.
Experimental results indicate that the performance of
multimodal biometric identification outperforms to the unimodal system when scores are combined using simply sum-rule.
V. CONCLUSION
Multimodal biometric systems are attractive due to their potential to improve the matching performance of single-modality identification system, and to provide an effective solution for anti-spoofing capability. This paper proposed a dual-biometric-modality personal identification system, which used both the fingerprint and the EEG technologies to achieve both high identification performance, and an effective anti-spoofing capability. According to our knowledge, this study represents the first effort to fuse the widely adopted fingerprint technology with a novel biometric modality--EEG. Experimental results suggest that the highest identification performance is obtained in the proposed dual-biometric-modality system, compared with the performance of the systems based on either fingerprint or EEG alone.
Furthermore, it is implied that EEG is an appealing complementary modality to add the anti-spoofing capability to the existing identification systems based on conventional biometric technologies, such as fingerprint.
Our future work will involve improving the fusion schemes, and further enhancing the robustness of the EEG modality.
ACKNOWLEDGMENT
This work is supported by The Key Project of The Ministry of Education of P. R. China (108012) and The National Natural Science Foundation of China (90920001, 60772114).
The authors would like to acknowledge Mr. Yibin Wu at China Easymonitor Technology Corporation (http://www.easymonitor.com.cn) for provision of the EEG recording equipment.
REFERENCES
[1] A. K. Jain, Biometric system security. http://biometrics.cse.msu.edu, 2006.
[2] V. Valencia, “Biometric liveness testing, in biometrics”, J. D. Woodward, Ed. New York, 2002, Chapter 8.
[3] S. A. C. Schuckers, “Spoofing and anti-spoofing measures,” Information Security Technical Report, 2002, vol. 7, no. 4, pp. 56 – 62.
[4] D. Martin, D. Lodrova, “Liveness detection for biometric systems based on papillary lines,” Int. J. Security and Its Applications, vol. 2, no. 4, Oct. 2008.
[5] F. Su, L.-W. Xia, A. Cai, J.-S. Ma, Y.-B. Wu, “EEG-based personal identification: from proof-of-concept to a practical system,” Proc. Int. Conf. Pattern Recognition, Istanbul, Turkey, Aug., 2010.
[6] A. K. Jain, L. Hong, R. Bolle, “On-line fingerprint verification,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 4, 1997.
[7] C. Wilson, C. Watson, E. Paek, “Effect of resolution and image quality on combined optical and neural network fingerprint matching,” Pattern Recognition, vol. 33, no. 2, 2000, pp. 317–331.
[8] T. Hatano, T. Adachi, S. Shigematsu, H. Morimura, S. Onishi, Y. Okazaki, et al., “A fingerprint verification algorithm using the differential matching rate,” Proc. Int. Conf. Pattern Recognition, Quebec City, 2002, pp. 799–802.
[9] X. Jiang, W.-Y. Yau, “Fingerprint minutiae matching based on the local and global structures,” Proc. Int. Conf. Pattern Recognition, Barcelona, 2000, pp. 1038–1041.
[10] D. Lee, K. Choi, J. Kim, “A robust fingerprint matching algorithm using local alignment,” Proc. Int. Conf. Pattern Recognition, Quebec City, 2002, pp. 803–806.
[11] X.-H. Xie, F. Su, A. Cai, “Ridge-based fingerprint matching,” Proc. Int. Conf. Biometric Authentication, Hong Kong, 2006, pp. 273–279.
[12] J.-J. Feng, Z.-Y. Ouyang, A. Cai, “Fingerprint matching using ridges,” Pattern Recognition, vol. 39, no. 11, 2006, pp. 2131–2140.
[13] A. Ross, A. K. Jain, J. Reisman, “A hybrid fingerprint matcher,” Proc. Int. Conf. Pattern Recognition, Quebec City , vol. 3, 2002, pp. 795–798.
[14] A. K. Jain, S. Prabhakar, L. Hong, S. Pankanti, “Filterbank-based fingerprint matching,” IEEE Trans. Image Processing, vol. 9, no. 5, 2000, pp. 846–859.
[15] F. Su, X.-H. Xie, J.-J. Feng, A. Cai, “Fingerprint matching using SVM-based similarity measure,” Chinese J. Electronics, vol. 16, 2007, Part 3, pp. 459–463.
[16] J.-J. Feng, “Combining minutiae descriptors for fingerprint matching,” Pattern Recognition, vol. 41, 2008, pp. 342–352.
[17] M. Poulos, M. Rangoussi, N. Alexandris, “Neural network based person identification using EEG features,” IEEE Int. Conf. Acoustics, Speech, and Signal Processing, vol. 2, 1999, pp. 1117–1120.
[18] R. B. Paranjape, J. Mahovsky, L. Benedicenti, Z. Koles, “The electroencephalogram as a biometric,” Proc. Canadian Conf. Electrical and Computer Engineering, vol. 2, 2001, pp. 1363–1366.
[19] C. R. Hema, M. P. Paulraj, K. Harkirenjit, “Brain signatures: a modality for biometric authentication,” Int. Conf. Electronic Design, 2008, pp. 1–4.
[20] N. Markus, W. Marc, S. Johannes, “Test–retest reliability of resting EEG spectra validates a statistical signature of persons,” Clinical Neurophysiology, vol. 118, 2007, pp. 2519–2524.
[21] M. Chisei, B. Sadanao, N. Isao, “Biometric person authentication using new spectral features of electroencephalogram (EEG),” Int. Symp. Intelligent Signal Processing and Communication Systems, 2008.
[22] S. Sun, “Multitask learning for EEG-based biometrics,” Int. Conf. Pattern Recognition, 2008, pp. 1–4.
[23] H. Chen, X.-D. Lv, Z. J. Wang, “Hashing the mAR coefficients from EEG data for person authentication,” IEEE Int. Conf. Acoustics, Speech and Signal Processing, 2009, pp. 1445–1448.
[24] A. Yazdani, A. Roodaki, S. H. Rezatofighi, K. Misaghian, S. K. Setarehdan, “Fisher linear discriminant based person identification using visual evoked potentials,” IEEE Int. Conf. Signal Processing, 2008, pp. 1677–1680.
[25] R. Palaniappan, D. P. Mandic, “Biometrics from brain electrical activity: a machine learning approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 4, 2007, pp. 738–742.
[26] S. Marcel, J. R. Millan, “Person authentication using brainwaves (EEG) and maximum a posteriori modal adaptation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 4, 2007, pp. 743–748.
[27] N. Isao, B. Sadanao, M. Chisei, “EEG based biometric authentication using new spectral features,” Int. Symp. Intelligent Signal Processing and Communication Systems, 2009.
[28] S. Sanei, J. Chambers, “EEG Signal Processing,” NewYork:Wiley, 2007. [29] A. Ross, K. Nandakumar, A. K. Jain, Handbook of Multibiometrics.
Springer, 2006. [30] K. Nandakumar, A. Ross, A. K. Jain, “Biometric fusion: does modeling
correlation really matter?” Proc. 3rd Int. Conf. Biometrics: Theory, Applications and Systems, Washington DC, Sep 2009.
[31] K. Nandakumar, A. K. Jain, A. Ross, “Fusion in multibiometric identification systems: what about the missing data?” Proc. 3rd Int. Conf. Biometrics, Alghero, Jun 2009.
[32] Lu Cheng, W. Jisong, Q. Miao, “Multimodal biometric identification approach based on face and palmprint,” 2nd Int. Symp. Electronic Commerce and Security, 2009.
[33] F. Anwar, M. A. Rahman, “Multibiometric systems based verification technique,” European J. Scientific Research, vol. 34, no. 2, 2009, pp. 260–270.
[34] F. Su, L.-W. Xia, A. Cai, J.-S. Ma, “Evaluation of recording factors in EEG-based personal identification: a vital step in real implementation,” Int. Conf. Systems, Men, and Cybernetics, 2010, in press.
