Driver Drowsiness Detection

Laveen Prabhu Selvaraj (Im No. 303657)

Abstract

Sleep disorders and various common acute directly or indirectly affect the quality and

quantity of one‘s sleep or otherwise cause excessive daytime fatigue. About 29 600

Norwegian accident-involved drivers received a questionnaire about the last accident

reported to their insurance company. About 9200 drivers (31%) returned the questionnaire.

The questionnaire contained questions about sleep or fatigue as contributing factors to the

accident. In addition, the drivers reported whether or not they had fallen asleep some time

whilst driving, and what the consequences had been. Sleep or drowsiness was a

contributing factor in 3.9% of all accidents, as reported by drivers who were at fault for the

accident. This factor was strongly over-represented in night-time accidents (18.6%), in

running-off-the-road accidents (8.3%), accidents after driving more than 150 km on one trip

(8.1%), and personal injury accidents (7.3%).The most frequent consequence of falling

asleep—amounting to more than 40% of the reported incidents—was crossing of the right

edge-line before awaking, whereas crossing of the centre line was reported by 16%. Drivers‘

lack of awareness of important precursors of falling asleep—like highway hypnosis, driving

without awareness, and similar phenomena—as well as a reluctance to discontinue driving

despite feeling tired are pointed out as likely contributors to sleep-related accidents. The

envisioned vehicle-based driver drowsiness detection system would continuously and

unobtrusively monitor driver performance (and ―micro-performance‖ such as minute steering

movements) and driver psychophysiological status (in particular eye closure). The system

may be programmed to provide an immediate warning signal when drowsiness is detected

with high certainty, or, alternatively, to present a verbal secondary task via recorded voice as

a second-stage probe of driver status in situations of possible drowsiness. The key

requirements and R&D challenges for a successful countermeasure include low

countermeasure cost, true unobtrusiveness, an acceptably-low false alarm rate, non-

disruption of the primary driving task, compatibility and synergy with other IVHS crash

avoidance countermeasures, and a warning strategy that truly sustains driver wakefulness or

convinces him/her to stop for rest.

1. Introduction

Fatigued or drowsy drivers have long been

acknowledged to constitute a potential

traffic safety hazard, and several research

studies have addressed various aspects of

the problem. Brown (1994) has presented

a comprehensive review and discussion of

the research literature on the nature of

fatigue and its effects on driver behavior

and traffic accidents, on the basis of which

he concluded that ‗fatigue is insufficiently

recognized and reported as a cause of

road accidents‘. In discussing the effects

of fatigue, Brown further points out that the

main effect is ‗a progressive withdrawal of

attention from road and traffic demands‘. A

most extreme form of withdrawal of

attention is obviously the closing of eyes

due to sleepiness. To prevent accidents

related to drowsiness and sleeping behind

the wheel, it is important to acquire

precise knowledge about the extent of the

problem as well as its preconditions and

consequences. The following issues, all of

which are investigated empirically in the

present study, are considered relevant for

a better understanding of sleep-related

accidents.

1. What is the proportion of accidents

caused by fatigue or sleep behind the

wheel, and what are the most likely types

of accidents to occur under these

conditions?

2. How prevalent is the problem of actually

falling asleep while driving?

3. What are the most frequent

consequences of falling asleep while

driving?

4. To what extent is sleeping behind the

wheel related to characteristics of the

driver, the road and traffic conditions, and

the trip?

These drowsiness detection methods can

be categorized into three major

approaches:

• Active driving or on-board detection:

Preventing accidents caused by

drowsiness behind the steering wheel is

highly desirable but requires techniques

for continuously estimating driver‘s

abilities of perception, recognition and

vehicle control abilities. This paper

proposes methods for drowsiness

estimation that combine the

electroencephalogram (EEG) log subband

power spectrum, correlation analysis,

principal component analysis, and linear

regression models to indirectly estimate

driver‘s drowsiness level in a virtual-

reality-based driving simulator. Results

show that it is feasible to quantitatively

monitor driver‘s alertness with concurrent

changes in driving performance in a

realistic driving simulator.

• Imaging processing techniques: this

approach analyzes the images captured

by cameras to detect physical changes of

drivers, such as eyelid movement, eye

gaze, yawn, and head nodding. For

example, the PERCLOS system

developed by W. W. Wierwile et. al. used

camera and imaging processing

techniques to measure the percentage of

eyelid closure over the pupil over time.

The three-in-one vehicle operator sensor

developed by Northrop Grumman Co. also

used the similar techniques. Although this

vision based method is not intrusive and

will not cause annoyance to drivers, the

drowsiness detection is not so accurate,

which is severely affected by the

environmental backgrounds, driving

conditions, and driver activities (such as

turning around, talking, and picking up

beverage). In addition, this approach

requires the camera to focus on a relative

small area (around the driver‘s eyes). It

thus requires relative precise camera

focus adjustment for every driver.

• Physiological signal detection

techniques: this approach is to measure

the physiological changes of drivers from

biosignals, such as the

electroencephalogram (EEG),

electrooculograph (EOG), and

electrocardiogram (ECG or EKG). Since

the sleep rhythm is strongly correlated

with brain and heart activities, these

physiological biosignals can give accurate

drowsiness/sleepiness detection.

However, all the researches up to date in

this approach need electrode contacts on

drivers‘ head, face, or chest. Wiring is

another problem for this approach. The

electrode contacts and wires will annoy

the drivers, and are difficult to be

implemented in real applications.

2. Active Driving or On board Detection

Basic Concepts

As indicated earlier, the basic idea behind

vehicle-based detection is to monitor the

driver unobtrusively by means of an on-

board system that can detect when the

driver is materially impaired by

drowsiness. The concept involves sensing

various drivers related and driving related

variables. Computing measures from

these variables online and then using the

measures in a combined manner to detect

when drowsiness is occurring. Measures

are combined because no single

unobtrusive operational measure appears

adequate in reliably detecting drowsiness.

The most promising approach uses

mathematical optimization procedures to

develop algorithms with the highest

potential detection accuracy. Techniques

normally employed include multiple

regression and linear discriminated

analyses. More exotic techniques could

also be employed in the future, including

neural networks, pattern recognition and

fuzzy logic. Optimization of algorithms for

detection of drowsiness requires a

definitional measure of ―actual‖

drowsiness. Such a measure may be

based on physiological, performance, or

subjective attributes and need not be

obtainable operationally. However, the

measure must be available in experiments

so that operational detection algorithms

can be ―trained‖ to indicate the value of

the definitional measure. This concept is

depicted in

Figure 1.

Figure 1. concept of using operational

measures to predict definitional measures

of drowsiness

On the left are measures that can be

obtained in the driving environment. These

measures (with the exception of

secondary task measures) are obtainable

operationally from the vehicle without

disturbing the driver. They can be used in

various combinations for algorithm

development. On the right are various

candidate definitional measures. AVEOBS

is an observer rating measure, EYEMEAS

and PERCLOS are measures of slow eye-

closure, and NEWDEF is a measure

composed of slow eye-closure, various

EEG waveform amplitudes (Alpha, Beta,

and Theta), and mean heart rate. A given

algorithm would be directed at indicating

the level of only one definitional measure,

or possibly a linear combination of them.

In any case, operationally available

measures (on the left) are used to detect

the level of the definitional measure of

drowsiness (on the right), with thresholds

set to indicate when drowsiness has

exceeded a pre-specified level.

Figure 2: Application of on board

detection in Mercedes

On-Board Detection System

Figure 3: Block Diagram of on board

detection

The on-board drowsiness detection

system would gather signals from sensors

on the vehicle, process these signals into

measures, and then compute the

algorithm (or algorithms) to determine if

the drowsiness threshold has been

exceeded.

Figure 3 shows a block diagram of the

envisioned system. Aspects of the

envisioned system already determined

through research efforts include the

following:

Signals input to the microcomputer

will include:

Steering-related signals

o A lateral

accelerometer-

related signal

and

o A lane position

signal (assumes

availability of

machine vision

technology for

optical tracking

of existing

highway lane

markings).

Measures will be computed using

six-minute running averages

(which provide the best prediction

accuracy).

An adjustable drowsiness

threshold feature will allow

sensitivity to be set according to

conditions.

. A step-up/step-down routine will ensure

that, when all incoming signals are valid,

the best algorithm is used. When one or

more of the incoming signals is invalid (for

example, inability to establish a lane

track), then the best algorithm excluding

the invalid signal(s) would be used. This

procedure will ensure that at least one

detection algorithm is always capable of

being computed.

A ―baselining‖ procedure will be used to

tailor detection algorithms to the individual

driver. It will record each driver‘s

performance measures on-line initially and

then subtract such values from all

subsequent values. Accordingly,

measures obtained are actually deviations

from the driver‘s own baseline.

Domain of Application

On-board drowsiness detection systems

will be applicable primarily to driving on

rural and other ―open‖ highways, such as

limited-access highways, at speeds at or

above 50 mph. There are two reasons for

limiting the drowsiness detection system

to this domain. First, as discussed earlier,

most drowsiness-related crashes occur on

these roads at these speeds. Second, it

appears that this domain is the one in

which feasibility is maximized. The

influences of stop-and-go traffic, traffic

signals, turning maneuvers, etc., would

probably introduce sufficient ―noise‖ into

the detection process that unobtrusive

detection would be unfeasible. As we can

see, we have the fortuitous circumstance

of ―feasibility in the most needed domain,‖

or in other words ―the coin we are

searching for was lost under the

streetlamp, where the light gives us the

best chance of finding it.‖

Nature and Accuracy of Algorithms

To provide a better idea of what a typical

algorithm looks like and what its

anticipated level of accuracy would be, a

specific algorithm will be described. It is

one of perhaps 120 that were recently

derived in a major, moving-base driving

simulator experiment using sleep deprived

drivers (Wreggit, Kim, and Wierwille,

1993). The algorithm was derived using

multiple regression analysis with

PERCLOS (the proportion of total time

that the driver‘s eyelids are closed 80% or

more) as the definitional measure. Figure

4 shows the actual values of PERCLOS

(open circles) and the algorithm-predicted

values (closed triangles) for 12 driver

subjects. Each interval on the abscissa

corresponds to a six-minute average, with

25 intervals per driver-subject. Increasing

values of the ordinate represent increasing

drowsiness levels. The algorithm generally

does an excellent job of mimicking the

values of

PERCLOS, particularly in the intermediate

ranges of PERCLOS where the threshold

would most likely be set Figure 4 shows

the specific thresholds used on the

definitional measure (PERCLOS) for the

determination of prediction accuracy. The

data in Figure 4 correspond to ―circle‖

values in Figure 4.

. Figure 4 algorithm-predicted values

(closed triangles) for 12 driver subjects

As can be seen, two thresholds have been

specified, thus breaking the plot into three

regions: alert, questionable, and drowsy.

When these thresholds are applied to the

output of the detection algorithm, which

provides an assessment of accuracy. In

the table, the boldface diagonal values

show the number of correct classifications.

The off-diagonal elements represent

errors, and in particular, the upper-left and

lower-right cells represent large errors. As

can be seen, three intervals were

classified (predicted) as drowsy when the

driver was observed as alert (false

alarms), and another three intervals were

classified as alert when the driver was

observed as drowsy (failure to detect).

Since there were 300 intervals, two

percent were seriously misclassified,

resulting in an apparent accuracy rate of

0.98 (for large errors).Of course there

were smaller classification errors as well,

but these are not as serious -- for

example, the 16 intervals in which the

system diagnosed a drowsy driver when

the driver‘s actual status was

―questionable‖ (i.e., somewhat drowsy).

Overcoming the False Alarm Problem

As already indicated, a major research

objective will be to overcome the false

alarm problem inherent to the

identification/diagnosis of low-probability

events. Since drowsiness is infrequent in

relation to all time spent driving, false

alarm rates must be very low. If not, the

number of false alarms will greatly

outnumber correct detections (―hits‖), even

if drowsiness is correctly detected with

100% accuracy (Knipling, 1993). This

problem may be overcome through

refinements to the performance

measurement algorithms, addition of

qualitatively different measures (i.e., direct

psychophysiological measures and/or

secondary tasks), and the use of graded

alarm intensities for different degrees of

drowsiness or levels of certainty. In

particular, the false alarm problem

appears less daunting from the

perspective of multiple degrees of

alertness and intensities of

warnings/advisories. Figure 5, which is

similar in concept to the two-threshold

algorithm concept, shows a theoretical

relation between ―actual‖ drowsiness level

(and thus actual risk of loss-of-alertness)

and ―operational‖ drowsiness level as

measured/derived by a detection system.

Figure 5. Three-level detection matrix

Three levels of ―actual‖ and ―operational‖

drowsiness are shown in the figure, but

note that dashed lines are used for the

three ―actual‖ drowsiness levels since the

variable represents a continuum without

qualitative breakpoints. Since the system

is not perfect, its data points would form

an ellipse rather than a straight line. Within

this scheme, zones G, E, and C represent‘

perfect classification, zones D, B, H, and F

represent small is classifications (or ―half

right‖ classifications), and zones A and I

represent large misclassifications. Drowsy

driver detection algorithms must be refined

to a point where zones A and I are very

small or non-existent. The effects of small

misclassifications (Zones D, B, H, and F)

on crash prevention, driver performance,

and driver acceptance must be

determined through further research. For

example, the ―half-false alarm‖ zones D

and B may be a source of irritation to

drivers or, on the other hand, they may

have the positive effect of reassuring the

driver that the system is functioning

continuously. Another way to increase

detection and reduce false alarms might

be to consider not just the current

measurement time interval but also trends

evident from preceding intervals. Were

there early signs of developing drowsiness

based either on the overall operational

measure or among specific indicators?

Fuzzy logic may be employed to further

enhance the accuracy of diagnosis by

considering the driver‘s recent time-history

of drowsiness.

3. Image processing

Image processing is a method to convert

an image into digital form and perform

some operations on it, in order to get an

enhanced image or to extract some useful

information from it. It is a type of signal

dispensation in which input is image, like

video frame or photograph and output may

be image or characteristics associated

with that image. Usually Image

Processing system includes treating

images as two dimensional signals while

applying already set signal processing

methods to them.

It is among rapidly growing technologies

today, with its applications in various

aspects of a business. Image Processing

forms core research area within

engineering and computer science

disciplines too.

Image processing basically includes the

following three steps.

· Importing the image with optical

scanner or by digital photography.

· Analyzing and manipulating the

image which includes data

compression and image

enhancement and spotting patterns

that are not to human eyes like

satellite photographs.

· Output is the last stage in which

result can be altered image or

report that is based on image

analysis.

Purpose of Image processing

The purpose of image processing is

divided into 5 groups. They are:

1. Visualization - Observe the objects

that are not visible.

2. Image sharpening and restoration -

To create a better image.

3. Image retrieval - Seek for the image

of interest.

4. Measurement of pattern – Measures

various objects in an image.

5. Image Recognition – Distinguish the

objects in an image.

Each application that benefit from smart

video processing has different needs, thus

requires different treatment. However,

they have something in common moving

objects. Thus, detecting regions that

correspond to moving objects such as

people and vehicles in video is the first

basic step of almost every vision system

since it provides a focus of attention and

simplifies the processing on subsequent

analysis steps. Due to dynamic changes in

natural scenes such as sudden

illumination and weather changes,

repetitive motions that cause clutter (tree

leaves moving in blowing wind), motion

detection is a difficult problem to process

reliably. Frequently used techniques for

moving object detection are background

subtraction, statistical methods, temporal

differencing and optical flow.

Figure6. A generic framework for smart

video processing algorithms.

There are various techniques for moving

object detection and tracking like optical

flow, low change of illumination,

segmentation, background subtraction,

frame difference, etc. The problem is

formulated in a sequential manner. It has

different step with different set of operation

will take place at each step and the output

of that step will be used as the input to the

other step .Each step is in charge of

specific function which it will perform on

each frame of the video sequence and the

final result of that step will be used in the

another step and each step will follow the

same things. The last step will give the

final output in the form of a video in a well

structured way. The formulation of step

are defined as follows-

1. Take video from Vision System.

2. Read 1st image to avi read that is

reference image

3. Read other image

4. Take subtraction of them and set

Thresholding

5. Applied Gaussian filter for noise remove

6. Applied morphological operation like

dilation and erosion for small noise

removes

7. Fill holes in resulted image

8. Take label connected component with

its properties like bounding box, centroid

and area of all no of object move in this

scene.

9. For i=1:n % n is no of object move

A=(length of object) Find(L==1) % find

white pixel whose length is A

If (A>100 && A<8000)

Then draw rectangle plot centroid

of that rectangle end to end

10. Take distance of centroid to

reference point

11. Take velocity estimation by ratio of

distance to time per Frame.

12. Take acceleration estimation by ratio

of velocity to time per Frame

Head movement measures

Head movement was measured using an

accelerometer that has 3 degrees of

freedom. This three dimensional

accelerometer has three one dimensional

accelerometers mounted at right angles

measuring accelerations in the range of 5g

to +5g where g represents earth

gravitational force.

Facial Action Classifiers

The facial action coding system (FACS)

[12] is arguably the most widely used

method for coding facial expressions in

the behavioral sciences. The system

describes facial expressions in terms of 46

component movements, which roughly

correspond to the individual facial muscle

movements. An example is shown in

Figure 7. FACS provides an objective and

comprehensive way to analyze

expressions into elementary components,

analogous to decomposition of speech

into phonemes. Because it is

comprehensive, FACS has proven useful

for discovering facial movements that are

indicative of cognitive and affective states.

In this paper we investigate whether there

are Action units (AUs) such as chin raises

(AU17), nasolabial furrow

deepeners(AU11), outer(AU2) and inner

brow raises (AU1) that are predictive of

the levels of drowsiness observed prior to

the subjects falling sleep.

Figure 7. Example facial action

decomposition from the Facial Action

Coding System.

In previous work we presented a system,

named CERT, for fully automated

detection of facial actions from the facial

action coding system [10]. The workflow of

the system is based is summarized in

Figure 3. We previously reported detection

of 20 facial action units, with a mean of

93% correct detection under controlled

posed conditions, and 75% correct for less

controlled spontaneous expressions with

head movements and speech. For this

project we used an improved version of

CERT which was retrained on a larger

dataset of spontaneous as well as posed

examples. In addition, the system was

trained to detect an additional 11 facial

actions for a total of 31 (Table 1). The

facial action set includes blink (action unit

45), as well as facial actions involved in

yawning (action units 26 and 27). The

selection of this set of 31 out of 46 total

facial actions was based on the availability

of labeled training data. The facial action

detection system was designed as follows:

First faces and eyes are detected in real

time using a system that employs boosting

techniques in a generative framework [13].

The automatically detected faces are

aligned based on the detected eye

positions, cropped and scaled to a size of

96 × 96 pixels and then passed through a

bank of Gabor filters. The system employs

72 Gabor spanning 9 spatial scales and 8

orientations. The outputs of these filters

are normalized and then passed to a

standard classifier. For this paper we

employed support vector machines. One

SVM was trained for each of the 31 facial

actions, and it was trained to detect the

facial action regardless of whether it

occurred alone or in combination with

other facial actions. The system output

consists of a continuous value which is the

distance to the separating hyper plane for

each test frame of video. The system

operates at about 6 frames per second on

a Mac G5 dual processor with 2.5 ghz

processing speed.

Figure 8. Overview of fully automated

facial action coding system.

Drowsiness prediction

The facial action outputs were passed to a

classifier for predicting drowsiness based

on the automatically detected facial

behavior. Two learning-based classifiers,

Adaboost and multinomial ridge

regression are compared. Within-subject

prediction of drowsiness and across-

subject (subject independent) prediction of

drowsiness were both tested.

Table 1. Full set of action units used for

predicting drowsiness

Within subject drowsiness prediction.

For the within-subject prediction, 80% of

the alert and non-alert episodes were

used for training and the other 20% were

reserved for testing. This resulted in a

mean of 19 non-alert and 11 alert

episodes for training, and 5 non-alert and

3 alert episodes for testing per subject.

The weak learners for the Adaboost

classifier consisted of each of the 30

Facial Action detectors. The classifier was

trained to predict alert or non-alert from

each frame of video. There was a mean of

43,200 training samples, (24+11)×60×30,

and 1440 testing samples, (5 + 3) × 60 ×

30, for each subject. On each training

iteration, Adaboost selected the facial

action detector that minimized prediction

error given the previously selected

detectors. Adaboost obtained 92% correct

accuracy for predicting driver drowsiness

based on the facial behavior.

Classification with Adaboost was

compared to that using multinomial ridge

regression (MLR). Performance with MLR

was similar, obtaining 94% correct

prediction of drowsy states. The facial

actions that were most highly weighted by

MLR also tended to be the facial actions

selected by Adaboost. 85% of the top ten

facial actions as weighted by MLR were

among the first 10 facial actions to be

selected by Adaboost.

Table 2. The top 5 most discriminant

action units for discriminating alert from

nonalert states for each of the four

subjects. A‘ is area under the ROC curve.

Across subject drowsiness prediction.

The ability to predict drowsiness in novel

subjects was tested by using a leave one-

out cross validation procedure.

Table 3. Performance for drowsiness

prediction, within subjects. Means and

standarddeviations are shown across

subjects.

The data for each subject was first

normalized to zero-mean and unit

standard deviation before training the

classifier. MLR was trained to predict

drowsiness from the AU outputs several

ways. Performance was evaluated in

terms of area under the ROC. For all of

the novel subject analysis, the MLR output

for each feature was summed over a

temporal window of 12 seconds (360

frames) before computing A‘. MLR trained

on all features obtained an A‘ of .90 for

predicting drowsiness in novel subjects.

Table 4. MLR model for predicting

drowsiness across subjects. Predictive

performance of each facial action

individually is shown.

Brain Waves

Relationship between the EEG

Spectrum and Subject Alertness

To investigate the fluctuations in driving

performance to concurrent changes in the

EEG spectrum, correlations between

changes in the EEG power spectrum and

driving performance to form a correlation

spectrum is measured. The spatial

distributions of these positive correlation

spectra on the scalp at dominant

frequency bins are investigated 7, 12, 16

and 20Hz, separately, as shown in Fig. 9.

The correlations are particularly strong at

central and posterior channels, which are

similar to the results of previous studies in

the drowsy experiments. The relatively

high correlation coefficients of EEG log

power spectrum with driving performance

suggests that using EEG log power

spectrum may be suitable for drowsiness

(micro-sleep) estimation, where the

subject‘s cognitive state might fall into the

first stage of the non-rapid-eye-movement

(NREM) sleep. To be practical for routine

use during driving or in other occupations,

EEG-based cognitive assessment

systems should use as few EEG sensors

as possible to reduce the preparation time

for wiring drivers and the computational

load for estimating continuously the level

of alertness in near real time. According to

the correlations shown in Fig. 9, we

believe it is adequate to use 2-channel

EEG signals having the highest correlation

coefficients to assess the alertness level

of drivers.

Figure 9: Scalp topographies for the

correlations between EEG power and

driving performance at dominant

frequencies 7, 12, 16, and 20 Hz,

computed separately for 40 EEG

frequencies between 1 and 40 Hz.

Next, we compared correlation spectra for

individual sessions to examine the stability

of this relationship over time and subjects.

The time interval between the training and

testing sessions of the lane-keeping

experiments distributes over one day to

one week long for the selected five

subjects. Fig. 10 plots correlation spectra

at cites Fz, Cz, Pz and Oz, of two

separate driving sessions with respect to

subjects A. The relationship between EEG

power spectrum and driving performance

is stable within the subjects, especially the

spectrum below 20 Hz. These analyses

provide strong and converging evidence

that change in subject alertness level

indexed by driving performance during a

driving task are strongly correlated with

the changes in the EEG power spectrum

at several frequencies at central and

posterior cites. This relationship is

relatively variable between subjects, but

stable within subjects. It is consistent with

the findings from a simple auditory target

detection task. These findings suggest

that information available in the EEG can

be used for real time estimation of

changes in alertness of human operators.

However, to achieve maximal accuracy,

the estimation algorithm should be

capable of adapting to individual

differences in the mapping between EEG

and alertness.

Figure 10. Correlation spectra between

the EEG power spectrum and the driving

performance at Fz, Cz, Pz, and Oz

channels in two separate driving sessions

with respect to subject A. Note that the

relationship between EEG power

spectrum and driving performance is

stable within this subject.

EEG-based Driving Performance

Estimation/Prediction

In order to estimate/predict the subject‘s

driving performance based on the

information available in the EEG power

spectrum, a 50-order linear regression

models with a least-square-error cost

function is used. We used only two EEG

channels with the highest correlation

coefficients in place of using all 33

channels to avoid introducing more

unexpected noise. Fig. 11 plots the

estimated and actual driving performance

of a session with respect to subject A. The

linear regression model in this figure is

trained with and tested against the same

session, i.e. within-session testing. As can

been seen, the estimated driving

performance matched extremely well with

the actual driving performance (r = 0.91).

Figure 11. Driving performance estimates

for a session with respect to subject A,

based on a linear regression (red line) of

PCA-reduced EEG log spectra at two

scalp sites, over plotted against actual

driving performance time series for the

session (solid line). The correlation

coefficient between the two time series is r

= 0.91.

When the model was tested against a

separate test session with respect to the

same subject, the correlation between the

actual and estimated driving performance

though decreased but remained high (r =

0.87) as shown in Fig. 12. Across 10

sessions, the mean correlation coefficient

between actual driving performance time

series for within session estimation is 0.85

± 0.11, whereas the mean correlation

coefficient for cross-session estimation is

0.82 ± 0.07. These results suggest that

continuous EEG-based driving

performance estimation using a small

number of data channels is feasible, and

can give accurate information about

minute-to-minute changes in operator

alertness.

Figure. 12. Driving performance estimates

for a test session, based on a linear

regression (red line) of PCA-reduced EEG

log spectra trained from a separate

training session with respect to the same

subject, over plotted against actual driving

performance time series of the test

session (solid line). The correlation

coefficient between the two time series is r

= 0.87. Note that the training and testing

data in this study were completely

disjoined.

Conclusion

This paper presented a system for

automatic detection of driver drowsiness

from video. Previous approaches focused

on assumptions about behaviors that

might be predictive of drowsiness. Here, a

system for automatically measuring brain

waves, facial expressions and vehicle

monitoring was employed to datamine

spontaneous behavior during real

drowsiness episodes. This is the first work

to our knowledge to reveal significant

associations between facial expression

and fatigue beyond eye blinks. The project

also revealed a potential association

between head roll and driver drowsiness,

and the coupling of head roll with steering

motion during drowsiness. Of note is that

a behavior that is often assumed to be

predictive of drowsiness, yawn, was in fact

a negative predictor of the 60-second

window prior to a crash. It appears that in

the moments before falling asleep, drivers

yawn less, not more, often. This highlights

the importance of using examples of

fatigue and drowsiness conditions in which

subjects actually fall sleep. The real

advantages of these following techniques

are, these can be combined into one

system and integration without affection or

interrupting each other detection and

function. This helps to give a exact alert

level to avoid the accidents due to the

driver drowsiness.

ACKNOWLEDGMENT

I like to thank Prof. Dr.-Ing. Olfa

Kanoun, Head of the Chair for

Measurement and sensor technology, TU

Chemnitz for organizing this course and

giving an opportunity to develop our

presentation and report writing skills.

References

[1] Knipling, R., & Wang, J, Crashes and

fatalities related to driver

drowsiness/fatigue. Washington, DC:

National Highway Traffic Safety

Administration (1994).

[2] Fridulv Sagberg, Road accidents

caused by drivers falling asleep, Norway,

23 December 1998.

[3] S. F. Liang, C. T. Lin, R. C. Wu, Y. C.

Chen, T. Y. Huang and T. P. Jung,

Monitoring Driver‘s Alertness Based on

the Driving Performance Estimation and

the EEG Power Spectrum Analysis,

Engineering in Medicine and Biology 27th

Annual Conference Shanghai, China,

September 1-4, 2005.

[4] Yi˘githan Dedeo glu, Moving object

detection, tracking and classification for

smart video surveillance, August, 2004

[5] Esra Vural, Mujdat Cetin, Aytul Ercil,

Gwen Littlewort, Marian Bartlett and Javier

Movellan, Drowsy Driver Detection

Through Facial Movement Analysis,

Grenoble, France (2000).

[6] Lawrence Barr, Heidi Howarth,

Stephen Popkin, Robert J. Carrol, A

review and evaluation of emerging driver

fatigue detection measures and

technologies, Washington, DC.

[7] Jiyan Pan, Quanfu Fan, Sharath

Pankanti, Robust abandoned object

detection using region-level analysis, IBM

T.J. Watson Research Center, Hawthorne,

NY, U.S.A.

[8]Kalpesh R Jadav, Prof. M.A.

Lokhandwala,Prof.A.P.Gharge,Vision

based moving object detection and

tracking,Limda,vadodara,India 13-14 May

2011.

[9] Smith, P.,& Shah, M.,& da Vitoria Lobo,

N. ,―Determining driver visual attention

with one camera‖ Intelligent

Transportation Systems, IEEE

Transactions, Dec, 2003.

[10] Bergasa, L.M.,& Nuevo, J.,& Sotelo,

M.A.,& Barea, R.,& Lopez, M.E., ―Real-

time system for monitoring driver

vigilance‖ Intelligent Transportation

Systems, IEEE Transactions, March,

2006.

[11] Qiang Ji,& Zhiwei Zhu,& Lan, P.,

―Real-time nonintrusive monitoring and

prediction of driver fatigue‖ Vehicular

Technology, IEEE Transactions, July,

2004.