Part v P300-Based Brain Computer Interfaces

8/10/2019 Part v P300-Based Brain Computer Interfaces

1/111


2/111

Prepared by Ozgen Sumer LACIN

Class: EE 517 Therapeutic and ProstheticDevices

Date: 11/12/2012

P300-based Brain Computer

Interface (P300 BCI)


3/111

Outline of Presentation BCI Definition & Methods

Potential Users of BCI and brief explanation of disease

Measuring Brain Activity Invasive Methods (ECoG, Cortical Microelectrodes)

Non-invasive Methods ( EEG,MEG,fMRI,NIRS)

BCI Approaches to Communication

Slow Cortical Potentials (SCP)

Steady State Visual Evoked Potentials (SSVEP) Motor Imagery Tasks

Evoked potentials (EP)

Framework of P300 System

1) Signal Acquisition

2) Feature Extraction

3) Feature Selection

4) Feature Classification

Comparison of classification techniques

METU BCI Research

BCI Companies

References


4/111

Brain Computer Interface (BCI)

Brain Computer Interface (BCI), is a system which allow people tocommunicate with their environment and control prosthetic or otherexternal devices by using only their brain activity.

As aformal definition BCI is: a communication system in which messages or commands sends to the

external world DO NOT PASS THROUGH THE BRAINSNORMALOUTPUT PATHWAYS OF PERIPHERAL NERVES AND MUSCLESmeaning BCI provides a new pathway for its user to communicate withan external world. [1]


5/111

BCI methods

BCI can be divided into 2 subsections :

1) Dependent BCI: Doesnt use brains normal output pathwaysto convey messages but activity in these pathways is needed togenerate activity. [1]

2) Independent BCI: Does not depend on any way of the brainsnormal output pathways and the message is not carried withperipheral muscles or nerves. The activity in these neurons isnot needed to generate the signal. [1]


6/111

BCI Approaches for

Communication

Slow Cortical Potentials (SCP) Anticipation tasks

Steady State Visual Evoked Potentials (SSVEP) Flickering light of specific frequency

Motor Imagery Tasks Changes of mu rhytm, alpha and beta activity over the

sensorimotor areas

Imagination of hand, foot, tongue, movement

Evoked Potentials (EP) Focus of attention to a visual or auditory stimulation

P300 Signals[5]


7/111

BCI system is mainly designed for people sufferingfrom the following disease:

Amyotrophic Lateral Sclerosis (ALS)

Multiple Sclerosis

Muscular Dystrophy

Cerebral Palsy

Brainstem Stroke

Spinal Cord Injury

Other types of Stroke [5]

Potential users of BCI


8/111

Amytrophic Lateral Sclerosis(ALS)

ALS is also known as motor neuron disease andoccurs due to the degeneration and lack of neuralcells in the Central Nervous Systems (CNS),

brainstem and spinal cord.Due to these missing neural cells, disease ischaracterized by rapidly progressive weakness,muscle atrophy and fasciculations, muscle spasticity,difficulty speaking (dysarthria), difficulty

swallowing (dysphagia), and difficulty breathing(dyspnea). [25]

ALS Disease
http://www.youtube.com/watch?v=pOvvW8gbWSAhttp://www.youtube.com/watch?v=pOvvW8gbWSAhttp://www.youtube.com/watch?v=pOvvW8gbWSAhttp://www.youtube.com/watch?v=pOvvW8gbWSA


9/111

Multiple Sclerosis (MS)

Multiple sclerosis (MS), also known as"disseminated sclerosis". It is aninflammatory disease in which the fattymyelin sheaths around the axons of thebrain and spinal cord are damaged, leading

to demyelination and scarring. Disease onset usually occurs in young

adults, and it is more common in women.It has a prevalence that ranges between 2and 150 per 100,000. [3]Fig. Multiple Sclerosis Explanation

[21]

Fig. 3 Demyelination [22]


10/111

Muscular dystrophy (MD)

Muscular dystrophy (MD) is a group ofmuscle diseases that weaken themusculoskeletal system and hamperlocomotion. Muscular dystrophies arecharacterized by progressive skeletalmuscle weakness, defects in muscleproteins, and the death of muscle cells

and tissue.[4]

Fig.4 Cell condtionafterMuscular dystrophy [23]

Fig.5 demonstration of musculardystrophy on human body [24]


11/111

Cerebral Palsy & Brainstem Stroke

Cerebral palsy (CP) is a group of non-progressive, non-contagious motor conditions that cause physicaldisability in human development, chiefly in the variousareas of body movement

A Brain stem stroke syndrome is a condition involving astroke of the brain stem. Because of their location, theyoften involve impairment both of the cranial nuclei and

of the long tracts.


12/111

Ways to overcome the disabilities

1) Improving the capabilities of remaining pathways

2) Restoring function by detouring around breaks in

the neural pathways that control muscles.

3)BCI for conveying messages and commands to

external world. [1]


13/111

The applications of BCI

Fig.6 Applications of BCI: i) Wheel chair, ii) robotic arm, iii)speller


14/111

Framework of BCI System

Fig. 7: Frame work of BCI system [1]


15/111

Measuring the brain activity Several ways to measure the brain activity which are electrical,

magnetic or hemodynamic activity measurements. Electrical measurements are preferred due to their practical usage whereas

other measuring methodologies are not practical due to their size and non-portability, MEG, fMRI etc.

There are two basic methods for brain activity measurement. i) Invasive methods i.e, require a surgical operation such as

electrocorticogram (ECoG) and microelectrode arrays

ii) Non-invasive method which does not require a surgical operation such as

electroencephalography(EEG), magnetoencephalography(MEG), near infraredspectroscopy(NIRS), functional magnetic resonance imaging(fMRI)


16/111

Invasive Methods Electrocorticogram and Cortical Microelectrodes

Electrocorticogram (ECoG) is an invasive method in which theelectrical signals of the brain are measured under the skull, from thesurface of the cortex.

The electrodes are usually madeup of a conductive

biocompatible needle or a grid

of needles and are implementedon the cortex surface with asurgical operation.

Fig.8 ECoG demonstration ofthe position of electrodes [14]


17/111

Cortical Microelectrodes Similar to ECoG but placed inside the cortex. Electrodes developed with VLSI technology.

The signal quality is improved by integrated analog circuitsdesign.

Possible to detect the activity of a single neuron with high spatialresolution and excellent signal-to-noise ratio (SNR)

Fig. 9 Cortical Microelectrodes [5]


18/111

Non-invasive methods

1) Electroencephalography (EEG)

2) Magnetoencephalography (MEG)

3) Functional Magnetic Resonance Imaging (fMRI) 4) Near Infrared Spectroscopy (NIRS)


19/111

1)Electroencephalography(EEG)

Electroencephalography (EEG) is the recording of electricalactivity along the scalp.

EEG measures voltage fluctuations resulting from ionic currentflows within the neurons of the brain. [26]

EEG has three main clinical usage:1) In neurology, the main diagnostic application of EEG is in the case

of epilepsy, as epileptic activity can create clear abnormalities on astandard EEG study.

2) Diagnosis of coma, encephalopathies (disorder of brain), and braindeath.

3) Investigating sleep and sleep disorders. [26]


20/111

EEG and its instruments

Fig 10. The measurement system consists of a number of electrodes, abiopotential amplifier and recording/monitoring devices. [28]


21/111

First usages of EEG

The first human EEG recordingobtained by Hans Berger in1924. The upper tracing is EEG,and the lower is a 10 Hz timingsignal.

EEG used to be a first-line method for the diagnosis of tumors, stroke andother focal brain disorders, but this use has decreased with the advent ofanatomical imaging techniques with high (


22/111

Recording principle of EEG(1/2)

1) Electrodes are placed on a scalp with a conductive gel or paste afterpreparing the area by light abrasion i.e., corrosion and remove deadskins to reduce impedance. Cap is used for when high density array ofelectrodes are needed. [26]

2) Each electrode is connected to one input of a differential amplifier, acommon system reference electrode is connected to the other input ofeach differential amplifier. These amplifiers amplify the voltage betweenthe active electrode and the reference (typically 1,000100,000 times,or 60100 dB of voltage gain) because a typical adult human EEG signal

is about 10V to 100 V in amplitude when measured from the scalpand is about 1020 mV when measured from subdural electrodes. [26]


23/111

Recording principle of EEG(2/2)

In analog EEG, the signal is then filtered, and the amplified signalis digitized via an analog-to-digital converter

Mainly there are 2 kinds of filters which are low-pass(LPF) andhigh-pass filters(HPF).

LPF: filters out high-frequency artifacts, such aselectromyographic signals

HPF: filters out slow artifact, such as electrogalvanic signals andmovement artifact.


24/111

EEG CAP Specifications and

Electrode Positions

Fig. 12 The 10-20 international system is the standard naming and positioningscheme for EEG applications [27]


25/111

Channel Selection

10 channels Fz, Cz, C3, C4, Pz, P3, P4, PO7, PO8, Oz

MeinickeKaper

Guger tech.


26/111

Advantages vs. Disadvantages

1) Low spatial resolution2) Determines only the

activity occurs on theupper part of cortex

3) Unlike PET and MRS,cannot identifyspecific locations inthe brain

4) Takes long time toconnect

5) Low Signal to NoiseRatio (SNR)

1)Cheap, silent, portable.2) very high temporal resolution3) relatively tolerant of subject movement,unlike all other neuroimaging techniques

4) does not involve exposure to high-intensity (>1 Tesla) magnetic fields, as insome of the other techniques, especiallyMRI and MRS.5) studies can be conducted with relatively

simple paradigms


27/111

2) Magnetoencephalography(MEG)

Magnetoencephalography (MEG) is a technique formapping brain activity by recording magnetic fieldsproduced by electrical currents occurring naturally inthe brain, using very sensitive magnetometers.

Due to its size, its just impracticalfor BCI applications.

Fig. 13 MEG device for clinical usage, [30]


28/111

3) Functional Magnetic

Resonance Imaging (fMRI)

fMRI is a method to measure the amount of oxygen inthe blood flowing through brain. When the neuronsare active the consumption of the oxygen increases inthe cells. Therefore it gives an idea about neuralactivity in different regions of brain.

High spatial resolution

Low temporal resolution

Fig. 14 fMRI device for clinical usage[31]


29/111

Near Infrared Spectroscopy (NIRS)

Similar to fMRI. The principle is to detect the

amount of blood oxygen in the brainfrom the reflection of the emitted

infrared light. As the hemodynamicactivity is measured, the temporalresolution is poor in NIRS systems,which makes the method impractical

for BCI applications.

Fig. 15 NIRS and its characteristics


30/111

BCI Approaches to Communication

1) Slow Cortical Potentials

2) Steady State Visual Evoked Potentials

3) Motor Imagery Tasks 4) Evoked Potentials


31/111


They are among the lowest frequency features of scalp recordedEEG.

These potential shifts occur over 0.510.0 s and are called slow

cortical potentials (SCPs). Negative SCPs are typically associatedwith movement and other functions involving cortical activation,while positive SCPs are usually associated with reduced corticalactivation .

They can be generalized as anticipationtasks[1]


32/111


Fig.16 SCP characteristics [1]

Success in patients in late stage ALS


33/111

2) Steady state evoked potentials

(SSVEP)

SSVEPs are oscillating signals elicited in the brainaccording to frequency of presented visualstimulation.

These signals are more distinctive in occipital regions

of the brain that is related to visual activities. SSVEP is employed in BCI applications by the

presentation of several flickering light sources withdifferent frequencies. In such a paradigm, the focusedlight elicits a signal pattern of the same frequency orharmonics with that of the source. [1]


34/111

2) SSVEPs

1) http://www.bsp.brain.riken.jp/~hova/ssvep_data_Bakardjian_LABSP.html

2) Show videos
http://www.bsp.brain.riken.jp/~hova/ssvep_data_Bakardjian_LABSP.htmlhttp://www.bsp.brain.riken.jp/~hova/ssvep_data_Bakardjian_LABSP.htmlhttp://www.bsp.brain.riken.jp/~hova/ssvep_data_Bakardjian_LABSP.htmlhttp://www.bsp.brain.riken.jp/~hova/ssvep_data_Bakardjian_LABSP.html


35/111

Sensory Motor Rhythms (SMR)

Idling activity can be called as mu-rhythm.(8-12 Hz during noengagement)

The amplitude of the signals may change during different brainactivities such as concentrating, voluntary muscle movement.

Movement or preparation for movement is typicallyaccompanied by a decrease in mu and beta rhythms andincrease in alpha rhythm called as event-related de-synchronization or ERD Its opposite, rhythm increase, orevent-related synchronization (ERS) occurs after movement

and with relaxation [1] ERD and ERS do not require actual

movement, they occur also with motor imagery. [1]


36/111

SMR&ERD&ERS

Fig.17: Sensorimotor characteristics and ERD &ERS waves respectively [1]


37/111

Wave Characteristics

Table 1: Wave frequencies and characteristics [3]


38/111

P300 Speller

The idea in this paradigm is to detect the P300 responses elicitedby the subject and predict the focused character according to thestarting time of the P300 response.

The target character is at the intersection of 1 row and 1 column

intensification. When these two stimulations are found, it is easyto predict the target character

Fig.17 P300 Characteristic Wave [4]


39/111

Oddball Paradigm P300 Signals

The oddball paradigmis a technique used in evokedpotential research in which trains of stimuli that areusually auditory or visual are used to assess the neuralreactions to unpredictable but recognizable events.

The subject is asked to react either by counting or bybutton pressing incidences of target stimuli that are hiddenas rare occurrences amongst a series of more commonstimuli, that often require no response. It has been found

that an evoked research potential across the parieto-central area of the skull that is usually around 300 ms andcalled P300 is larger after the target stimulus. [3]


40/111

Spelling Paradigm

2 target 12nontarget visualstimulations

Counting the

targetintensificationswill elicit the socalled P300responses

Video 1: P300 speller [5]


41/111

P300 Speller

Target (P300) responses Positive signal pattern peaking nearly 300 ms after the presentation of the target

stimulation Have latency of 300 - 400ms

Nontarget responses Have lower amplitute

Pattern similar to a sinusoidal of the same frequency with the stimulation

100 200 300 400 500 600 700

-50

0

50

time (ms)

Amplitude(

ADC

value) Channel FZ

100 200 300 400 500 600 70

-50

0

50

time (ms)

Amplitude(ADC

value) Channel FZ

Fig. 18 Target vs non-target amplitude in P300 [5]


42/111

P300 Based BCI Systems

Spelling Application

Intelligent House Systems with VirtualReality (VR)

Controlling robotic or prostheticdevices.


43/111

P300 Speller

Problems : The noise in EEG recordings

Factors in the cognitive process (fatigue, being unable to focus)

Repeating the intensification procedure for the focused

character Reducing the effect of noise by ensemble averaging of the

observations.

Main Problem:

Decreasing prediction time. [5]


44/111

P300 Speller - Studies

Graz University of Technology, Pfurtscheller et al. Wadsworth Center - Albany, Wolpaw et al. Tsinghua University, Gao et al. Fraunhofer-FIRST - Berlin, Blankertz & Mller

University of Rome - La Sapienza, Babiloni et al. University of Tuebingen, Kbler & Birbaumer University of Gttingen, Meinicke GtecGuger Technologies Sabanc niversitesi, Argunsah et al. ... [5]


45/111

P300 Signal Processing


46/111

1) Signal Acquisition

The extracted signal has verylow amplitude which is in thelevel of microvolts.

Highly sensitive to external andinternal distortions. Need for pre-processing

technique (filter) to enhance thesignal rather than just amplifying

the signal.

Fig. 19 Signal AcquisitionBlock [1]


47/111

Signal Enhancement (1/2)

Signal enhancement is applied prior to featureextraction to increase SNR. [2]

The use of a pre-processing technique has been proven

to be useful. [10] Number of electrodes, recording technology and

neuromechanism of BCI are some of factors todetermine for a suitable technique.


48/111

Signal Enhancement(2/2)

Commonly used signal enhancement techniques are

1) Surface Laplacian (SL)

2) Common Average Referencing (CAR)


49/111

1) Surface Laplacian

Smeared and intermixed current flow from brain to head. The spatial resolution of EEG decreases.

Surface Laplacian counters this effect by refocusing thesensitivity characteristic of the EEG electrodes to a smallvolume right below each electrode, thus eliminating theintermixing of the brain currents. [15]

Figure 20: Surface Laplacian[15]


50/111

Common Average Referencing(CAR)

The common average reference spatial filter calculates the mean of all channels,and subtracts this value from the output channel of interest. [16]

If electrodes are equally spaced result is zero mean spatial distribution. [6]

Fig. 21 Different Filtering Techniques i) Ear Reference, ii) CommonAverage Reference, iii) Small Laplacian, iv) Large Laplacian [1]


51/111

Which spatial filter provides the

highest SNR?

Since noise is highly complex, hence; there is a need for a filterwith high SNR.

1) Ear reference?

2) Common Average Reference? 3) Small Laplacian?

4) Large Laplacian?


52/111

Small and Large Laplacian

Electrode numbers over entirescalp and the surroundingelectrodes of reference pointsare important.

The distances to the set of

surrounding electrodesdetermine the spatial filteringcharacteristics of theLaplacian.

Small distance is more

sensitive to higher spatialfrequencies and less sensitiveto lower spatial frequencies

Fig. 22: Demonstration of Small and LargeLaplacian [1]


53/111

Results (1/2)

Fig. 23 Average voltage spectra for top targets (solid lines) and bottom targets (dashed lines) andaverage spectra of r^2 for the top/bottom difference for all sessions of all subjects for thelocations that controlled cursor movement online.[6]


54/111

Results (2/2)

CAR and Large Laplacian have the highest SNR then Small Laplacian thenear reference. [6]

Table 2: SNR values of different filter techniques [6]

Signal Enhancement Techniques


55/111

Signal Enhancement Techniques

used in literatureERN= event-related negativitySA-UK= Succesive averagingand / or considering choice ofunknownDSLVQ= Distinctive Sensitivelearning vector quantizationPCA= Principal ComponentAnalysis

GA= Genetic AlgorithmFreq-Norm= FrequencynormalizationCSSD= Common spatialsubspace decompositionCSP= Common spatialpatterns

ICA= Independent componentanalysisPCA= Principal componentanalysisSL= Surface LaplacianCAR= Common Average

Reference

Fig 24. Signal enhancement, feature selection /dimensionality reduction and post-processing methods in

BCI designs. [2]


56/111

Signal Processing part

Since the raw data from signalacquisition block might containredundant information.(e.g.EEG data)

1) Signals are digitally filtered2) Unnecessary information iseliminated by data selection. Inthe preprocessing stage

3) Noise reduction, downsampling etc. is done

Fig. 25 Signal Processing Block and itscomponents. [1]


57/111

1) Feature Extraction

The feature extraction is the stage inwhich the most relevant information forclassifying the EEG patterns is

investigated. Depending on thecomplexity of the BCI application, thefeature extraction is performed eithermanually or with the application ofoptimization algorithms. The aim of this

stage is to improve the classificationperformance of the BCI system and it isusually performed together with theclassification stage.

Fig 26. Feature Extraction

MATLAB [17]

Fig 27. FeatureExtraction of a face. [18]

Feature Extraction Methods used


58/111

Feature Extraction Methods used

in literature

In literature, scientists dealing with P300 based BCIused the following techniques to extract features oftheir signals.

Table 3. Feature extraction methods in BCI designs. Refer to appendix B insupplementary data for a more detailed version of this table. [2]


59/111

The distribution of the feature extraction

techniques with respect to application

Our interest is P300signals!!

Fig 27. Feature Extraction methods in BCI designs based on sensorimotor activity,VEP, P300, SCP, response to mental tasks, activity of neural cells and multiple

neuromechanisms. [2]


60/111

Feature Extraction

Commonly used methodologies for BCI are:1. Time and/or frequency methods.

Time methods have great temporal resolution whereas frequency methodsare preferred due to simplicity in use and fast computation.

2. Combination of temporal content with spectral informationthe time-frequency (TF)

Short time Fourier Transform & Wavelet Transformation are well knownones.


61/111

Time Frequency (TF) Analysis

The main approach of TF analysis is the combinationof time and frequency by using both of theiradvantages.

Time and frequency characteristics of ERD/ERS varyaccording to subject and yields a lot of temporal &spectral features.


62/111

Time Frequency Analysis

temporal resolution Spectral resolution

Fig.28 Combination of temporal vs spectral resolution to obtain a TFfilter


63/111

What is Time-Frequency Analysis

briefly?

Analysis providing time-varying spectralrepresentation of a signal which corresponds to thepower spectrum w.r.t time.

There are 2 methods:

Short Time Fourier Transform(STFT)

Morlet Wavelet Transform


64/111

STFT- briefly

STFT is fundamental for analyzing the slowly timevarying signal.

In contrast to FT, it can give information on the timeresolution of the spectrum by analyzing the frequencyresponse at different time instant.

Most popular one is Fast Fourier Transform(FFT)based on STFT.


65/111

The methodology of STFT

Signal is multiplied bya moving fixed lengthwindow functionwhich is non- zero for

a short period oftime[4]. Then FT isapplied within thewindow.

Fig. 29 STFT method for rectangular windowingwith 50% overlapping [10]


66/111

Morlet Wavelet Transform

Wavelet Transform decomposes signals into waveletswhich are localized both in time & frequency domain.

It is suitable for non-stationary signals (EEG signals).

Wavelet Transform is more realistic than STFT. Varying window as a function of frequency (in STFT

fixed window).


67/111

Why Morlet Wavelet Transform?

It is used in P300-based BCI, because EEG signal has aGaussian distribution in both time & frequencydomain and also suitable for motor imagery patterns

The width of its sliding windows varies as a function offrequency. [4]

Types of wavelet is determined according to thecharacteristics of the signal to be processed.


68/111

Morlet Wavelet characteristics

Fig. 30 Morlet Wavelet characteristic Equation [4]


69/111

Feature Selection

Algorithms are used to find the most informativefeatures for classification. [2]

Transformation of raw signal into a new structure toperform a better classification.

Remove the unnecessary information, keep thediscriminative ones.

Necessary in high dimension training data.

Higher classification accuracy and time saving.

Commonly used Feature Selection


70/111

y

methods

There are two mainly used feature selection methodswhich are :

1) Principal Component Analysis (PCA)

2) Genetic Algorithms (GA)

3) Learning Vector Quantization (LVQ)

4) Common Spatial Pattern (CSP)

Principal Component


71/111

p p

Analysis(PCA)

PCA is linear transformation that reducesdimensionality while retaining the ones thatcontributes to the variance most by keeping lowerorder principal components and ignoring high-order

ones. Since low-order components contain mostimportant aspects of the data.


72/111

Genetic Algorithms

Heuristic (depends on exploring) search techniques. Typically maintain a constant-sized population.

Tries to minimize the features to be used in

classification and maximize the performance ofclassification..

Ideal for applications where domain knowledge andtheory is difficult or impossible to provide. (De Jong

1975)


73/111

Feature Selection Results (1/2)

[11]

[11]

[11]


74/111

Feature Selection Results (2/2)

As can be seen, the results using only the selected features are far better than thoseusing all features. This shows how important feature selection is in the context ofEEG classification where a lot of channels only partially contain information aboutthe studied phenomenon.[11]

Learning Vector Quantization


75/111

g Q

(LVQ)

Neural Network Based Method. Aim is to find the proper reference vectors to be used

as the nearest neighbor classifier's reference set [4].

LVQ creates clusters of the training data and assignsthem to relevant classes.

The goal of LVQ is to find an optimal distribution ofthe clusters in the n-dimensional vector space.[4]


76/111

Common Spatial Pattern (CSP)

The principal idea is to project the multi-channel EEGdata into a low-dimensional data by weighting thesignals measured from electrodes. [4]

The idea of CSP is to find a spatial filter such that theprojected signals have high power for one class andlow power for the other in order to provideseparability.


77/111

Classification

Translating brain signals into device commands isachieved mainly by classification.

Understanding the features and their properties isnecessary to select the most appropriate classifier forgiven BCI system.

Amplitude of EEG signals, Band Power (BP), PowerSpectral Density(PSD), Auto-regressive

parameters(AR) should be determined for the designof BCI.


78/111

Critical Features of BCI system

noise and outliers high dimensionality

time information

non-stationary small training sets


79/111

Classifier Taxonomy

In order to choose the most appropriate classifier, theproperties of the available classifiers must be known.

1. Generative-discriminative

2. Static-dynamic3. Stable-unstable

4. Regularized


80/111

Main classification problems

The curse of dimensionality Training data should be at least 5-10 times more than

feature vector.

Unfortunately this cannot be applied in all BCI systems

due to training data set size. The Bias-Variance trade-off

Classification error can be described under 3 majorpossible sources

noise: noise in the system. it is irreducible. bias: divergence between estimated and best mapping

variance: reflects the sensitivity to the training set.

Popular classification techniques


81/111

in BCI research

1) Linear Classifiers

2) Neural Networks

3) Non-linear Bayesian classifiers

4) Nearest Neighbor Classifiers 5) Combination of Classifiers


82/111

1) Linear Classifiers

Discriminant algorithms to distinguish the classes.

Probably most popular algorithms

There are two main classifier have been used:

Linear Discriminant Analysis (LDA) Support Vector Machine (SVM)


83/111

Linear Discriminant Analysis (LDA)

Use hyperplanes to separate the different data.

One versus the rest.

For a two class problem:

Fig 32: A hyperplane which separates two classes: the circles andthe crosses [12]


84/111

Pros and Cons

Pros 1) Low computational requirement

2) Simple to use

3) Provides good and accurate results

4) Great number of success in BCI system [12]

Cons

1) Provides poor results on complex non-linear EEG

data. [12]


85/111

2) Support Vector Machine (SVM)

Also uses hyper-plane(s)

Good separation is achieved by the hyper-plane thathas the largest distance to the nearest training datapoint of any class (called functional margin).

The larger the margin the lower the generalizationerror of the classifier.


86/111

SVM

H3 (green) doesn'tseparate the two classes.H1 (blue) does, with asmall margin and H2(red) with themaximum margin. [13]

Fig 33. Support vector machine representation [13]


87/111

Neural Networks (NN)

Together with linear classifiers, they are mostly usedin BCI research.

NN is an assembly of artificial neurons.

NNs can be clustered under two categories: 1) Multilayer Perceptron (MLP)

2) Other Neural Network architectures


88/111

Multilayer Perceptron (MLP)

MLP is composed of several layers of neurons : an input layer

several hidden layers

output layers

when composed of enough neurons, MLP canapproximate any continuous function.

Fig 34. Artificial Neural Network of agroup of interconnected nodes [20]

Other Neural Network


89/111

Architectures

1) There is one that among all NN architectures whichhas been specially created for BCI: Gaussian Classifier[13].

This classifier has been applied with success to motorimagery and mental task classification.

BCI team in EPFL state that this NN outperformsMLP on BCI data. [14]


90/111

Non- Linear Bayesian Classifiers

Bayesian classifiers produce nonlinear decisionboundaries.

Their generative characteristics enables them toperform more efficient rejection of uncertain samplesthan discriminative classifiers.

They are not widespread as linear classifiers or NeuralNetworks in BCI applications because they are not fast

enough for real time BCI applications. [12].


91/111

4) Nearest Neighbor Classifiers

Supervised learning algorithm where classification ofnew coming signal is based on nearest neighborclassification.

The purpose is to sample the signal according to theattribute of training samples.

Assume Dtis the distance between the training sampleand the actual sample. Choosing the minimum

distance will allow us to choose the prediction of class.


92/111

Nearest Neighbor Classification - NN

Fig. 35 Demonstration of the Nearest Neighbor Classification. Circlesand rectangles represent different classes. An unknown object (star) isclassified as a circle because the closest object is a circle.[12]


93/111

5) Combinations of Classifiers

Recent trend is to combine different classifiers.Strategies are: 1) Boosting: Using several classifiers in cascade. Each

classifier focuses the errors committed by the previous

one. 2) Voting: Each different classifier assign the input

feature vector to a class. Majority will be the final class.It is simple and efficient. (like political voting)

3) Stacking: Each of several classifiers classify the inputfeature vector. Output of each of these classifiers isgiven as input to a so-called meta-classifier.


94/111

Nominees for Classification

1) SVM 2) Dynamic classifiers

3) Combination of classifiers

Properties of Classifications


95/111

p

Table 4: Accuracy of classifiers in movement intention based

BCI [2]


96/111

Table 5: Accuracy of classifiers in pure motor imagery based BCI [2]


97/111

Table 5: Accuracy of classifiers in pure motor imagery based BCI:multiclass and / or asynchronous case [2]


98/111

Table 5: Accuracy of classifiers in P300 speller BCI [2]

The classification award for BCI


99/111

goes to..

SVM

Classification Translation


100/111

Algorithm

Fig. 35 Classification Translation Algorithm [37]

I METU B i R h LAB


101/111

In METU Brain Research LAB

There has been made two P300 based research: 1) Hasan Balkar Erdogan: A DESIGN AND

IMPLEMENTATION OF P300 BASED BRAIN-COMPUTER INTERFACE, 2009

2) Berna Akinci: REALIZATION OF A CUE BASEDMOTOR IMAGERY BRAIN COMPUTERINTERFACE WITH ITS POTENTIAL APPLICATION

TO A WHEELCHAIR, 2010

METU BCI R h


102/111

METU BCI Research

http://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdi

http://www.youtube.com/watch?v=gnWSah4RD2E

http://www.youtube.com/watch?v=ppILwXwsMng&feature=fvwrel

BCI C i i th ld
http://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://www.youtube.com/watch?v=gnWSah4RD2Ehttp://www.youtube.com/watch?v=ppILwXwsMng&feature=fvwrelhttp://www.youtube.com/watch?v=ppILwXwsMng&feature=fvwrelhttp://www.youtube.com/watch?v=ppILwXwsMng&feature=fvwrelhttp://www.youtube.com/watch?v=ppILwXwsMng&feature=fvwrelhttp://www.youtube.com/watch?v=ppILwXwsMng&feature=fvwrelhttp://www.youtube.com/watch?v=gnWSah4RD2Ehttp://www.youtube.com/watch?v=gnWSah4RD2Ehttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdihttp://video.cnnturk.com/2010/bilim-teknoloji/4/15/odtululer-beyin-sinyallerini-harekete-cevirdi


103/111

BCI Companies in the world

1) Gtec 2) Emotiv Epoc

Gt P j t


104/111

Gtec Projects:

1) ALIAS: Adaptable Ambient Living Assistant -Mobile Robot System that interacts with elderly users,monitors physiology and uses BCI for control.

2) SM4ALL: smart homes for all - use BCIs to controlsmart homes

3) VERE: Virtual Embodiment and Robotic Re-Embodiment - BCIs for avatar control

Gt h


105/111

Gtec research areas:

Emoti Epoc


106/111

Emotiv - Epoc

Show video!

Thank you!!


107/111

Thank you!!

References


108/111

References [1] Wolpaw J.R., Birbaumer N., McFarland D.J., Pfurtscheller G., Vaughan T.M., Brain

Computer Interfaces for Communication and Control,Clinical Neurophysiology, 113:767-791, March 2002

[2] Bashashati A. , Fatourechi M. , Ward R, Asurvey of signal processing algorithms inbrain-computer interfaces based on electrical brain signals. J. Neural Eng. R32-R572007

[3] http://en.wikipedia.org/wiki/P300_(neuroscience)#P3a_and_P3b [4] Akinci,B. Realization of a cue based motor imagery brain computer interface with its

potential application to a wheel chair.,METU Library, 2010

[5] Erdoan H. B., A Design and Implementation of P300 Based Brain- Computer Interface,Metu Library, 2009.

[6] McFarland D.J., McCane L.M., David S.V., Wolpaw J.R., Spatialfilter selection for EEG-basedcommunication,Electroencephalogr. Clin. Neurophysiol,Vol. 103, pp. 386-394.

References


109/111

References

[7] Chapin J.K., Nicolelis M.A. L., "Principle component analysis of neuronal ensemble activityreveals multidimensional somatosensory representations", J. Neurosci. Meth, Vol. 94, pp. 121-140,1999.

[8] Bayliss J.D., Ballard D.H., RecognizingEvoked Potentials in a Virtual Environment,NIPS, pp.3-9, 1999. 137

[9] Ramoser H., Muller-Gerking J., Pfurtscheller G., Optimalspatial filtering of single trial EEGduring imagined hand movement, Rehabilitation Engineering, IEEE Transactions on NeuralSystems and Rehabilitation, Vol. 8, No. 4, pp. 441-446, Dec. 2000.

[10] http://en.wikipedia.org/wiki/Short-time_Fourier_transform

[11] Pregenzer M., Pfurtscheller G., "Frequency component selection for an EEG-based braincomputer interface (BCI)", IEEE Trans. Rehab Eng. Vol. 7, No. 3, Sep. 1999.

[12] Cover T. M., Hart P. E., Nearest neighbor pattern classification, IEEE Transactions InformationTheory, Vol. No. 13, pp. 21-27, 1967.

[13] http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/

[14] https://wiki.engr.illinois.edu/display/BIOE414/ECoG

References
http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/http://www.engadget.com/2009/05/04/mind-controlled-wheelchair-prototype-is-truly-insanely-awesome/


110/111

References [15] http://ajatubar.feld.cvut.cz/bisig/research

[16]http://www.bci2000.org/wiki/index.php/User_Reference:SpatialFilter#CAR

[17] http://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.png

[18] http://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpg

[19]http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/

[20] http://en.wikipedia.org/wiki/Artificial_neural_network

[21] http://www.aafnh.org/wp-content/uploads/2012/07/17089nlm_nih_gov.jpg

[22]http://en.wikipedia.org/wiki/File:MS_Demyelinisation_CD68_10xv2.jpg [23]http://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularD

ystrophy.png/230px-MuscularDystrophy.png

[

References
http://ajatubar.feld.cvut.cz/bisig/researchhttp://www.bci2000.org/wiki/index.php/User_Reference:SpatialFilterhttp://www.bci2000.org/wiki/index.php/User_Reference:SpatialFilterhttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/http://en.wikipedia.org/wiki/Artificial_neural_networkhttp://www.aafnh.org/wp-content/uploads/2012/07/17089nlm_nih_gov.jpghttp://www.aafnh.org/wp-content/uploads/2012/07/17089nlm_nih_gov.jpghttp://en.wikipedia.org/wiki/File:MS_Demyelinisation_CD68_10xv2.jpghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://upload.wikimedia.org/wikipedia/commons/thumb/7/75/MuscularDystrophy.png/230px-MuscularDystrophy.pnghttp://en.wikipedia.org/wiki/File:MS_Demyelinisation_CD68_10xv2.jpghttp://en.wikipedia.org/wiki/File:MS_Demyelinisation_CD68_10xv2.jpghttp://en.wikipedia.org/wiki/File:MS_Demyelinisation_CD68_10xv2.jpghttp://www.aafnh.org/wp-content/uploads/2012/07/17089nlm_nih_gov.jpghttp://www.aafnh.org/wp-content/uploads/2012/07/17089nlm_nih_gov.jpghttp://www.aafnh.org/wp-content/uploads/2012/07/17089nlm_nih_gov.jpghttp://www.aafnh.org/wp-content/uploads/2012/07/17089nlm_nih_gov.jpghttp://en.wikipedia.org/wiki/Artificial_neural_networkhttp://en.wikipedia.org/wiki/Artificial_neural_networkhttp://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/http://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://phucopierservice.com/nashuatec_micro/image-features-extraction-matlab-i0.jpghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.idiap.ch/~marcel/labs/faceverif/face-verif-for-dummies/facefeature-to-dct.pnghttp://www.bci2000.org/wiki/index.php/User_Reference:SpatialFilterhttp://www.bci2000.org/wiki/index.php/User_Reference:SpatialFilterhttp://www.bci2000.org/wiki/index.php/User_Reference:SpatialFilterhttp://ajatubar.feld.cvut.cz/bisig/research


111/111

References [24]http://www.humanillnesses.com/original/images/hdc_0001_0002_0_i

mg0181.jpg [25] http://en.wikipedia.org/wiki/Amyotrophic_lateral_sclerosis

[26] http://en.wikipedia.org/wiki/Electroencephalography

[27]https://wiki.engr.illinois.edu/download/attachments/44733162/ecog.jpg?version=1&modificationDate=1292382236000

[28] http://www.gtec.at/Research/Projects/ALIAS [29]http://www.bci2000.org/wiki/index.php/User_Tutorial:EEG_Measure

ment_Setup

[30]http://www.theredmenmovie.com/2009/11/magnetoencephalography-meg-scanner.html

[31]http://blogs.oem.indiana.edu/scholarships/index.php/2009/10/26/neu

rons-and-electrodes/fmri_groot/ [32] Hoffman, U. Bayesian Machine Learning Applied in a Brain-Computer

Interface for Disabled Users, EPFL, 2007
http://www.humanillnesses.com/original/images/hdc_0001_0002_0_img0181.jpghttp://www.humanillnesses.com/original/images/hdc_0001_0002_0_img0181.jpghttp://en.wikipedia.org/wiki/Amyotrophic_lateral_sclerosishttp://en.wikipedia.org/wiki/Electroencephalographyhttps://wiki.engr.illinois.edu/download/attachments/44733162/ecog.jpg?version=1&modificationDate=1292382236000https://wiki.engr.illinois.edu/download/attachments/44733162/ecog.jpg?version=1&modificationDate=1292382236000http://www.gtec.at/Research/Projects/ALIAShttp://www.gtec.at/Research/Projects/ALIAShttp://www.gtec.at/Research/Projects/ALIAShttps://wiki.engr.illinois.edu/download/attachments/44733162/ecog.jpg?version=1&modificationDate=1292382236000https://wiki.engr.illinois.edu/download/attachments/44733162/ecog.jpg?version=1&modificationDate=1292382236000https://wiki.engr.illinois.edu/download/attachments/44733162/ecog.jpg?version=1&modificationDate=1292382236000https://wiki.engr.illinois.edu/download/attachments/44733162/ecog.jpg?version=1&modificationDate=1292382236000http://en.wikipedia.org/wiki/Electroencephalographyhttp://en.wikipedia.org/wiki/Electroencephalographyhttp://en.wikipedia.org/wiki/Amyotrophic_lateral_sclerosishttp://www.humanillnesses.com/original/images/hdc_0001_0002_0_img0181.jpghttp://www.humanillnesses.com/original/images/hdc_0001_0002_0_img0181.jpghttp://www.humanillnesses.com/original/images/hdc_0001_0002_0_img0181.jpg

Documents

Part v P300-Based Brain Computer Interfaces