Elg5121 Project Prsentation Anisur Mohammad

8/3/2019 Elg5121 Project Prsentation Anisur Mohammad

1/41

BRAIN COMPUTER INTERACTIONELG5121 (MULTIMEDIA COMMUNICATION)

Anisur Rahman , student ID: 3087689

Mohammad Upal Mahfuz, student ID: 5819545


2/41

Outline

Introduction to Brain Computer Interaction(BCI)

Early work (1970 2000)

Animal BCI

Success Stories

Human BCI

Invasive, Partially-invasive, Non-invasive BCIs Case Study: BCI in Health Care

Future of BCI: Technical Challenges

Ethical Considerations

Scientific American(Nov. 2008)


3/41

Introduction to BCI

A braincomputer interface (BCI) is a directcommunication pathway between a brain andan external device

sometimes called a direct neural interface or abrainmachine interface

BCIs are often aimed at:

assisting, augmenting or repairing human cognitive or

sensory-motor functions

Signal processing for BCI


4/41

Motivation:

Developing technologies for people withdisabilities:

Need to develop hardware and software to

disable people Assist blind people to visualize external images

Assist paralyzed people to operate external deviceswithout physical movement

Decode information stored on human brain (asmemory)

Decode information from brain to display

human thinking or dream on a screen


5/41

Early Work (1970-2000)

Animal BCI:

University of California started research in BCIs inthe 1970s (Schmidt et al 1978)

experimental BCI started on animals: monkey and rats Several laboratories managed to record signals from

monkey and rat cerebral cortex in order to operateBCIs

Scientist started to work on developing BCIalgorithm


6/41

Success Stories

Research in1970s found: monkeys can control the firing rates of individual and

multiple neurons Can generate appropriate patterns of neural activity

Algorithms were developed to reconstruct movements frommotor cortex neurons

In the 1980s: Research in JHU found a mathematical relationship

(based on a cosine function) between the electrical responses of single motor-cortex neurons in

rhesus macaque monkeys, and the direction that monkeys moved their arms

Established that dispersed groups of neurons in differentareas of the brain collectively controlled motor commands

(Schmidt et al 1978)

(Georgopoulos et al 1989)


7/41

Human BCI

Human BCI Types: Invasive

Partially invasive

Non Invasive

Invasive: Implanted directly into the grey matter of the brain during

neurosurgery

Partially-Invasive: Partially invasive BCI devices are implanted inside the

skull but rest outside the brain rather than within the greymatter

Non-Invasive: Implanted outside the skull


8/41

Invasive BCI

implanted directly into the grey matter

of the brain during neurosurgery

targeted repairing damaged sight

Providing new functionality to personswith paralysis

produce the highest quality signals

Vision Science:

Direct brain implants have been used to treat non-congenital (acquired) blindness

William Dobelle was one of the first scientists to comeup with a working brain interface to restore sight

The Human Brain(Lennon, J. 2010)


9/41

Invasive BCI (Continues)

William Dobelle (1st Generation):

First prototype was implanted into Jerry blinded in adulthood, in 1978

Single-array BCI containing 68 electrodes wasimplanted onto visual cortex

Succeeded in producing phosphenses thesensation of seeing light

The system included cameras mounted onglasses to send signals to the plant

Enable him to perform daily tasks unassisted


10/41

Invasive BCI (Continues)

William Dobelle (2nd Generation): More sophisticated implant enabling better

mapping of phosphenes into coherent vision

Jens Naumann blinded in adulthood (2002) wasable to drive slowly in the parking lot immediatelyafter the implant

Disadvantage:

Prone to Scalar tissue build up Causes signal to become weaker or even lost as

the body reacts to a foreign object


11/41

Partial Invasive BCI

Implanted inside the skull but rest outside thebrain

Produce better resolution signals than non-

invasive BCIs having a lower risk of formingscar-tissue in the brain than fully-invasiveBCIs.

Examples: Electrocorticography (ECoG)

Light Reactive Imaging BCI


12/41

Partial Invasive BCI (Continues)

Electrocorticography (ECoG)

Measures the electrical activity of the brain takenfrom beneath the skull

Electrodes are embedded in a thin plastic padthat is placed above the cortex, beneath the duramater.

First trialed in humans in 2004 by Eric Leuthardt and

Daniel MoranEnabled a teenage boy to play Space Invaders

using ECoG implant:

Controls are rapid, and requires minimal training


13/41

Partial Invasive BCI (Continues)

Light Reactive Imaging BCI Light Reactive Imaging BCI devices are still in the

realm of theory

involve implanting a laser inside the skull.

laser is trained on a single neuron and the neuron'sreflectance measured by a separate sensor. When the neuron fires, the laser light pattern and

wavelengths would change

Advantages of Partial Invasive BCI Better signal to noise ratio Higher spatial ratio

Better Frequency Range


14/41

Non-Invasive BCI

Recorded signal have been used to power muscleimplants and restore partial movement

Signals are weaken as skull dampens the signal

Although the waves are still detectable, it is hardto determine the area of the brain or the neuronthat created the signal

Examples:

Electroencephalography (EEG) Magnetoencephalography (MEG)

Magnetic resonance imaging (MRI)


15/41

Non-Invasive BCI (Continues)

Electroencephalography (EEG)

Most studied potential non-invasive interface

Fine temporal resolution

EEG in Mid1990s:

Niels Birbaumer (University of Tbingen in Germany) trainedseverely paralyzed people to self-regulate the slow corticalpotentials in their EEG

EEG signal was used as a binary signal to control acomputer cursor

Ten patients were able to move computer cursors bycontrolling their brainwaves

Slow required an hour to write 100 characters


16/41


EEG in 2000s:

Jessica Bayliss (University of Rochester) showed thatvolunteers wearing virtual reality helmets could controlelements in a virtual world using their P300 EEG readings

including turning lights on and off

bringing a mock-up car to a stop

(Ebrahimi et al. 2003)


17/41


Advantages of EEG

Ease of use

Portable

Low setup cost


18/41


Magnetoencephalography (MEG)

MEG is a technique for mapping brain activity byrecording magnetic fields produced by electrical currentsoccurring naturally in the brain

By using Arrays of SQUIDs (superconducting quantuminterference devices)

Application :

Localizing the regions affected by pathology, before surgical

removal

determining the function of various parts of the brain

(Ranganatha 2007)


19/41


Magnetic resonance imaging (MRI)

MRI is a technique used in radiology to visualize detailedinternal structures

Functional MRI or fMRI is a type of MRI scan that

measures the hemodynamic response (change in bloodflow) related to neural activity in the brain or spinal cord

fMRI allowed two users being scanned to play Pong in real-time

by altering their haemodynamic response or brain blood flow

Recent research in ATR (Advanced TelecommunicationsResearch, in Kyoto, Japan) on fMRI

allowed the scientists to reconstruct images directly from thebrain and display them on a computer.


20/41

Case-study: TOBI Project

TOBI (Tools for Brain Computer Interaction) Budget:12 millions

Duration: Nov. 2008 Dec. 2012

Coordinator: Ecole Polytechnique Fdrale de Lausanne

Selected list of partners:

T. U. Berlin, T.U. Graz, U. Heidelberg, U. of Glasgow and someothers.

Non-invasive type BCI applications

TOBI is a large European integrated project which will develop

practical technology for brain-computer interaction (BCI) that willimprove the quality of life of disabled people and the

effectiveness of rehabilitation.Ref. http://www.tobi-project.org/welcome-tobi


21/41

TOBI Project- (Motor Disability)

Four emerging application areas

Communications and control

Motor substitution

Entertainment

Motor recovery


22/41

1. Communication & Control

Deriving useful EEG control signals is high priority thaninteractions

As a result, BCI systems are often clumsy and awkward.

TOBI will provide suitable and comfortable devices to Suppress noise

Better dynamic properties

of control signals

Multimodal interfacesVisual

Audio

Haptic F/B(TOBI website, 2010)


23/41

2. Motor Substitution

High priority is to restore lost motor functions forthe disabled.

TOBI has worked on developing

neuroprostheses Case-1: Two operations will be developed

hand (grasping)

elbow (reaching)

assistive mobility Case-2:

User can mentally drive mobile robot.

(TOBI website, 2010)


24/41

3. Entertainment

Target group: Giving patients controls of ambientfeatures wall display, lighting and music

Non-verbal way of interaction.

New features under investigation

photo browsing

music navigation

Couple BCI interfaces to social networking

BCI-controlled games/interactive games

(TOBI website, 2010)


25/41

4. Motor Recovery

BCI enhances motor function recovery after acerebrovascular accident.

In addition to active and/or passive residual movements,

imaging movements can be a way to access the motorsystem in absence of any "real" movements

TOBI will introduce the mental practice of motor actionsvia BCI training, that might boost the clinical

rehabilitation strategies

This in turn would lead to a better

functional outcome.(TOBI website, 2010)
http://www.youtube.com/watch?v=gvR0kHm9fwo


26/41

Future of BCI

Challenges have to overcome in hybrid BCI architectures

user-machine adaptation algorithms

BCI reliability analysis by exploiting users mentalstates

BCI performance analysis and confidencemeasures

Incorporate HCI to improve BCI

Development of novel EEG devices


27/41

Research Challenges (1/5)

Improve Non-invasive BCI based assistivetechnologies

Develop Hybrid BCI (hBCI)

Severe motor disabilities do not allow people tohave full benefit of current assistive products.

BCI

Enhancement

Assistive

Products (AP)

+

hBCI approach

The hBCI needs at least one BCI channel to work: otherchannel(s) can be AP input/biosignals or another BCI channel.

Millan et al. (2010)


28/41


Dynamic Adaptation Two-level adaptationprocess

First LevelSelf Adaptation

2nd levelDynamic Adaptation

The best interaction channelshould be dynamically chosen

The best EEG phenomena that eachuser better controls should bedynamically chosen:

P300 or SSVEP

This necessitates the development of novel trainingprotocols to determine the optimal EEG phenomenon foreach user, working on psychological factors in BCI.

Millan et al. (2010)


29/41


Improvement needs on current BCI outputs

Current BCI

has low bit rate,

noisy and has less reliability

Promising solution:

To adjust the dynamics of BCI, modern Human-ComputerInteraction (HCI) principles can be used

Alternative solution:

Use shared autonomy (or shared control) to shape thedynamics between user and brain-actuated device such thattasks are able to be performed as easily as possible.


30/41


BCI assisted technology can benefit from therecent research on the following-Recognition of users mental states

mental workload, stress, tiredness, attention level

Recognition of users cognitive processes awareness to errors made by the BCI

the dynamics and complexity of theinteraction will be simplified

This is another aspect of self-adaptation.

it will trigger OFF brain interaction and

move on to muscle-based interaction

High mentalworkload

Or stress levelOR

Example:


31/41


There are challenges to develop easy-to-use andaesthetic EEG equipment.

Issues to address: portability

aesthetic design

Aesthetic and engineering design should be merged.

One key issue for any practical BCI for disabledpeople.

Users dont want to look unusual socialacceptability

Example of advanced devices:

Dry electrodes instead of gel


32/41

Most Recent BCI News: (27 October2010)

http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/

Scientists from Germany, Israel, Korea, the UnitedKingdom, and the United States have performed

combined experiments: Are able to monitor individual neurons in a human

brain associated with specific visual memories

Display visual memory onto a television monitor, andto replace with another

Scientists have found a neural mechanismequivalent to imagination and daydreaming the mental creation of images overrides visual input
http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/http://spectrum.ieee.org/biomedical/bionics/braincomputer-interface-eavesdrops-on-a-daydream/


33/41

Most Recent BCI News: (Continues)

The researchers inserted microwires into thebrains of patients with severe epilepsy as part of apre-surgery evaluation to treat their seizures

The subjects were interviewed after the surgeryabout places theyd recently visited or movies ortelevision shows theyd recently seen:

images of the actors or visual landmarks the subjectshad described are shown on a display

Slides of the Eiffel Tower, for instance, or MichaelJacksonwho had recently died at the time of theexperimentwould appear on a screen.


34/41

Most Recent BCI News: (Continues)

Technical Challenges: about 5 million neurons in the brain encode for the

same concept, Cerf says.

Need to decode 5 millions neurons to get the completepicture

We are only able to read a limited number (for example:64 )

Complexity of Neuralnetwork (Lennon, J. 2010)


35/41

More news headings on BCI

Researchers Using Rat-Robot Hybrid toDesign Better Brain Machine Interfaces

Monkey Controls Advanced Robot Using

Its Mind

Monkey's Brain Can "Plug and Play" toControl Computer With Thought

IEEESpectrum(Oct. 2010)


36/41

Ethical Considerations

There has not been a vigorous debate about theethical implications of BCI

Important topics in neuroethical debate are:

Risk/benefit analysis

Obtaining informed consent

Possible side effects and consequences in life stylesfor the patient relatives

Professor Michael Crutcher expressed concernabout BCI specially for ear and eye implants: If only the rich can afford it, it puts everyone else at a

disadvantage


37/41

Ethical Considerations (Continues)

Clausen concluded in 2009:

BCIs pose ethical challenges, but these areconceptually similar to those that bioethicists haveaddressed for other realms of therapy

Recently more effort is made inside the BCIcommunity to initiate the development of ethicalguidelines for BCI research, development anddissemination

Requirements for the social acceptance: Sound ethical guidelines

Appropriately moderated enthusiasm in mediacoverage

Education about BCI systems


38/41

Conclusions

Brain Computer Interaction is:

Send outside signal to brain neuron

vision signal for blind person

Read the neuron signal To operate external devices without physical

intervention

To read memory or display user imagination

Significant progress in last ten years Technical challenges need to be overcome

Significant potential uses in medical science to

assist physically disabled persons


39/41

References

Ebrahimi, T.; Vesin, J., Garcia, G. Brain-Computer Interface in MultimediaCommunication. IEEE Signal Processing Magazine: January 2003.

TOBI Project by EU, Website: http://www.tobi-project.org/

J.d.R. Millan, R. Rupp, G.R. Muller-Putz, R. Murray-Smith4, C. Giugliemma, M.Tangermann, C. Vidaurre, F. Cincotti, A. Kubler, R. Leeb, C. Neuper, K.R. Muller, D.

Mattia (2010) "Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges," Frontiers in Neuroscience, vol 4, August, 2010,doi:10.3389/fnins.2010.00161.

B.Z. Allison, C. Brunner, V. Kaiser, G.R. Muller-Putz, C. Neuper, and G. Pfurtscheller.Toward a hybrid brain-computer interface based on imagined movement and visualattention. J. Neural Eng., 7, 2010.

B.Z. Allison, E.W. Wolpaw, and J.R. Wolpaw. Brain-computer interface systems:

Progress and prospects. Expert Rev. Med. Devices, 4:463474, 2007. G. Alon, A.F. Levitt, and P.A. McCarthy. Functional electrical stimulation enhancement

of upper extremity functional recovery during stroke rehabilitation: A pilot study.Neurorehabil. Neural Repair, 21:207215, 2007.

K.D. Anderson. Targeting recovery: Priorities of the spinal cord-injured population.Neurotrauma, 21:13711383, 2004.

K.K. Ang, C. Guan, K.S.G. Chua, B.T. Ang, C. Kuah, C. Wang, K.S. Phua, Z.Y. Chin,

and H. Zhang. A clinical study of motor imagery-based brain-computer interface forupper limb robotic rehabilitation. In Proc. 31th A. Int. Conf. IEEE Eng. Med. Biol. Soc.,
http://www.tobi-project.org/http://www.tobi-project.org/http://www.tobi-project.org/http://www.tobi-project.org/


40/41

References (Continues)

F. Babiloni, F. Cincotti, L. Lazzarini, J.d.R. Millan, J. Mourino, M. Varsta, J. Heikkonen,L. Bianchi, and M.G. Marciani. Linear classification of low-resolution EEG patternsproduced by imagined hand movements. IEEE Trans. Neural. Syst. Rehabil. Eng.,8:186188, 2000.

J.D. Bayliss. Use of the evoked potential P3 component for control in a virtualapartment. IEEE Trans. Neural. Syst. Rehabil. Eng., 11:113116, 2003.

Schmidt, EM; McIntosh, JS; Durelli, L; Bak, MJ (1978). "Fine control of operantly

conditioned firing patterns of cortical neurons.". Experimental neurology 61 (2): 34969.doi:10.1016/0014-4886(78)90252-2. PMID 101388.

Georgopoulos, A.; Lurito, J.; Petrides, M; Schwartz, A.; Massey, J. (1989). "Mentalrotation of the neuronal population vector". Science 243 (4888): 234.doi:10.1126/science.2911737. PMID 2911737.

Ranganatha Sitaram,Andrea Caria,Ralf Veit,Tilman Gaber,Giuseppina Rota,AndreaKuebler and Niels Birbaumer(2007) "FMRI BrainComputer Interface: A Tool for

Neuroscientific Research and Treatment Kennedy, PR; Bakay, RA (1998). "Restoration of neural output from a paralyzed patient

by a direct brain connection.". Neuroreport9 (8): 170711.

Lennon, JF, The human Brain , http://www.jflennon.com/ResearchJFLTech.htm, Nov.2010

Cover Image source : Medic Magic website: http://medicmagic.net/wp-content/uploads/2010/03/human-brain.jpg
http://www.jflennon.com/ResearchJFLTech.htmhttp://www.jflennon.com/ResearchJFLTech.htm


41/41

Questions & Answers

Documents

Elg5121 Project Prsentation Anisur Mohammad