Upload
gamindu-udayanga
View
33
Download
0
Tags:
Embed Size (px)
Citation preview
Smart SMS-SMS Application Management Platform
BRAIN COMPUTER INTERFACE APPLICATION
FRAMEWORK
K.W.S.D. Kaluarachchi
(IT 10 0273 70)
Degree of Bachelor of Science in Information Technology
Department of Information Technology
Sri Lanka Institute of Information Technology
October 2013
H. H. Rajamanthrie IT 10 0296 64
Smart SMS-SMS Application Management Platform
BRAIN COMPUTER INTERFACE APPLICATION
FRAMEWORK
K.W.S.D. Kaluarachchi
(IT 10 0273 70)
Dissertation submitted in partial fulfillment of the requirements for the degree
of Science
Department of Information Technology
Sri Lanka Institute of Information Technology
October 2013
H. H. Rajamanthrie IT 10 0296 64
Declaration “I declare that this is my own work and this dissertation does not incorporate without
acknowledgment any material previously submitted for a Degree or Diploma in any
Other University or Institute of higher learning and to the best of my knowledge and
belief it does not contain any material previously published or written by another
person except where the acknowledgment is made in the text.
Also, I hereby grant to Sri Lanka Institute of Information Technology the nonexclusive
right to reproduce and distribute my dissertation, in whole or in part in print, electronic
or other medium. I retain the right to use this content in whole or part in future works
(such as articles or books) “
Signature: Date: 2013/10/23
The above candidate has carried out research for the B.Sc Dissertation under my
supervision.
Signature of the supervisor: Date: 2013/10/23
i
Brain Computer Interface Application Framework
Acknowledgement We take this opportunity to express our deep sense of gratitude to those who contributed to either our collective or individual efforts.
Our heartfelt thanks go out to the supervisor of our project, Dr. Rohana Priyantha Thilakumara and the co-supervisor, Mr. Darshika Koggalahewa for the kind patience, guidance and constant support rendered to us at all the stages of the project, right from the very be- ginning of the project.
We would also like to thank the lecturer in-charge, Mr. Jayantha Amararachchi who provided the required lecture materials and the necessary guidance to do the project.
The team extends sincere gratitude to all colleagues, who forwarded their enthusiastic ideas during the requirements gathering phase of the project. The teammates would also like to thank all the staff members of the SLIIT Malabe campus for their valuable suggestions and opinions that helped us to do the project.
Finally, we thank all who lend their kind support; friends and families who continued to give their insights, patience, support and co-operation which motivated us in reaching greater heights.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY ii
Brain Computer Interface Application Framework
Abstract Brain-Computer Interface (BCI) technology is a potentially powerful communication and control option in the interaction between Human and Computer systems. A Brain Computer Interface (BCI) is a direct communication pathway between the brain and an external device. This technology enables signals that are generated in the brain to control an external activity such as control of a keyboard of a computer or control of a wheelchair. Over the past few years, new games have been developed that are exclusively for use with an EEG headset by companies like Neurosky and Emotiv. In this project we have developed a set of programs that can be controlled by brain signals. In this report I explain about the Ludo Game which is controlled by Signals generated inside the Brain. Our research part is to find out about an effective and an efficient way to use to brain signals for this application. This Ludo Game is going to be a new experience and will define a new dimension in the Game industry. In another way, this can be an exercise for the Brain.
ilitated in game developing. By proposing this frame work we try break down barrier between brain computer interface researchers and game development community. Our proposed framework will facilitate to professional game designers as well as independent gaming enthusiast to dive in to the depth of Brain Computer Interfacing gaming world without bothering about brain wave analyzing, feature extraction and classification. Furthermore they wouldn't want to brother about devices without considering devices they would be able to develop their creative gaming ideas. The objective of the proposed project is to bring expert knowledge of Neurophysicists toward the game developers and motivate them to develop more exciting and challenging games and expand the gaming industry.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY iii
Brain Computer Interface Application Framework
Table of Contents DECLARATION 1
ACKNOWLEDGEMENT II
ABSTRACT III
LIST OF FIGURES V
LIST OF ABBREVIATIONS VI
1 INTRODUCTION 1
1.1 BACKGROUND CONTEXT 1 1.2 RESEARCH PROBLEM TO BE ADDRESSED 2 1.3 RESEARCH QUESTIONS 3
2 CONTENT 4
2.1 ADDRESSING THE LITERATURE 4 2.2 METHODOLOGY 5
2.2.1 Overview 5 2.2.2 Overview of the System Design 7 2.2.3 User Characteristics 7 2.2.4 Product Functions 9 2.2.5 Tools and Technologies 10 2.2.6 Product Constraints 11 2.2.7 Assumptions and Dependencies 11
2.3 RESEARCH FINDINGS 12
3 RESULTS AND DISCUSSION 18
4 CONCLUSION 24
5 REFERENCES 25
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY iv
Brain Computer Interface Application Framework
LIST OF FIGURES
Figure 1: System Diagram ........................................................................................................ 7 Figure 2: Student A – Attention Level Values ........................................................................ 14 Figure 3: Student A – Variance Between Attention Level Values .......................................... 14 Figure 4: Student B – Attention Level Values ........................................................................ 15 Figure 5: Student B – Variance Between Attention Level Values .......................................... 15 Figure 6: Student C - Attention Level Values while listening to a Song ................................. 17 Figure 7: Student D - Attention Level Values while listening to a Song ................................. 17 Figure 8: Attention Level Values ............................................................................................ 18 Figure 9: Running Average of Attention Level Values ........................................................... 19 Figure 10 - The simple keyboard ............................................................................................ 20 Figure 11 - Using the Simple Keyboard. ................................................................................. 21 Figure 12 - The Advance Keyboard. ....................................................................................... 22 Figure 13: Ludo Game User Interface .................................................................................... 23
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY v
Brain Computer Interface Application Framework
List of Abbreviations
EEG Electro - Electroencephalogram
BCI Brain Computer Interface
fMRI functional Magnetic Resonance Imaging
EEC Encephalogram
SSVEP Steady State Visual Evoked Potential
ALS Amyotrophic Lateral Sclerosis
API Application Programming Interface
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY vi
Brain Computer Interface Application Framework
1 INTRODUCTION
1.1 Background Context
The advent of computers was a significant breakthrough in this area of work, as it lead
the way towards what we today know as “Brain Computer Interface (BCI)”. The
computational processing aspect of computers now opened up doors to the possibility
of being able to analyze and distinguish the patterns of the electrical signals being
produced by the human brain, and thus allowing to trigger a desired outcome for a
given identified brain signal pattern. For example, when a brain signal pattern is
detected by the computer then it could be associated to trigger a tangible action like
activating a switch to turn on/off the lights. In simple terms this kind of a setup that
detects electrical signals from the brain, and computes these signal patterns to initiate
a substantial tangible action is known as a BCI.
BCI has always been a subject matter of interest in the fields of Rehabilitation and
Assistive Technology. The potential of BCI is unbounded with regards to improving
the lives people with disabilities. For example, BCI based systems could allow an
individual with severely restricted or no range of movements (e.g. spinal cord injuries)
to drive a power wheelchair, operate household appliances, etc. Or even help someone
in a vegetative state to communicate by speaking out the words that the individual
would like to say.
Primarily there are two categories of BCI systems – “invasive” and “noninvasive”.
Invasive systems interact with the brain directly via electrodes / sensors that are
implanted into the brain or its surface. While noninvasive systems interact with the
brain indirectly via electrodes / sensors placed on the surface of the head that detect
brain signal emissions (e.g. Electro-Encephalography (EEG), functional Magnetic
Resonance Imaging (fMRI), and Magnetic Sensor Systems).
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 1
Brain Computer Interface Application Framework
Noninvasive BCI systems usually involve wearing a head cap (aka EEG cap) with
multiple holes / slots to put on the electrodes at the relevant areas of the surface of the
head to detect and record the electrical signals emitted by the brain. Electro gel is used
on the electrodes to improve contact between the scalp and the electrode. Due to the
requirement of a gel such electrodes are also known as wet-electrodes. Existing
systems can have anywhere between a few to more than a 100 electrodes. The
practicalities of using wet electrodes, as an example drying up of gel, repeated cleaning
of electrodes and head skin to setup EEG Cap, irritation of sensitive skin due to
application of gel, etc. – does not make them convenient for quick setup and daily use.
Over the recent few years this has prompted the development of dry electrodes. Unlike
wet electrodes, dry electrodes do not require the use of gel and can be setup direct into
the EEG Cap. Although still in the infancy of its developmental cycle, dry electrodes
are quickly catching up in terms of detecting high quality brain signals as their wet
counterpart. Regular comparison studies are being carried out to evaluate the
performance of Wet vs. Dry Electrodes within the context of EEG based noninvasive
BCI.
1.2 Research Problem to be addressed
When it comes to Brain Computer Interface technology, the development in this field
is still less. There are few device that uses BCI technology and those have many
limitations. And another problem is using the signals that are given by the BCI devices
in a useful way. In our research we are addressing this problem of using the Brain
Signals that gets captured by a BCI device in a useful way.
It is very difficult to use brainwaves to control BCI games, because each user will have
a different distribution of brainwaves. For example, some users’ brainwaves will have
a bigger amplitude than others. Therefore, we require to address this and find a suitable
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 2
Brain Computer Interface Application Framework
method that transfers the brainwaves to the signals that are good enough to be used in
a BCI game.
We checked and tested few methods to choose a best way to use the signals. We
collected many signal samples using few SLIIT students. Then calculated them using
our methods and chose the best method.
Fourth, a systematic BCI serious game development process is needed for experts and
game developers dealing with brain functions and brainwaves. This will allow diverse
developers to quickly and easily produce games together.
1.3 Research Questions
1. The Brain Signals from person to person are different from each other. How to
handle it?
2. There can be problems in signal strength. How to handle it?
3. What is the age range and duration of brains waves taken from a person?
4. How the brain waves are collected accurately without any error occurred?
5. How the accuracy of collected brain waves is tested?
6. How the BCI Device is used in suitable Application?
7. How the BCI application is developed to suit anyone at different age ranges
and the gender?
8. How the BCI device is utilized in efficiently?
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 3
Brain Computer Interface Application Framework
2 CONTENT
2.1 Addressing the Literature
In what research has been done so far, the field of computer and information
technology has developed a wealth of devices that capture brain signals. And there are
applications that use them. These applications do not have a proper way to use the
brain signals that gets captured by the device. In our research we test and suggest a
proper method to use the captured signals.
The current applications have faults in signal handling. The brain signals gets vary
from person to person. The strengths of signals get dropped during occasions.
In the applications we create, we use Running Average. The values that are given by
the device are sent through a program function which calculates the Running Average
of those values. And then Running Average values are sent to the game. Because of
this we can avoid the big variances in the signals.
What is Running Average?
In statistics, a moving average, also called rolling average, moving mean, rolling
mean, sliding temporal average, or running average, is used to analyze a set of data
points by creating a series of averages of different subsets of the full data set.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 4
Brain Computer Interface Application Framework
2.2 Methodology
2.2.1 Overview
Not only as an assistive technology, Brain Computer Interface technology can also be
used to give entertainment in a new way taking the entertainment into a new level.
Imagine a person playing a game using only his/hers thinking power.
In the past 10 years, the gaming industry has been a growing multi-billion-dollar
business, this shows that the demand of videos games has been growing, and this
rocketing demand also attracts a vast investment on new gaming interfaces, such as
Dance Pad in PlayStation and Wii controllers in Wii, which furthered feedback to the
snowball (i.e. the demand) positively. However, while new interfaces for console
games (e.g. touch screens for NDS and remote motion sensors for Wii) has been
developed, emergence of new gaming interfaces for PC games seem to slow down
after the introduction of game pads, and we think a new gaming interface could perhaps
give birth to a new genre of games in the big PC games market, where almost been
captured by interfaces like Microsoft Kinect. Hence one may wonder if anything new
would appear in the future gaming world, and what that would be recently, brain
computer interfaces for consumer level have been released to the market, making BCI
entertainment possible.
There are communities who are involved in developing Brain Computer Interface
games. However, we see a discrepancy between the BCI games developed by the
communities. Many of the BCI games developed by the BCI community aim at testing
some psychological hypotheses or evaluating the performance of signal analysis and
classification techniques. Thus, less attention is paid to game characteristics than to
technical aspects. These games do not usually have any narrative or rich feedback or
visuals. User (i.e. player) experience evaluations are almost never carried out. This
leads to BCI games that are reliable but often not enjoyable. On the contrary, BCI
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 5
Brain Computer Interface Application Framework
games from the games community are developed with respect to game design
principles. However, the neurophysiology and signal analysis techniques they rely on
are largely unknown. Because, these games mostly make use of the commercial BCI
headsets which have their private technical details. This leads to BCI games that are
potentially entertaining but unsatisfactory in terms of feeling of control.
In proposed method, we will try to transfer some knowledge from the games and the
BCI communities into a shared preliminary framework to make them aware of each
other’s research. From the games community, we will show some game playing
motivations which can be satisfied by the features of BCI. From the BCI community,
we will take the current interaction paradigms used in general and show the ways they
can be used in games. This way, we hope to contribute to bridging the gap between
the two communities and promoting the development of entertaining and reliable BCI
games.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 6
Brain Computer Interface Application Framework
2.2.2 Overview of the System Design
Figure 1: System Diagram
2.2.3 User Characteristics
The users of the system can be categorized under user groups as follows;
1. BCI device makers. 2. BCI game developers. 3. BCI game players. 4. Disabled persons.
The characteristics of those identified user groups of the Interactive Programming
Assistance Tool are illustrated below.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 7
Brain Computer Interface Application Framework
BCI device makers.
They are the people who develops and manufactures BCI devices. There are various
brands in World Market nowadays. And from those, Neurosky and Emotiv owns a
major market share.
BCI game developers.
They develop the games which can be played using BCI devices. The most part of
these of games get controlled directly using brain signals.
BCI game players.
BCI game players get an experience which is much different from the experience that
normal game players receive. They normally use their thinking power, concentration
power to control games rather than hand.
Disabled persons.
Because this game is controlled by the signals that are generated inside the Brain, even
a person who does not have any hands can play this game.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 8
Brain Computer Interface Application Framework
2.2.4 Product Functions
BCI enhanced Ludo Game
This Ludo game is developed to be controlled by the brain waves. By wearing this
device on your head and using your attention level, you can generate the value of the
Die.
There are two main functions in this Ludo Game.
- Calculating Running Average.
We tested few methods and finally chose to use running average to get a better
signal value from the device. This function does the part of calculating running average
of the signals values that gets captured by the device.
- Get a value to map with the game in order to control it.
We have developed a Ludo game to as a sample game. This function will map the
signal values that get calculated by the “Calculating Running Average” function.
BCI enhanced Keyboard
This application is a virtual keyboard which can be controlled by the eye blink instead
of the hands. There is some functionality which provides users to press the appropriate
keys using just the eye blink than using the hand.
BCI enhanced Calculator
This application also is same as the above mentioned keyboard. It also provides the
same functionality to the users as it allows users to control the options of the calculator
using the eye blink. For calculating purposes user have to train his/her way of blinking
in order to get the correct options.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 9
Brain Computer Interface Application Framework
BCI enhanced Snake and Ladders Game
This gaming application is for entertaining the users providing opportunity to play the
game using their eye blink than using the attention and the focus level. Depending on
the eye blink user has to toss the dice and win the game. There fore it provides users a
great opportunity to play the game in a new way apart from the traditional way.
2.2.5 Tools and Technologies
• We have used latest tools and technologies to do our research and to develop
the Ludo game. The tools and technologies include the following:
- Microsoft Visual Studio 2012 - .NET Framework 4.5 - Neurosky MindWave Device
• Microsoft Visual Studio 2012 is the IDE that we used to develop the Software
components of our product. These Software components mainly include the
Ludo game, the connection classes which connect the Neurosky device with
the software components, the running average calculation classes.
• Neurosky MindWave is the device we used for the BCI device.
• Microsoft Excel was used to store signal values we have collected from some
SLIIT students. Excel was also used to analyze the stored signal values by
drawing charts and by calculating using few formulas.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 10
Brain Computer Interface Application Framework
2.2.6 Product Constraints
• Our applications are specifically developed for Windows platform and tightly
coupled with windows based computer architecture. Developers do not have
sufficient hard-ware resources to build this application on different system
architectures such as Linux platform.
• When getting Brain Signal values using the device, the device takes like a
second to calibrate. So , when you start collecting the signal values, the values
come one second after you started it.
• The BCI device’s sensor which touches the forehead should have a good
contact with the forehead. Otherwise it won’t give good readings and will say
the signal quality is poor.
• In order for the device to work properly, the batteries that are inserted to the
device should be good quality batteries with a good charge.
• The driver for the BCI device should be installed in the Computer.
2.2.7 Assumptions and Dependencies
• It is assumed that the computer which runs our applications is running on the
Windows operating system and the Visual Studio framework is installed in the
respective machine. It is essential that the Visual Studio is up and running by
means of the accurate .NET framework.
• The source codes that are to be input to the system should be developed in C#
language.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 11
Brain Computer Interface Application Framework
• Microsoft Windows based supported architecture should be available and
implemented
- x86
- x64
• Microsoft Windows based operating system (Windows XP service pack 3 or
higher version) should be available and functioning successfully.
• All the machines should support the .NET framework 3.5 or higher.
2.3 Research Findings
While we are doing the research, through the tests we did, we found out that
• The device we are using have many limitations.
• The signals can get weakened at times.
• The variations of signal values are high.
• The way persons react to a one particular activity is different, because of that
the signals their brains emit are different to each other.
The device we are using have many limitations.
Our neurosky device comes with one sensor. And it can detect only meditation,
attention, eye blink activities of the user. Because of that, the applications for this
device can be only controlled by meditating, keeping the attention and by blinking
eyes.
Other movements of a user, like moving eyes, moving the head, moving a hand, can
not be recognised by this device.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 12
Brain Computer Interface Application Framework
The signals can get weakened at times.
The signal strength depends on few factors
- Whether the sensor is touching the forehead.
- Whether the battery of the neurosky device has a good charge level.
- The distance between the user who is wearing the device and the Computer
which is running the application that is controlled by brain signals of the user.
The variations of signal values are high.
The values of signals we gained by the tests were added to excel sheets. Then we
generated charts using those values. When considering those charts, we noticed a big
variation between the signal values. We calculated the variations of those signals.
Those calculations also gave higher values.
The following charts provide some examples for these higher variations. The data were
collected from a SLIIT Student. First we asked him to play a Car Race Game, then we
asked him to listen to a song. And while he was playing and listening to the song, we
collected the data. And using those data we calculated Running Average of those
values & variations of them.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 13
Brain Computer Interface Application Framework
Figure 2: Student A – Attention Level Values
Figure 3: Student A – Variance Between Attention Level Values
0
20
40
60
80
100
120
1 28 55 82 109
136
163
190
217
244
271
298
325
352
379
406
433
460
487
514
541
568
595
622
649
676
703
730
757
784
811
838
865
892
919
946
Attention Level Values
0
10
20
30
1 27 53 79 105
131
157
183
209
235
261
287
313
339
365
391
417
443
469
495
521
547
573
599
625
651
677
703
729
755
781
807
833
859
885
911
Variance Between Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 14
Brain Computer Interface Application Framework
Figure 4: Student B – Attention Level Values
Figure 5: Student B – Variance Between Attention Level Values
0
10
20
30
40
50
60
70
80
90
1 10 19 28 37 46 55 64 73 82 91 100
109
118
127
136
145
154
163
172
181
190
199
208
217
226
235
244
253
262
271
280
289
298
307
316
325
Attention Level Values
0
5
10
15
20
25
1 9 17 25 33 41 49 57 65 73 81 89 97 105
113
121
129
137
145
153
161
169
177
185
193
201
209
217
225
233
241
249
257
265
273
281
289
Variance Between Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 15
Brain Computer Interface Application Framework
The way persons react to a one particular activity is different, because of that the signals their brains emit are different to each other.
The way different persons react to a particular activity is different from each other.
Because of this the Brain Signals generated by different persons for a particular
activity is different from each other.
As an example some persons might like classic songs, while some don’t. While some
may have mixed emotions towards classic songs.
The following two charts were created using the data collected which were collected
using two SLIIT Students. The data were collected when they were listening to a song.
Both of them listened to the same song. Though they listened to the same song, the
way their brain reacted to that song was different. The two charts provide evidence for
that.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 16
Brain Computer Interface Application Framework
Figure 6: Student C - Attention Level Values while listening to a Song
Figure 7: Student D - Attention Level Values while listening to a Song
0
20
40
60
80
100
120
1 10 19 28 37 46 55 64 73 82 91 100
109
118
127
136
145
154
163
172
181
190
199
208
217
226
235
244
253
262
271
280
289
298
307
316
325
Attention Level Values
0102030405060708090
1 10 19 28 37 46 55 64 73 82 91 100
109
118
127
136
145
154
163
172
181
190
199
208
217
226
235
244
253
262
271
280
289
298
307
316
325
Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 17
Brain Computer Interface Application Framework
3 RESULTS AND DISCUSSION
Why using the Running Average is better
When we consider the Attention or Meditation values that are directly given by the
neurosky device, we can notice a high variance between the values. That can be a
disadvantage while playing a game or using an application that is created based on the
BCI Technology.
Imagine you are trying to move a ball up by using the attention level of your brain.
Because of the high variance, the signal values get high and low so frequently so the
ball also will be going high and down frequently. That can be annoying. Using the
Running Average can reduce the variance and the variance problem can be avoided.
When we calculate the Running Average of those values, we managed to lower that
variance.
The following two charts prove this.
The first chart shows the direct values we retrieved from the device. The second chart
shows the Running Average values we calculated by those.
Figure 8: Attention Level Values
0102030405060708090
1 10 19 28 37 46 55 64 73 82 91 100
109
118
127
136
145
154
163
172
181
190
199
208
217
226
235
244
253
262
271
280
289
298
307
316
325
Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 18
Brain Computer Interface Application Framework
Figure 9: Running Average of Attention Level Values
We can notice the difference between the two charts. The first chart has a high
variance. The second chart, which is drawn using the Running Average values, has a
low variance.
Because of that fact, using the Running Average is better.
There can be occasions when there are sudden drops of signal strength. You can notice
it when the signal value suddenly goes for ‘0’ and then values start to come back again.
During that occasions the Running Average can be a solution. Because of the running
average, you can avoid sudden drops in signal values.
Why this device is not capable of controlling big games.
You may have imagined of big games that get controlled totally by brain signals. Or
you may have imagined when you think a letter, that letter gets automatically printed
on the Computer. But since this device is giving out only the Attention level values,
Meditation level values and Eyeblink values, it can not be done.
01020304050607080
1 10 19 28 37 46 55 64 73 82 91 100
109
118
127
136
145
154
163
172
181
190
199
208
217
226
235
244
253
262
271
280
289
298
307
Running Average of Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 19
Brain Computer Interface Application Framework
The Mind-Controlled Keyboards.
We have developed two keyboards for disabled persons. These will be best suited for Mute Motor-disabled persons. A big challenge these type of disabled persons have is that they find it difficult to communicate with others. And this system we have developed will be helpful for them to communicate with others.
They keyboard can be operated by using Eye-Blinks, which is convenient for the users.
Figure 10 - The simple keyboard
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 20
Brain Computer Interface Application Framework
The previous image shows our simple keyboard. It can be controlled by using Eye-Blinks.
In this keyboard, first row by row gets highlighted. When the user blinks, it goes one by one in the row that was highlighted when the user blinked.
As an example : If the user want to tell someone to “Switch on a Light”, initially the user has to blink when the particular row gets highlighted. Then blink again when the Key which shows an emitted bulb gets highlighted.
Figure 11 - Using the Simple Keyboard.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 21
Brain Computer Interface Application Framework
Next we have developed a keyboard for the purposes of typing messages. This also
can be operated just like the previous keyboard.
Once the message is typed the user can make the Computer pronounce the message by
clicking on “SAY IT”.
Figure 12 - The Advance Keyboard.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 22
Brain Computer Interface Application Framework
Ludo Game we have developed.
We have developed a Ludo game also. The Ludo game is developed to provide
entertainment for the user. The value of the “Die” of this game is generated according
to the Attention level of the user. If the user is having a higher attention level, the Die
will get higher values. If the attention level of the user is low, the Die will get lower
values.
The Attention level values of the user is given to the application by the BCI device
that is worn by the user. Then Running Average of those values are automatically
generated and those Running Average values decide the value that the Die should get.
Figure 13: Ludo Game User Interface
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 23
Brain Computer Interface Application Framework
4 CONCLUSION
For a long time, researchers have been working on a marriage of human and machine
that sounds like something out of science fiction: a brain computer interface. The
technology holds great promise for people who can’t use their arms or hands normally
because they have had spinal cord injuries or suffer from conditions such as
amyotrophic lateral sclerosis (ALS) or cerebral palsy. BCI could help them control
computers, wheelchairs, televisions, or other devices with brain activity.
To success with the projects like above mentioned, the capabilities of the used BCI
devices should be in a high standard level with the fully functional options. Also there
should be many sensors which are capable of capturing brain signal within vast areas
of the brain.
Since we are using the NeuroSky mindwave head set which is contain only one sensor
is not much capable of developing advance systems and applications. And the
generated raw data also not much accurate and reliable as it can be used for a
considerable application since it does not gives a stable value and can not find a certain
pattern in order to use as an average value for controlling an application
Therefore the team decided to implement applications depending on the capabilities of
the using BCI mindwave headset. So the implemented applications are Mind
controlling Calculator and a Keyboard which is working based on the users eye blink.
Also a Ludo Game controlling based on the users attention level and the focus level as
well as a Snakes and Ladders game based on the eye blink. The sound generator is an
application working based on the alpha, beta and gamma brain signals.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 24
Brain Computer Interface Application Framework
5 REFERENCES
1. "Brain-Computer Interfaces: Beyond Medical Applications," 2013. [Online]. Available: http://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-applications.
2. E. Kader, B. P. B. P. Emilie Belley, J.-A. Filion, A. Nutter, M. Parent-Vachon, M. Saulnier, B. P. Stephanie Shedleur, B. P. Tsz Ting Wan, B. B. Elissa Sitcoff and P. O. Nicol Korner-Bitensky, "MOTOR IMAGERY - Information for Patients and Families," 2010. [Online]. Available: http://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdf.
3. "P300 (neuroscience)," 2012. [Online]. Available: http://en.wikipedia.org/wiki/P300_(neuroscience). [Accessed 2013].
4. K. Nakayama and M. Mackeben, "Steady State Visual Evoked Potentials In the Alert Primate," 1982. [Online]. Available: http://visionlab.harvard.edu/members/ken/Ken%20papers%20for%20web%20page/027VisionRes82.pdf.
5. "Event-related Potential: An overview," 2011. [Online]. Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3016705/. [Accessed 2013].
6. "Electroencephalogram (EEG)," WebMD, LLC., 2010. [Online]. Available: http://www.webmd.com/epilepsy/electroencephalogram-eeg-21508.
7. "What Is A Framework?," 2003. [Online]. Available: http://www.codeproject.com/Articles/5381/What-Is-A-Framework.
8. "Neurosky MindWave Mobile," Neurosky, 2012. [Online]. Available: http://neurosky.com/Products/MindWaveMobile.aspx.
9. Microsoft, "Microsoft Visual Studio," Microsoft, 2013. [Online]. Available: http://www.microsoft.com/visualstudio/eng/products/visual-studio-ultimate-2012.
10. S. Du and M. Vuskovic, "Temporal vs. Spectral Approach to Feature Extraction," [Online]. Available: http://medusa.sdsu.edu/Robotics/Neuromuscular/Our_Publications/FE_Sijiang_press.pdf.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 25