ResearchArticle Classification of BCI Users Based on Cognitiondownloads.hindawi.com/journals/cin/2018/6315187.pdf · ComputationalIntelligenceandNeuroscience their primary purpose

Research ArticleClassification of BCI Users Based on Cognition

N Firat Ozkan and Emin Kahya

Industrial Engineering Department Eskisehir Osmangazi University 26480 Eskisehir Turkey

Correspondence should be addressed to N Firat Ozkan fozkanoguedutr

Received 8 January 2018 Revised 21 March 2018 Accepted 3 April 2018 Published 9 May 2018

Academic Editor Justin Dauwels

Copyright copy 2018 N Firat Ozkan and Emin Kahya This is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in any medium provided the original work isproperly cited

Brain-Computer Interfaces (BCI) are systems originally developed to assist paralyzed patients allowing for commands to thecomputer with brain activities This study aims to examine cognitive state with an objective easy-to-use and easy-to-interpretmethod utilizing Brain-Computer Interface systems Seventy healthy participants completed six tasks using a Brain-ComputerInterface system and participantsrsquo pupil dilation blink rate and Galvanic Skin Response (GSR) data were collected simultaneouslyParticipants filled Nasa-TLX forms following each task and task performances of participants were also measured Cognitive stateclusters were created from the data collected using the119870-means method Taking these clusters and task performances into accountthe general cognitive state of each participant was classified as low risk or high risk Logistic Regression Decision Tree and NeuralNetworks were also used to classify the same data in order to measure the consistency of this classification with other techniquesand the method provided a consistency between 871 and 100 with other techniques

1 Introduction

One of the most important issues which ergonomicsaddresses is the reduction of human errors and accidentsHuman errors may have serious consequences from systemdisruptions to loss of life Considering that humans havelimited capacities both physically andmentally it is evidentlyan important necessity to create designs appropriate forhuman capacity regardless of the task In-depth studies onhumanrsquos physical capacity have been performed and arestill being performed For physical fatigue measurementand calculation methods have been developed which havegeneral validity in numerous subjects such as muscle fatiguerest period and audiovisual capabilities However although itis known that humans have limitedmental capacities aswell amethod determiningmental fatigue andmental performanceaccurately is not available yet Although there are someacceptedmethods predicting the cognitive load caused by thetask at hand strengths and weaknesses of these methods areopen to dispute

On the other hand it is not possible to explain themental aspect of human errors solely with cognitive loadAlthough excessive cognitive load is a cause for error factors

which make it more difficult to maintain attention duringperformance such as extreme stress mind wandering anddrowsiness due to monotony should also be considered Forthis reason in addition to ongoing studies on measurementof cognitive load researchers show an interest in the conceptof cognitive state as well which is usually determined withmore than one measure

Within the scope of this study we performed an experi-mental study based on the use of some of themethods used inprediction of cognitive load and cognitive state and utilizingBCI systems BCI systems were utilized for experimentsThe idea was to use the ability of giving commands to thecomputer with brain activities of these systems originallydeveloped for medical purposes and have the participantscomplete tasks during experiments in this way 70 studentsattending Osmangazi University participated in the experi-ment and completed tasks designedwith the Brain-ComputerInterface (BCI) Subjective and objective data collected fromeach participant were evaluated to make a cognitive stateclassification and a risk identification method was proposedbased on instantaneous cognitive states of individuals

BCI systems have the potential to facilitate life for thosewho have severely damaged motor skills in accordance with

HindawiComputational Intelligence and NeuroscienceVolume 2018 Article ID 6315187 10 pageshttpsdoiorg10115520186315187

2 Computational Intelligence and Neuroscience

their primary purpose At this point all BCI applicationsespecially Motor Imagery are beneficial [1] The use of audi-tory stimuli based BCI has also been investigated for peoplewho have serious vision disorders besides limitedmotor skills[2 3] The use of BCI systems by healthy people is also a fieldof interest It can be said that healthy people are quite suc-cessful in using BCI systems especially after training Healthyparticipants were found to be highly successful in the BCIexperiments for each of the P300 SSVEP andMotor Imagerybrain activities [4ndash6] Besides in an experimental work byBai et al [7] participants showed that they could completethe tasks involving typing and selection moves although theyspent a long time due to errors This has led to the researchesin using of BCI systems in various nonmedical fields forvarious purposes These potential nonmedical areas includedevice control user-state monitoring trainingeducationsafetysecurity gaming cognitive effects of music andmediaapplications [8 9] These findings have been supported byWang et al [10] with an experimental study In their studythe authors designed the Brain-Computer Interface as acellphone screen and used SSVEP as the potential requiredfor data entry In the study conducted with ten volunteersthe participants were asked to dial a ten-digit phone numberand press the ENTER key on an interface controlled with theSSVEP potential In another study Perego et al [11] appliedtwo different intelligence tests to 19 volunteers One of thesetests was on paper whereas the other was designed as aBrain-Computer Interface controlled with SSVEP potentialAccording to the results of the study there was no significantdifference between the BCI-based intelligence test and theon-paper test in terms of performance

On the other hand cognitive load detection cognitivestate monitoring and attention performance are on theforefront of the use of BCI systems in nonmedical areas [8]It is possible to reach experimental studies that deal withthese subjects and some outstanding ones were tried to besummarized

In their experimental study Schultheis and Jameson [12]used the P300 potential as an indicator of cognitive load andhad 13 participants read texts on a computer screen Afterthe participants read the text they were asked 7 questionsrelated to the text Pupil size and P300 amplitude weremeasured throughout the experiment Reading speed of textswith increasing difficulty level was computed as behavioralmeasures of cognitive load and the participants were askedto evaluate the difficulty level of tasks subjectively Althoughreading speed and P300 amplitude indicated a cognitiveload as the text difficulty increased no significant changewas observed in pupil size which was interpreted as anunexpected result in the study

In their study Roy et al [13] stated that BCI systemscould be used in cognitive state monitoring and cognitiveload determination and conducted an experimental studyThe authors designed a BCI task where the participants hadto memorize a list of sequential digits visually presented ona screen Some numbers were missing and the participantshad to remember which digit was supposed to be placed inthe empty spot in the sequence The difficulty of the taskwas set by changing the number of missing digits between

2 and 6 Subjective cognitive load data and task completiontimes of the participants were considered It was found thatBCI classification performance inclined as the mental fatiguelevels of the participants increased

Chuang et al [14] designed a BCI system as a drivingsimulation in their study It was highlighted in the studythat changes in the mental state of the BCI users in thestudy led to significant activity changes in the occipitalregion of the brain that is the region related to visionand the mental states of participants were addressed in twoclasses as alertnessdrowsiness In the study in which variousclassification methods were used the Principal ComponentAnalysis (PCA) method showed the highest classificationperformance with 90

Hashemi et al [15] performed an experimental study with5 participants and investigated the feasibility of BCI systemsto detect instants when the driver feels drowsy In this studyutilizing SSVEP potential the participants were invited toa laboratory prepared in accordance with conditions insidea car and asked to focus on the target direction accordingto flickering lights placed on four main directions It washighlighted in the study that SSVEP potential was highlyrelated to attention level The Artificial Neural Networkmethod was used to classify mental state that is detectinstants of drowsiness

Rozado et al [16] examined effects of Motor Imagery onpupil diameter In experiments the participants were asked toimagine they had performed a graspingmotionwith their lefthands and focus on this thought and then to stop this thoughtand to not continue theMotor Imagery process that is to restThe participantsrsquo pupil dilation was recorded in both statesThe results showed that a dilation occurred in pupil duringMotor Imagery and a slight constriction occurred during theresting state These results once again proved that the MotorImagery is a cognitive process

Myrden and Chau [17] investigated the relationshipbetween mental state and task performance of participantsusing BCI According to the experimental results BCI perfor-mance was 7 lower than average when self-reported fatiguewas low and 7 higher than average when self-reportedfrustration was moderate

In their study Lim et al [18] reported that individualswhohad difficulties with using a SSVEP-based BCI technologyhad lower blink rates and reported higher fatigue

Although studies on nonmedical use of BCI systemshave increased in number in recent years experimentalstudies comprehensively using all features of BCI systemshave not been conducted yet From this point we designedan experimental study containing tasks requiring differentbrain activities and different cognitive states and requiringindividuals to complete tasks with different difficulty levelscreating both a drowsiness effect and cognitive loading

2 Materials and Methods

21 Apparatus The participants completed tasks using thegtec BCI system in this study In addition a GSR measure-ment tool and the Tobii X2-60 eye tracker were used in order

Computational Intelligence and Neuroscience 3

to measure pupil diameter remotely For all the tasks thesame 1710158401015840 laptop computer was used

22 Participants Seventy students between the ages of 18ndash35participated in the experiment on a voluntary basis 40participants were female and 30 participants weremale All ofthe participants were undergraduate or graduate students atIndustrial Engineering Department Osmangazi University64 participants were undergraduate students whereas 6 par-ticipantswere graduate students Also 9 participants had full-or part-time jobs in addition to their studies Additionally 28participants had an ECTS credit load over 30 8 participantshad a Grade Point Average (GPA) below 200 51 had a GPAbetween 200 and 300 and 11 had a GPA of 300 or above

23 Procedure It was possible to assess the SSVEP potentialas a support to the P300 potential which is one of thepotentials associatedwith cognitive loadmost frequently andthe Motor Imagery application with the BCI system used inthe study The following physiological data were measuredand recorded simultaneously in order to predict how muchdifficulty the participants had while completing the tasks bygiving commands to the computer with brain activities

(i) Change in pupil diameter

(ii) Blink rate

(iii) Galvanic Skin Response

We asked the participants to complete the tasks inBCI applications and conducted classification and predictionstudies based on mental loading data during these tasksin order to determine their current cognitive state andperformance in tasks requiring attention andmental effort inthis state In experiments the participants completed threeBCI applications twice each with different parameters anddifficulty levels with 2- to 5-minute rest periods (P300SSVEP and Motor Imagery) Participants filled the Nasa-TLX form after each application Also signal controls wereperformed before each BCI application

24 Measures In order to calculate the change in pupildiameter the average diameter in the first 1 second of thetask was used as the base value and the average value in theremainder of the task was proportional to this base valueTo find the blink rate records taken by the eye tracker with17-millisecond intervals were used Instants when the datarelated to eye movements were missing (ie the lines wereblank) were accepted as instants of blinking and these lineswere counted to find the blink rate

The data normalized with the method proposed byNourbakhsh et al [19] were used to calculate the GSR valueTo this end the GSR value in each second of each task wasdivided into the average GSR value of all tasks completed bythe individual in question Then these values obtained foreach second of the relevant task were summed and dividedinto taskrsquos total length

3 Tasks

The BCI system used in the study creates the P300 potentialvia visual stimuli A character matrix is used for this andthe participants spell words on the screen via this matrixThe spelling of the target character is made possible by therecurring emergence of the P300 potential Before startingthis task we had the participants spell a random characteron the screen initially to familiarize the participants withthe matrix Then the participants were given the task ofspelling twowordsThefirst task to complete in this studywasdefined as spelling the characters ABCDE which were sideby side on the matrix During this task the number of visualstimuli sent by target characters that is flashing was set to30 The length of visual stimulus was set to 100 millisecondsAlso the character duplication method was used in thisfirst task Accordingly target characters were shown on thescreen and the place of the target character was shown to theparticipants by flashing by itself for a second initially Thefree spelling method was used in the second task In otherwords the character to be spelled by the participant was notshown on the screen Also flashing count was set to 10 andflashing length was set to 50 milliseconds in order to have theparticipants make more effort The array of characters to bespelled on the screen in the second task was determined to beESOGU2014This task was explained to the participants onlyverbally Thus the participants had to remember the order ofthe characters and find the place of the relevant character onthematrix on hisher own Figure 1 illustrates a sample screenof P300 application

The SSVEP potential occurs by focusing on stimuliwith fixed visual frequency To this end LEDs representingdirections need to flash on a box showing up down rightand left and the participants need to follow stimuli comingfrom the target direction The participants had to completetwo tasks the first requiring a shorter focus and the secondrequiring a longer focus Difficulty levels of the tasks wereset by changing the length and interval of the stimuliThe participants focused on LEDs flashing in the order ofleft up right and down in the first task There were 105seconds between two directions In the first task 70 of these105 seconds were determined as stimulus length and theremaining 30 were waiting period in transitions betweentwo directions In the second iteration the visual stimuluslength was set to 95 and the transition time was set to 5 toensure longer focus time and faster transition SSVEP controldevice can be seen in Figure 2

EssentiallyMotor Imagery is an event-related desynchro-nizationsynchronization (ERDERS) occurring in the brainThe participants are guided by the 119909-axis on the 119909-119910 plainon the screen A red arrow appearing on the 119909-axis once in7 seconds shows the participants which direction (right andleft) to focus on The participants focus on moving the handon the target direction for 7 seconds however they do notmake an actual motion with their hands After 20 iterationsin the first task the second Motor Imagery task started Thistime an additional blue arrow providing feedback to theparticipants was also present The blue arrow appearing oncethe red arrow shows the direction moves to the correct or the


Figure 1 P300 letter matrix

Figure 2 SSVEP control device

opposite direction depending on the participantsrsquo focusingsuccess A screenshot of Motor Imagery task is presented asFigure 3

The main purpose of the study is to give an opinionabout the cognitive state of an individual to complete a taskrequiring mental effort (thinking interpreting calculatingdecision-making etc) and attention before said task andperformance of the individual during said task A summaryof task properties is presented in Table 1

Nasa-TLX scores were used to evaluate difficulty levels oftasks completed Nasa-TLX scores showed a normal distribu-tion in at least one normality test and thus were analyzed withpaired 119905-test The results are shown in Table 2 and revealed asignificant difference as expected that is the second task wasmore difficult compared to the first task in all task types SPSS22 software was used for paired 119905-test analyses

Task performances of the participants (Table 3) wereassessed based on correctly spelled characters in P300 taskswhich involve spelling on a screen In SSVEP and MItasks successful classification percentage during the task was

Figure 3 Screenshot of Motor Imagery task

used as the indicator of task performance The performanceclassification recommended by gtec Brain-Computer Inter-face system consisting of 3 classes (bad-good-excellent) wasutilized for interpretations These performance classes arebased on Guger and Edlinger [4] Guger et al [5] and Gugeret al [6] In these studies the success level of individualsin P300 SSVEP and Motor Imagery tasks was investigatedFor each of the 3 task types at least 50 of the participantsachieved 70 classification success and thus those whoremained below this limit were accepted as low performancethose who achieved a classification success between 70 and90 were accepted as good and those who achieved aclassification success above 90 were accepted as excellentSince the aim was to achieve a classification consisting of twoclasses within the scope of this study 70was accepted as thethreshold value and below this level was interpreted as lowperformance

31 Proposed Method for Classification of the ParticipantsData mining is a collection of methods used to extractimportant data by analyzing large data sets which allowsfor obtaining and interpreting information [20] It is com-monly used thanks to algorithms developed and programsfacilitating the use of these algorithms Classification andclustering are among the most commonly used data miningmethods In classification classes are predetermined andobservations are assigned to appropriate classes within cer-tain rules Commonly used classification methods includeLogistic Regression Bayes classifiers Linear DiscriminantAnalysis Decision Tree and Artificial Neural NetworksClustering is the process of dividing data into groups accord-ing to similarity In general algorithms which allow forminimizing the distance between the data in the same clusterare used Clustering analysis may be defined as optimizationproblems where variables are cluster membership of datapoints and the objective function is tomaximize the similaritybetween members assigned to these clusters [21] There arehierarchical and nonhierarchical methods in clustering Thebest known hierarchical methods include Nearest Neighborand Furthest Neighbor algorithms The 119870-means algorithmcan be listed as a nonhierarchical method


Table 1 Task properties and expected physiological effects

Task Task properties Expected main risk factors Expected physiological effects

P3001(i) Creating cognitive load(ii) Constant attention requirement(iii) Intense visual stimulus

(i) Excessive cognitive loading(ii) Increased stress in case of spellingincorrectly

(i) Pupil dilation(ii) Increase in GSR value(iii) Increase in blink rate

P3002(i) Creating cognitive load(ii) Constant attention requirement(iii) Very intense visual stimulus


(i) Pupil dilation(ii) Increase in GSR value(iii) Decrease in blink rate

SSVEP1(i) Monotony(ii) Constant attention requirement(iii) Intense visual stimulus

(i) Drowsiness effect(i) Pupil contraction(ii) Decrease in GSR value(iii) Decrease in blink rate



MI1(i) Focusing on a thought(ii) Constant attention requirement(iii) Low intensity visual stimulus

(i) Drowsiness effect(ii) Inability to focus

(i) Pupil contraction(ii) Decrease in GSR value(iii) Decrease in blink rate

MI2

(i) Focusing on a thought(ii) Constant attention requirement(iii) Constantly following an additionalstimulus on the screen

(i) Excessive cognitive loading(ii) Increased stress due to instantfeedback


Table 2 Paired 119905-test results for Nasa-TLX scores

Mean Std dev Std error mean 95 confidence interval119905 df 119875

Lower UpperP 3001 Nasa-TLXP 3002 Nasa-TLX minus18000 13418 1604 minus21199 minus14800 minus11223 69 000

SSVEP1 Nasa-TLXSSVEP2 Nasa-TLX minus8929 9217 1102 minus11127 minus6731 minus8105 69 000

MI1 Nasa-TLXMI2 Nasa-TLX minus12439 10116 1209 minus14851 minus10026 minus10287 69 000

Table 3 Task performances ()

Tasks Min Max Avg Std devP 3001 0 100 5629 3236P 3002 0 8889 2936 2587SSVEP1 53 100 7099 995SSVEP2 56 100 7121 925MI1 68 85 7566 394MI2 70 100 7676 672

Within the scope of this study the participants wereclustered with the 119870-means method using physiologicalmeasurement data A classification method was proposedtaking task performances of participants in these clusters intoaccount It was expected that the proposed method would bea flexible multidimensional and comprehensive tool for BCIstudies that can include implementations addressing variouscognitive states and brain activitiesThe steps of the proposedmethod are as follows

(i) Collecting physiological data in order to interpretcognitive state

(ii) Determining Silhouette scores in order to achieveclusters of 2 3 and 4 with this data (Table 4)

(iii) Selecting the cluster number with the best score andtagging these clusters by interpreting in terms ofcognitive state

(iv) Dividing task performances into at least two classes

(v) Assessing cognitive states and task performances anddetermining individuals in the positive class in both

Clusters of 2 were interpreted according to cognitiveeffort required by the task and cognitive state expected to posea risk

(i) The cluster indicating a higher cognitive loading forP300 tasks whichmostly require cognitive effort wastagged as the risky cluster

(ii) Since the main risk was monotony for the SSVEPtask due to its nature the group where physiologicalindicators opposite to cognitive loading concentratedwas tagged as risky

(iii) Since the cognitive demand was lower and the con-centration requirement was higher in the Motor


Imagery1 task the cluster showing a tendency oppo-site to cognitive loading was tagged as the riskycluster

(iv) Since the Motor Imagery2 task had an indicatorproviding instantaneous feedback during concentra-tion and followed constantly which resulted in moresources requiring cognitive effort the group showinga cognitive loading tendency was tagged as risky

(v) In case all physiological indicators supported theinterpretation when interpreting clusters the pupildiameter change was used as base because it showedsignificant differences both between different diffi-culty levels of the same task and between different tasktypes in previous statistical analyses and stood out asthe most sensitive physiological measure

For all tasks clusters which seemed more risky weretagged with 0 and clusters which seemed less risky weretagged with 1 taking expected physiological outcomes intoaccount (Table 5)

Individualsrsquo cognitive states and task performances wereconsidered to determine task risk classes The combinationof Risk-free Cognitive State-High Performance (70 successat least) was required for the low-risk class All other combi-nations were assigned to the high-risk class In this way theparticipants whose cognitive state posed a low risk due to theproperties of the task and whose task performance was highin tasks requiring constant attention and concentration wereaccepted to be low-risk for mental-weighted tasks in general

Risk classes of all 6 tasks were taken into account indetermination of general risk classes In this classification the50 cut-off condition also used in classificationwith LogisticRegression was applied that is those who were in the low-risk group for 3 or more of 6 tasks were assigned to the lowgeneral risk classThosewho did notmeet this conditionwereassigned to the high general risk class Thus 59 out of 70participants were assigned to the high-risk class whereas 11participants were assigned to the low-risk class

32 Consistency of the Proposed Method The consistencyof the proposed classification was assessed by comparingthe classification to Logistic Regression Decision Tree andArtificial Neural Network methods respectively The aim ofthis benchmarkingwas to confirm the technical validity of theproposed methodrsquos steps through different and well-knownclassification methods that are using various approaches ForLogistic Regression classification consistency was examinedby creating a binary model where physiological indicatorsand task performance values were used as independentvariables in the analysis which was used for classification inthis study and employed the ENTER method which adds allvariables to the model at the same time as a block The cut-off value was 050 in the first trial and 070 in the secondtrial The same classification table formed in both trials andthe classification consistency was 100 (Table 7) ModelrsquosNagelkerke 1198772 value was found to be 10 that is the modelexplained the variance in the dependent variable (risk classes)completely According to the results of the Omnibus test

Table 4 Silhouette scores

Tasks Cluster numbers Silhouette scores Percentage

P 30012 065 82503 057 78504 064 82

P 30022 060 803 054 774 059 7950

SSVEP12 078 893 059 79504 060 80

SSVEP22 059 79503 058 794 055 7750

MI12 059 79503 059 79504 058 79

MI22 060 803 052 764 060 80

(Table 6) which shows the validity of the model the modelwas significant with 95 confidence interval

Following Logistic Regression the Decision Tree methodwas used to test prediction consistency At this stage trialswere made for CHAID CART and QUEST algorithms pro-vided by SPSS 22 package program In decision trees formedwith task performance percentage values and physiologicalmeasurement data the CHAID algorithm provided 90 theCART algorithm provided 100 and the QUEST algorithmprovided 871 prediction accuracy (Table 8)

Following Logistic Regression and Decision Tree thelast method used to test the consistency of the proposedclassification method based on cognitive state clusters andtask performances was Artificial Neural Networks

At this stage of the study amultilayered network structurewas used with the Hyperbolic Tangent as the hidden layeractivation function and Softmax as the output layer Inthis application which used physiological measurements andtask performances as input the entire data was allocated astraining set Trials were made by creating multilayered net-work structures with Batch Online and Mini-Batch trainingmethods included in SPSS 22 package program and the Batchtraining method provided 100 the Online training methodprovided 971 and the Mini-Batch method provided 986prediction accuracy (Table 9)

When the results of the three methods are examined itcan be seen that some differences appeared in the decisionpoints for low-risk class of the methods used In trials thatfailed to be 100 consistent only Artificial Neural Network(Online)method assigned a high risky participant to the low-risk class When the data of this participant were examinedit was seen that the participant achieved high success in thetasks of P300 and SSVEP compared to average performanceIn the case of all the remaining inconsistent classifications


Table 5 Clustering results

Tasks Physiological indicators Min Max Avg Std dev Cluster center (0) Cluster center (1)

P 3001P 3001 pupil dilation () minus372 3693 11364 875 2084 576

P 3001 blink rate per second 002 03 0064 0052 008 006P 3001 normalized GSR 0001 002 0005 0003 47 times 10minus4 46 times 10minus4



SSVEP1SSVEP1 pupil dilation () minus2367 5068 minus4909 12069 minus721 3299

SSVEP1 blink rate per second 002 0257 0056 0062 005 015SSVEP1 normalized GSR 00003 0024 0004 0003 4 times 10minus3 85 times 10minus4



MI1MI1 pupil dilation () minus2328 2216 minus1956 9026 minus735 776

MI1 blink rate per second 002 048 0061 0104 004 01MI1 normalized GSR 0001 001 0004 0002 39 times 10minus4 41 times 10minus4

MI2MI2 pupil dilation () minus2062 4063 2947 9589 1482 minus148


Table 6 Omnibus test

Chi-square df 119875

Step 60886 24 0000Block 60886 24 0000Model 60886 24 0000

low risky participants were assigned to the high-risk classWhen these participantsrsquo datawere examined it was observedthat their pupil dilations were dominant especially for P300and SSVEP tasks

4 Conclusion

Errors and accidents caused by human error may occurin every area of life These errors may have more seriousconsequences for both human health and performance ofthe system in areas such as production or transportationHuman error has more than one dimension These dimen-sions include the design and conditions of the working envi-ronment physical fatigue mental fatigue personrsquos cognitivestate and jobrsquos cognitive requirementsThe factor which is themost difficult tomonitor and detect is personrsquos cognitive stateCognitive state leads to risks at different levels dependingon the job such as excessive cognitive loading drowsinessdue to jobrsquos monotony distraction and lack of concentrationMost studies in this field focus on detection of drowsiness ofvehicle drivers In addition various approaches are discussedin studies on detection of cognitive load and mental fatigue

Although there are methods with an accepted generalvalidity such as Nasa-TLX among subjective methods theneed for stronger tools has encouraged researchers to findnew ways Behavioral measures such as monitoring task

performance and monitoring physical indicators related toautonomous nervous system have emerged as alternatives asa result of this search Particularly physiological indicatorsrelated to autonomous nervous system are of great interestdue to their objective nature Physiological methods such asheart rate variability respiration rate pupil diameter changesblink rate and duration and Galvanic Skin Response standout in studies on detection of cognitive load and cognitivestate On the other hand brain activities are preferred to col-lect data more directly The fact that developing technologyallows not only for tracking brain waves but also for imagingbrain activities in detail has steered studies to this directionAgain with the effect of developing technology the possibilityof tracking a measure with proven sensitivity such as pupildiameter rapidly and accurately provides a great advantagefor researchers in this field

Although the number of methods and devices to use totrack physiological indicators and brain activities increasesthis does not resolve the complexity of interpreting physio-logical indicators in particular Such indicators and activitiesare affected from environmental factors very quickly andthus data collection and interpretation are more possible incontrolled laboratory environments In particular interpret-ing brain activities requires expertise and it is not easy to usebrain activities in studies aimed directly at working life

At this point Brain-Computer Interfaces which wereoriginally developed to facilitate the lives of stroke patientsand allow for controlling computers and devices with brainactivities are of great interest for researchers conductingcognitive studies These interfaces which can be used withcertain special brain activities have begun to be used incognitive state and cognitive state detection studies in recentyears Detecting whether the relevant brain activity directlyoccurs via interfaces apart from the signal data means easierinterpretation of data provided by these devices


Table 7 Classification results of binary logistic regression

ObservedPredicted

General risk class Percent correctHigh Low

Step 1 General risk class High 59 0 1000Low 0 11 1000

Table 8 Classification results of decision tree

Observed PredictedHigh risk Low risk Percent correct

CHAID algorithmHigh risk 59 0 1000Low risk 7 4 364

Overall percent 943 57 900

CART algorithmHigh risk 59 0 1000Low risk 0 11 1000


QUEST algorithmHigh risk 59 0 1000Low risk 9 2 182


Table 9 Classification through artificial neural networks

PredictedHigh risk Low risk Percent correct

Batch TrainingHigh risk 59 0 1000Low risk 0 11 1000


Online TrainingHigh risk 58 1 983Low risk 1 10 909


Mini-Batch TrainingHigh risk 59 0 1000Low risk 1 10 909


Six tasks were developed using a BCI system within thescope of this study and data related to pupil diameter blinkrate and normalized GSR were recorded simultaneously Inthis way we ensured that tasks directly requiring attentionand concentration were completed directly with brain activi-ties and advantages of both physiological measurements andbehavioral measurements were drawn together Nasa-TLXthe best known subjective method was used as supplement

The data were clustered in later steps The119870-means tech-nique was used in clustering analyses and it was found withthe help of Silhouette scores that the strongest clustering wasobtained when the cluster count was 2 After forming clusterswith physiological data of the participants for each taskthe clusters were examined individually and the participantsrsquostates such as cognitive load drowsiness and concentrationwere interpreted The participantsrsquo expected cognitive statedue to the nature of the task and the expected physiologicaleffects of these states were taken into account when makingthese interpretations

Once clusters were formed and participants in the nega-tive group and the positive group were tagged for each taskcognitive risk classes were created for each task At this stagecognitive states andBCI task performances of the participantswere comparedThosewho had both a positive cognitive stateand a high task performance were assigned to the positivecognitive risk classThosewith negative cognitive state andorlow task performance were assigned to the negative cognitiverisk class Thus task performances of the participants intasks requiring attention and their cognitive states whenperforming these tasks were considered together Generalrisk classes on the other hand were created by seeking a 50cut-off condition in cognitive risk classes created for 6 tasksThose who were assigned to the positive cognitive risk classin at least 3 of 6 tasks were assigned to the positive (low-risk)general risk classThosewho did notmeet this conditionwereassigned to high general risk class

At the next stage this risk classification was compared toother classification methods Two trials (cut-off conditions


50 and 70) with Binary Logistic Regression the first ofthesemethods provided 100 consistencyTheDecisionTreemethod the second of these methods was tested with 3different algorithms provided by SPSS 22 package programOne of these algorithms the CRT algorithm showed 100consistency with our classification The last classificationmethod was the Artificial Neural Networks method Theclassification made with the Batch training method by theArtificial Neural Network created with 100 training dataprovided 100 consistency

This study constitutes an example for nonmedical useof BCI systems This experimental method preserving thereliability of brain activities in studies on cognitive state byremoving the interpretation difficulty was also supported byphysiological indicators Another advantage of the methodwas that the tasks performed during the experiment wereindependent of skill levels of the participants and completeddirectly with brain activities

BCI systems are expected to be used much more fre-quently in studies on cognitive loading and cognitive statein the near future The flexibility of these systems makesit possible to sector- and purpose-specific interfaces It isalso possible to create authentic simulation environmentsby integrating BCI systems with three-dimensional virtualenvironments Ease of application and interpretation of thesesystems indicates that they will find their place in workinglife

Additional Points

Practitioner Summary Brain-Computer Interfaces are sys-tems in which usage possibilities are investigated in differentareas Experiments using these systems have been carried outto comment on the cognitive status of healthy people in thisstudyThe outputs of the experiments were evaluated throughdata mining techniques

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

Acknowledgments

This study was supported by ESOGU Scientific ResearchProjects Office as a project (no 201315A208)

References

[1] T Mulder ldquoMotor imagery and action observation cognitivetools for rehabilitationrdquo Journal of Neural Transmission vol 114no 10 pp 1265ndash1278 2007

[2] M Schreuder A Riccio M Risetti et al ldquoUser-centered designin brain-computer interfaces-A case studyrdquoArtificial Intelligencein Medicine vol 59 no 2 pp 71ndash80 2013

[3] H Xu D Zhang M Ouyang and B Hong ldquoEmploying anactive mental task to enhance the performance of auditoryattention-based brain-computer interfacesrdquoClinicalNeurophys-iology vol 124 no 1 pp 83ndash90 2013

[4] C Guger and G Edlinger ldquoHow many people can control abrain-computer interface BCIrdquo in Proceedings of the BrainPlayconference pp 29ndash32 2007

[5] C Guger S Daban E Sellers et al ldquoHow many people areable to control a P300-based brain-computer interface (BCI)rdquoNeuroscience Letters vol 462 no 1 pp 94ndash98 2009

[6] C Guger B Z Allison B Groszligwindhager et al ldquoHow manypeople could use an SSVEP BCIrdquo Frontiers in NeuroscienceArticle ID Article 169 2012

[7] L Bai T Yu and Y Li ldquoA brain computer interface-basedexplorerrdquo Journal of Neuroscience Methods vol 244 pp 2ndash72015

[8] B Blankertz M Tangermann C Vidaurre et al ldquoThe Berlinbrain-computer interface non-medical uses of BCI technol-ogyrdquo Frontiers in Neuroscience vol 4 article 198 2010

[9] J B F Van Erp F Lotte and M Tangermann ldquoBrain-computerinterfaces beyondmedical applicationsrdquoTheComputer Journalvol 45 no 4 pp 26ndash34 2012

[10] Y-TWang YWang and T-P Jung ldquoA cell-phone-based brain-computer interface for communication in daily liferdquo Journal ofNeural Engineering vol 8 no 2 Article ID 025018 2011

[11] P Perego A C Turconi G Andreoni et al ldquoCognitive abilityassessment by brain-computer interface validation of a newassessment method for cognitive abilitiesrdquo Journal of Neuro-science Methods vol 201 no 1 pp 239ndash250 2011

[12] Schultheis H and Jameson A ldquoAssessing Cognitive Load inAdaptive Hypermedia Systems Physiological and BehavioralMethodsrdquo in Proceedings of the International Conference onAdaptive Hypermedia and Adaptive Web-Based Systems (AH2004) pp 225ndash234 2004

[13] R N Roy S Bonnet S Charbonnier and A Campagne ldquoMen-tal fatigue and working memory load estimation interactionand implications for EEG-based passive BCIrdquo in Proceedings ofthe 35th Annual International Conference of the IEEE Engineer-ing in Medicine and Biology Society (EMBC rsquo13) pp 6607ndash6610IEEE Osaka Japan July 2013

[14] C-H Chuang P-C Lai L-W Ko B-C Kuo and C-T LinldquoDriverrsquos cognitive state classification toward brain computerinterface via using a generalized and supervised technologyrdquoin Proceedings of the 2010 6th IEEE World Congress on Com-putational Intelligence WCCI 2010 - 2010 International JointConference on Neural Networks IJCNN 2010 Barcelona SpainJuly 2010

[15] A Hashemi V Saba and S N Resalat ldquoReal time driverrsquosdrowsiness detection by processing the EEG signals stimulatedwith external flickering lightrdquo Basic and Clinical Neurosciencevol 5 no 1 pp 22ndash27 2014

[16] D Rozado A Duenser and BHowell ldquoErratum Improving theperformance of an EEG-based motor imagery brain computerinterface using task evoked changes in pupil diameter (PLoSONE (2015) 107 (e0133095)rdquo PLoS ONE vol 10 no 7 2015

[17] A Myrden and T Chau ldquoEffects of user mental state on EEG-BCI performancerdquo Frontiers in Human Neuroscience vol 9 noJUNE article no 308 2015

[18] J-H Lim H-J Hwang C-H Han K-Y Jung and C-H Im ldquoClassification of binary intentions for individualswith impaired oculomotor function lsquoeyes-closedrsquo SSVEP-basedbrainndashcomputer interface BCIrdquo Journal of Neural Engineeringvol 102 Article ID 026021 2013

[19] N Nourbakhsh Y Wang F Chen and R A Calvo ldquoUsinggalvanic skin response for cognitive load measurement in


arithmetic and reading tasksrdquo in Proceedings of the 24th Aus-tralian Computer-Human Interaction Conference OzCHI 2012pp 420ndash423 Australia November 2012

[20] A Albayrak and S Yılmaz ldquoVeri Madenciligi Karar AgacıAlgoritmaları ve IMKB Verileri Uzerine Bir UygulamardquoSuleyman Demirel Universitesi Iktisadi ve Idari BilimlerFakultesi Dergisi vol 14 no 1 2009

[21] C C Aggarwal Data Mining The Textbook Springer 2015

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems


Volume 2018


ReconfigurableComputing



Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018


Civil EngineeringAdvances in


Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications


Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia


Biomedical Imaging



Engineering Mathematics


RoboticsJournal of



Computational Intelligence and Neuroscience


Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018


Human-ComputerInteraction

Advances in


Scientic Programming

Submit your manuscripts atwwwhindawicom


their primary purpose At this point all BCI applicationsespecially Motor Imagery are beneficial [1] The use of audi-tory stimuli based BCI has also been investigated for peoplewho have serious vision disorders besides limitedmotor skills[2 3] The use of BCI systems by healthy people is also a fieldof interest It can be said that healthy people are quite suc-cessful in using BCI systems especially after training Healthyparticipants were found to be highly successful in the BCIexperiments for each of the P300 SSVEP andMotor Imagerybrain activities [4ndash6] Besides in an experimental work byBai et al [7] participants showed that they could completethe tasks involving typing and selection moves although theyspent a long time due to errors This has led to the researchesin using of BCI systems in various nonmedical fields forvarious purposes These potential nonmedical areas includedevice control user-state monitoring trainingeducationsafetysecurity gaming cognitive effects of music andmediaapplications [8 9] These findings have been supported byWang et al [10] with an experimental study In their studythe authors designed the Brain-Computer Interface as acellphone screen and used SSVEP as the potential requiredfor data entry In the study conducted with ten volunteersthe participants were asked to dial a ten-digit phone numberand press the ENTER key on an interface controlled with theSSVEP potential In another study Perego et al [11] appliedtwo different intelligence tests to 19 volunteers One of thesetests was on paper whereas the other was designed as aBrain-Computer Interface controlled with SSVEP potentialAccording to the results of the study there was no significantdifference between the BCI-based intelligence test and theon-paper test in terms of performance

On the other hand cognitive load detection cognitivestate monitoring and attention performance are on theforefront of the use of BCI systems in nonmedical areas [8]It is possible to reach experimental studies that deal withthese subjects and some outstanding ones were tried to besummarized

In their experimental study Schultheis and Jameson [12]used the P300 potential as an indicator of cognitive load andhad 13 participants read texts on a computer screen Afterthe participants read the text they were asked 7 questionsrelated to the text Pupil size and P300 amplitude weremeasured throughout the experiment Reading speed of textswith increasing difficulty level was computed as behavioralmeasures of cognitive load and the participants were askedto evaluate the difficulty level of tasks subjectively Althoughreading speed and P300 amplitude indicated a cognitiveload as the text difficulty increased no significant changewas observed in pupil size which was interpreted as anunexpected result in the study

In their study Roy et al [13] stated that BCI systemscould be used in cognitive state monitoring and cognitiveload determination and conducted an experimental studyThe authors designed a BCI task where the participants hadto memorize a list of sequential digits visually presented ona screen Some numbers were missing and the participantshad to remember which digit was supposed to be placed inthe empty spot in the sequence The difficulty of the taskwas set by changing the number of missing digits between

2 and 6 Subjective cognitive load data and task completiontimes of the participants were considered It was found thatBCI classification performance inclined as the mental fatiguelevels of the participants increased

Chuang et al [14] designed a BCI system as a drivingsimulation in their study It was highlighted in the studythat changes in the mental state of the BCI users in thestudy led to significant activity changes in the occipitalregion of the brain that is the region related to visionand the mental states of participants were addressed in twoclasses as alertnessdrowsiness In the study in which variousclassification methods were used the Principal ComponentAnalysis (PCA) method showed the highest classificationperformance with 90

Hashemi et al [15] performed an experimental study with5 participants and investigated the feasibility of BCI systemsto detect instants when the driver feels drowsy In this studyutilizing SSVEP potential the participants were invited toa laboratory prepared in accordance with conditions insidea car and asked to focus on the target direction accordingto flickering lights placed on four main directions It washighlighted in the study that SSVEP potential was highlyrelated to attention level The Artificial Neural Networkmethod was used to classify mental state that is detectinstants of drowsiness

Rozado et al [16] examined effects of Motor Imagery onpupil diameter In experiments the participants were asked toimagine they had performed a graspingmotionwith their lefthands and focus on this thought and then to stop this thoughtand to not continue theMotor Imagery process that is to restThe participantsrsquo pupil dilation was recorded in both statesThe results showed that a dilation occurred in pupil duringMotor Imagery and a slight constriction occurred during theresting state These results once again proved that the MotorImagery is a cognitive process

Myrden and Chau [17] investigated the relationshipbetween mental state and task performance of participantsusing BCI According to the experimental results BCI perfor-mance was 7 lower than average when self-reported fatiguewas low and 7 higher than average when self-reportedfrustration was moderate

In their study Lim et al [18] reported that individualswhohad difficulties with using a SSVEP-based BCI technologyhad lower blink rates and reported higher fatigue

Although studies on nonmedical use of BCI systemshave increased in number in recent years experimentalstudies comprehensively using all features of BCI systemshave not been conducted yet From this point we designedan experimental study containing tasks requiring differentbrain activities and different cognitive states and requiringindividuals to complete tasks with different difficulty levelscreating both a drowsiness effect and cognitive loading

2 Materials and Methods

21 Apparatus The participants completed tasks using thegtec BCI system in this study In addition a GSR measure-ment tool and the Tobii X2-60 eye tracker were used in order






(ii) Blink rate





3 Tasks






























MI2































P 30012 065 82503 057 78504 064 82

P 30022 060 803 054 774 059 7950

SSVEP12 078 893 059 79504 060 80

SSVEP22 059 79503 058 794 055 7750

MI12 059 79503 059 79504 058 79

MI22 060 803 052 764 060 80























Step 60886 24 0000Block 60886 24 0000Model 60886 24 0000


4 Conclusion








ObservedPredicted



























Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in









(ii) Blink rate





3 Tasks






























MI2































P 30012 065 82503 057 78504 064 82

P 30022 060 803 054 774 059 7950

SSVEP12 078 893 059 79504 060 80

SSVEP22 059 79503 058 794 055 7750

MI12 059 79503 059 79504 058 79

MI22 060 803 052 764 060 80























Step 60886 24 0000Block 60886 24 0000Model 60886 24 0000


4 Conclusion








ObservedPredicted



























Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in






























MI2































P 30012 065 82503 057 78504 064 82

P 30022 060 803 054 774 059 7950

SSVEP12 078 893 059 79504 060 80

SSVEP22 059 79503 058 794 055 7750

MI12 059 79503 059 79504 058 79

MI22 060 803 052 764 060 80























Step 60886 24 0000Block 60886 24 0000Model 60886 24 0000


4 Conclusion








ObservedPredicted



























Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in




















MI2































P 30012 065 82503 057 78504 064 82

P 30022 060 803 054 774 059 7950

SSVEP12 078 893 059 79504 060 80

SSVEP22 059 79503 058 794 055 7750

MI12 059 79503 059 79504 058 79

MI22 060 803 052 764 060 80























Step 60886 24 0000Block 60886 24 0000Model 60886 24 0000


4 Conclusion








ObservedPredicted



























Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in














P 30012 065 82503 057 78504 064 82

P 30022 060 803 054 774 059 7950

SSVEP12 078 893 059 79504 060 80

SSVEP22 059 79503 058 794 055 7750

MI12 059 79503 059 79504 058 79

MI22 060 803 052 764 060 80























Step 60886 24 0000Block 60886 24 0000Model 60886 24 0000


4 Conclusion








ObservedPredicted



























Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in





















Step 60886 24 0000Block 60886 24 0000Model 60886 24 0000


4 Conclusion








ObservedPredicted



























Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in






ObservedPredicted



























Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in








Additional Points




Acknowledgments


References





























Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in













Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in









Advances in

FuzzySystems


Volume 2018













Journal of

Journal of



Hindawi


Advances in

Multimedia


Biomedical Imaging





RoboticsJournal of









Volume 2018



Advances in




Documents

ResearchArticle Classification of BCI Users Based on Cognitiondownloads.hindawi.com/journals/cin/2018/6315187.pdf · ComputationalIntelligenceandNeuroscience their primary purpose