Navigational BCI Using Acoustic Stimulation

1

Auditory BCI for Navigation using ERP classification

By Mario Vivero

2

Outline

• Description • Methodology • Results• Summary and Conclusion• Questions

Adapted from: http://www.tobi-project.org/sites/default/files/public/Publications/TOBI-159.pdf

3

AMUSE paradigm

AMUSE paradigm Stimulation Setup

Characters chosen in two rounds. 6-class selection

Custom designed maze. All textures and 3D objects were downloaded from: http://www.turbosquid.com

4

Navigational EnvironmentUpper Perspective

5

Auditory Navigational BCI based on the AMUSE paradigm

Navigational Setup for Acoustic Stimulation

Speaker #1

Speaker #2

Speaker #3Speaker #4

Speaker #5

1.- Forward Movement. 2.- Forward Right Movement. 3.- Turn Right. 4.- Turn Left. 5.- Forward Left Movement

6

Experimental ConditionsAuditory

Stimulation (5 Full Rounds)

Signal Processing & Classification

Control Strategy

Discrete Movement

Auditory Stimulation (5 Rounds)

Signal Processing & Classification

Discrete Condition

…

7

Experimental ConditionsAuditory

Stimulation (5 Full Rounds)

Signal Processing & Classification

Control Strategy

Discrete Movement

Auditory Stimulation (5 Rounds)

Signal Processing & Classification

Discrete Condition

…

Continuous Stimulation

Signal Processing & Classification

Control Strategy

Continuous Movement

Continuous Stimulation

Signal Processing & Classification

Continuous Condition

…

8

Experimental ConditionsAuditory

Stimulation (5 Full Rounds)

Signal Processing & Classification

Control Strategy

Discrete Movement

Auditory Stimulation (5 Rounds)

Signal Processing & Classification

Discrete Condition

…

Continuous Stimulation

Signal Processing & Classification

Control Strategy

Continuous Movement

Continuous Stimulation

Signal Processing & Classification

Continuous Condition

…

Joystick - Benchmark ConditionInput

MovementControl Strategy

Continuous Movement

9

Auditory Stimulation and Signal ProcessingEach stimulus lasted 40 ms. SOA was set to 175 ms.

The calibration phase consisted of an auditory oddball task without visual feedback.

Binary classification of target and non-target epochs was performed using a Linear Discriminant Analysis (LDA). Due to the data high dimensionality a shrinkage method was also applied.

10

Control Strategy

Penalization

Continuous

No Penalization - Discrete

W =

D’ = D ⊙ W

D ∈ R C X B

W ∈ R C X B

11

Control Strategy

Penalization

Sigmoid Function Transformation

Continuous

No Penalization - Discrete

W =

D’ = D ⊙ W

D ∈ R: C X B

W ∈ R: C X B

D’’ = f (D’, a, c)

12

Control Strategy

Penalization

Sigmoid Function Transformation

Re-mapping

Continuous

No Penalization - Discrete

W =

D’ = D ⊙ W

D ∈ R C X B

W ∈ R C X B

D’’ = f (D’, a, c)

V =ForwardForward-RightRight TurnLeft TurnForward-Left

∑𝑗=1

𝑛

(𝐷′ ′𝑉 )Out = --

13

PerformanceContinuous Condition Discrete Condition

Custom designed maze. All textures and 3D objects were downloaded from: http://www.turbosquid.com. Full Videos can be found on: http://www.youtube.com/watch?v=DUbqThGLykg

http://www.youtube.com/watch?v=dOcuLeYRzxE

14

ERP ComparisonCalibration Data ERP Grand Average

Cz

Calibration Mean of Accuracy: 71% according to cross-validation

15

Path Lengths and TimesFirst room

16

Data Spread at 37 seconds

17

Summary and Conclusions• The Discrete condition has a more

optimal trajectory.

• Time spent in rooms is the same for both conditions

• According to subject feedback the continuous condition was found more enjoyable

18

A more intelligent control strategy algorithm will improve the overall performance.

19

Thank you

20

21

• D = Data = 5*5 matrix stored in buffer• W = Window If discrete

If continuous• m = number of classes / speakers• n = desired dimensions = 2 for X and Y

• Sigmoid membership function(z, a, c) = g(z, a, c) =

V =

5 X 5 matrix of ones

∑𝐽=1

𝑛

∑𝑘=1

𝑚

(𝑔 (𝐷⊙𝑊 ,𝑎 ,𝑐 ) 𝑖𝑘∗𝑉 𝑘𝑗 )OUTPUT =

--

ForwardForward-RightRight TurnLeft TurnForward-Left

OUTPUT = ∑𝑗=1

𝑛

(𝑔 (𝐷⊙𝑊 ,𝑎 ,𝑐 )𝑉 )

22

Continuous Control Strategy

X

Penalization

Penalized Buffered Data

=

ELEMENTWISE MULTIPLICATION

Iterations

Classes / Speakers

23

Continuous Control Strategy (cont. 2)

Sigmoid membership function

Sigmoid Function Transformation

𝑓 (Penalized  Buffered  Data ,𝑎 ,𝑐 )=

ELEMENTWISE

Iter.

Class

24

Continuous Control Strategy (cont. 3)

Re-mapping

.

MATRIX MULTIPLICATION

=

ForwardForward-RightRight TurnLeft TurnForward-Left

Class

Iter.

𝑋 1 𝑌 1𝑋 2 𝑌 2𝑋 3 𝑌 3𝑋 4 𝑌 4𝑋 5 𝑌 5

=OUTPUT