EMORUSH TUTORIAL

Guide to using and understanding the EmoRush Application.

Developed by Daniel Ehrens in 2012 daniel_egp@hotmail.com

Tutorial Sections

This tutorial will be divided into two main sections.

• Understanding the application.

• How to use the application.

Understanding the application

The main purpose of the application is to provide the patient with a Brain-Computer Interface (BCI) that is designed to engage brain circuitry shared by the patient’s own seizure network to interfere with the progression of the seizure.

Understanding: EmoCube

The main purpose of the application is met by controlling the movement of a 3D cube (EmoCube) that will be displayed in a monitor in front of the patient. This cube will be cotrolled through “Cognitiv States”.

How does it work?

Basically the program works based on EEG mapping. EEG maps are built by the headset and the Emotiv software.

When you train a State of mind (e.g. Lift, Push, etc.) the headset is “listening”, and records that EEG map. When the training is finished and your EEG map resembles that of the training then the cube moves in the direction that was assigned.

*Actions NEED to be trained before using them.

Extra Features Besides the Cube movement control. The application also contains some extra features

that could be helpful mainly but not exclusively for the patient and its doctor.

(maybe not as wonderful, but maybe someday ehh)

Extra things

So after the program was finished some things were added and I was not going to change the whole tutorial, so they are explained at the end.

These extra things include: -Audio Feedback Menu -Independent Sz button in the Application Menu

Understanding: Profile Management

Since each person is unique, their EEG maps are unique, therefore every user needs a different profile.

*The profile contains the trained data

for moving the cube. The application lets you: • Add a new profile. • Load an old profile. • Save the current profile.

Understanding: Eye Status – Blink State

The application provides the user and anyone around with a visual feedback of the user’s eye state.

This has no other useful purpose in the application.

Understanding: Audible Alarm

There is a built-in constant “beeping” alarm in the program.

If turned ON it will start beeping every second. The user is able to control the beeping frequency of the alarm by moving the cube up or down.

Understanding: EEG recording and display

The application can provide a “real-time” display of the EEG. It displays 14 channels.

The headset has a fixed sample frequency of 128 Hz (samples per second).

Features in the EEG display

The EEG display includes some features of its own. • Timer, tells for how long the EEG has been

recorded. • Automatically saves the EEG recording in a *.csv

format in a previously selected folder by the user. • The option to see the EEG in three different

montages: Circumferential (default), Single Banana and Transverse.

Understanding: Seizure Onset Detection Algorithm (SODA)

The application also includes a SODA. This has to be activated. (The explanation of how it works is in the EmoRush Code

Tutorial) • Thresholding can only be modified from the code. *The SODA, if activated can produce false positives with

muscle or movement artifact. You know it went off if you hear three consecutive beeps.

Understanding: EEG event review If the SODA detects a possible

seizure. Then it will create two files:

An image in a *.bmp file and the EEG recording (*.csv ) of the EEG during the 40 seconds previous to the Seizure Onset (SO) and the 40 seconds following the SO.

The application lets you review the list of EEG events at any time, and load any of them in the display for analysis.

*** SODA has to

Understanding: the Audio Feedback

This was implemented for patients that can only experience the feedback through Audio. In other words the Cube is not an option for these patients.

This audio feedback is linked to the PULL cognitive state.

How to use the application

Now that you are familiarized with the features and functions of the application you will learn to how use them.

You will also learn about the limitations of the software and the possible errors that could happen.

First things first… You need to locate the folder that includes the *.exe file for the

application (EmoRush_WFA.exe or something similar). Also locate the EmoCube.exe file (Should be inside the EmoCube folder).

*The EmoRush_WFA.exe app file MUST be in the same folder that includes the following files:

• edk_utils.dll • edk.dll • DotNetEmotivSDK.dll Otherwise the application will not launch and messagebox will pop-up

saying there was an error. If you are going to use an old profile then you should also locate the

folder where the profile is. Profiles have an *.emu extension on the file.

During the study you will need to open both the EmoRush and the RushCube applications.

The EmoRush application

Components of the EmoRush app The first thing you

need to know about the EmoRush application is the Toolstriplabel.

Whenever an event occurs this is the way for the application to let you know what is happening. For example if a profile was successfully loaded or if the application couldn’t save a profile.

Here are some examples of possible messages

Buttons in the EmoRush app

No matter what, when starting the application Step 1 always is clicking on the “Connect to Engine” Button.

This button is in charge of connecting with the Emotiv libraries. If you are not connected you can’t do anything.

***If there is a problem with this, and you are not able to connect contact the engineer in charge of the programing and tell him to check the “Connect button” section of the code it is probably configured to connect to the Control Panel or the EmoComposer.

Toolstrip should say you are connected successfully.

Button: Toggle EEG Channels This button will change the montage on the display. You

can configure the montage during the EEG acquisition or before.

*The change in montage will not modify the EEG *.csv file.

Button: Cube Switch ON/OFF This button enables the communication between the EmoRush

application and the EmoCube application. This is not enough to move the Cube, but more on that later. *Toolstrip should say cube enabled or cube disabled. As the text

beside the button

Button: Seizure Training

This Button starts the seizure training. It should be clicked on when the patient is about o have a seizure or feels an aura. This will start a 16 second proccess in which the headset will train the cube to the EEG map of the seizure.

How to use the Beeping Alarm To enable or disable the alarm, click

on the ALARM ON/OFF button. • To decrease frequency: Cube must

be in Neutral and then Drop the cube with a “Cognitiv power” > 70.

• To increase frequency: Cube must be in Neutral and then Lift the cube with a “Cognitiv power” > 70.

*There must be a Neutral state between each attempt to increase or decrease the beeping frequency.

Beeping frequency will be displayed on the Toolstrip.

*Maximum beeping frequency is 1 Hz

Button: Bandwidth for SODA

Within the SODA there is a digital filtering process, with this button you are able to toggle that bandwidth.

Button: EEG acquisition ON/OFF This button enables

the acquisition and display of the EEG.

At the same time it is displaying the EEG it will also create a *.csv file with the recording of the EEG.

When you click on the button it will ask you where do you want to save the EEG files.

*If you click “Cancel” on the Browse Folder Dialog it will do the same thing as if you clicked on “Ok”, select that folder and start the acquisition.

EEG Display

If you have profile loaded it will use that name to create the EEG file (i.e. John_EEG_1.csv). If there is not a profile loaded it will save the file as “Tempuser_EEG_1.csv”.

As soon as you select the folder it should start displaying the EEG.

If it is not displaying the EEG probably the headset is not turned on.

If the SODA is activated it will save the Event files in the same folder as the EEG is being saved.

If you stop the EEG acquisition and then start it again it will create another EEG file but will increase the number so it will be named as “John_EEG_2.csv”

*This is an example of a folder with files. It has a profile file as Zazu.emu. He had three EEG recordings. During the 1st EEG he had 2 events, in the 2nd EEG he had 1 event and in the 3rd EEG he had 3 events. The *.bmp files are the snapshot of the events, this are for the listview.

EEG Display

The graphing of the EEG display has been removed because of hardware limitations on the notebook that was used for the study.

However to turn it on… you just have to go to the Application menu EEG display and turn it on.

Application Menu

*In the snapshots of the following slides a “Type Here” option in the menu appears, that will not appear in the program that is only for the developer.

ApplicationLoad The Load option in the menu will display to options to Load a Profile or EEG data. When you click on Profile a File Browser Dialog will appear for you to select the Profile file. The program will check if the file is a Profile file based on the extension of the file. *Profile files have an *.emu extension. *If you are not connected to the EmoEngine it will not be able to load the profile and a message of this will appear in the Tool Strip.

1

2

LoadEEG data When clickin upon EEG data a Folder Browser Dialog will appear. You should select the Folder where you have all the EEG event files, this are the files that will be available for display. After selecting the Folder a List of the EEG Events in that folde will appear.

1

2 3

ApplicationAdd

The Add Profile will pop up a Save File Dialog. Here you just need to type in the name for the new Profile and select a Folder. When you click OK it will automatically generate a Folder with the same name, in the Directory that you chose and within that Folder the *.emu file with that name.

Bug on the Add profile

If you have a Profile loaded when you add a profile. The new profile will be a duplicate of the previous loaded profile.

If you don’t save the new profile and load another profile and then load the new profile the new profile will duplicate the previous profile.

This will not end until you save the new profile.

ApplicationSave

Whenever you Load a profile and train some actions on the Profile, Before you close the application you need to Save the Profile otherwise the training will be lost. For that you just need to go to Application Save Current Profile The ToolStrip will let you know that the Profile was successfully saved.

ApplicationAbout If you go to Application About Software A message box will appear.. This box contains my contact information, In case you have questions about the program, feel free to shoot me an email and ask me.

ApplicationSODA To activate or deactivate the SODA you need to click on: Application SODA ON/OFF SODA The ToolStrip will let you know if it was activated or

deactivated. Also there is a label next to the SODA Bandwidth that at all times says “On” or “Off”, referring to SODA.

ApplicationShow independent Sz button

What this does is that a pop-up window comes up, this is a floating “Seizure Training” button. Works the same way as the button on the form just gives you the option to display it somewhere else and click it, instead of doing it on the form.

To set up AirDisplay and use the independent button

1. First log-in into the RushPub internet and “Accept” the terms and conditions on the phone and the laptop.

2. Open AirDisplay on the laptop and on the phone.

3. Check the IP address on the phone which is within the app.

4. Put the IP address in the AirDisplay as “other” device.

5. If it is not connecting restart the AirDisplay app in the laptop and start do step 4 again.

ApplicationEEG Display

By default the application won’t display the EEG. Instead it will display a big timer.

To activate the EEG display go to the Application menu

Training Menu

See trained states To see which states have been trained for a Profile. First you need to Load the profile and then go to Training Cognitive states See trained states In the ToolStrip the trained states and their skill rate will be displayed. The skill rate is in a 100% scale.

1

2

Train a Cognitive state If you want to train a Cognitive state then you go to: Training Train Cognitive State (Select) This will start the training. The training lasts 8 seconds. During this time the user should start picturing the action of the block doing that specific movement. The headset will train the cube to that movement with the current EEG map of the user. *Before training an action

you need to load a profile.

*If you are click on training An action and the program Freezes and the ToolStrip says It is training some action. The headset might be off.

Erase training If you want to erase a training action or all training actions you go to Training Erase training (Select action) This should be done if you think that the Cube is no longer responding well. The ToolStrip will let you know when the training has been erased.

RushCube App

This application is really simple.

You just have to double-click on it to open it.

When the Cube is working When the Cube is working besides the Cube

moving on the EmoRush app you can see what state is being active and the power of the state.

Audio Feedback Menu So this menu is linked to the

COG_PULL action. When the COG_PULL is detected, labels under the EEG display will appear saying which is the current feedback and the threshold.

Basically this plays a beep that increases in frequency according to the cognitive power. It goes from 150-600Hz, in steps of 50 Hz. If it goes over the threshold then a different sound is played. Some of the options are “I understand”, Applause and a Gong.

We are almost done!!

So by now you should be familiar with the following:

• Overall objective of the headset and the BCI. • The EmoRush and RushCube apps. • The buttons and menus in the EmoRush app. In the following slides I will provide with cheat

slides just so you know what steps to take when doing something.

Activating the Cube

To make the Cube work you have to do the following.

1. Click on “Connect to EmoEngine” button (1st step always)

2. Load/Add a profile (ApplicationLoad/AddProfile) *remember if you add a profile to save the profile.

3. Open RushCube.exe 4. If you want to do some training you can do it at

any time after the profile is loaded.

Acquiring EEG

1. Connect to EmoEngine button 2. Load/Add Profile 3. Click on EEG acquisition on/off 4. Select a folder to save the EEG files 5. Activate SODA if needed, if it is, check the

Bandwidth for the SODA. *** if you want to display the EEG then this

needs to be activated.(Application Menu)