Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

  • Published on

  • View

  • Download


Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds. Introduction. Before a BCI can be used for control purposes, several training sessions are necessary Operant conditioning Feed back, real-time changes to the user Machine learning - PowerPoint PPT Presentation


<p>Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds</p> <p>Chapter 23</p> <p>Combining BCI and Virtual Reality: Scouting Virtual WorldsBefore a BCI can be used for control purposes, several training sessions are necessary</p> <p>Operant conditioningFeed back, real-time changes to the user</p> <p>Machine learningAdaptive algorithms to detect brain patternsIntroduction2A technology which allows a user to interact with a computer-simulated environment.VR (Virtual Reality)</p> <p>VE (Virtual Environment)Allow usersto be shielded from the outside worldto be able to focus on the required mental task</p> <p>VR (Virtual Reality)Then, Why VR for BCI?</p> <p>FeedbackVisualization vs. VE feedback</p> <p>Ron Angevin et al. (2004)Control group (standard BCI feedback) reacted fasterVR group achieved less errorBasic principledetection &amp; classification of motor-imagery related EEG patterns</p> <p>Sensorimotor rhythms are analyzedC3, Cz and Cz</p> <p>Graz-BCI</p> <p>VETo let a user become immersed in a 3D sceneCaveMultiprojection stereo-based head-tracked VE systemGraz-BCI + VE3D virtual environmentCreation of a 3D model of the scene3D modeling Software Packages PerformerMaya</p> <p>Generation of a VR-application that controls and animates the modeled sceneVirtual Research V8 HMD640 X 480 pixels, refresh rate 60HzVrjuggler + single back-projected wall + shutter glassesCave-like systemGraz-BCI + VEBCI experimentsRequire a subject in a sitting positionNo positional information had to be considered</p> <p>Rotational information from the tracking system was ignoredRotation should be controlled by the BCIGraz-BCI + VEStudy 1Roation in a VE by Left- and Right-Hand Motor ImageryImagination of left and right hand movement</p> <p>Subjects2 male(23, 26 years old), 1 female(28 years old)7 months</p> <p>Feedback conditionsA standard horizontal bar graph on a desktop monitorVirtual conference room presented with an HMDVirtual pub populated with animated avatars (including music and chatter of the avatars)ExperimentThe order of feedback conditionsBar graph HMD Cave HMD bar graph</p> <p>InstructionTo imagine left or right hand movements depending on an acoustic cue (single or double beep)</p> <p>ControlEither the length and the orientation of the horizontal bar graph (in case of the standard BCI feedback)Rotation angle and direction within VRExperimentDuring experimentsCue at second 3Feedback for 4sScreen update (including rotation) 24 times/sOne run 40 trials</p> <p>Experiment</p> <p>Results</p> <p>No difference between HMD and Cave </p> <p>Performed well with VR than bar graphResultsStudy 2Moving Forward in a Virtual Street by Foot Mortor ImageryImagination in the virtual streetRight hand movement: to stopFoot movement to move (constant speed)Walking distance is scoredCAM (Cumulative achieved mileage)</p> <p>Male 23, 28 and 30 years old.</p> <p>ExperimentResults</p> <p>Study 3, 4Scouting through a Virtual Apartment</p> <p>Asynchronous freeSpace Experiments</p> <p>Virtual apartment on a single back-projected stereoscopic wall</p> <p>Subject could decide freely where to go along predefined pathways (through the corridors or rooms)Turn right, left, or straight)</p> <p>System automatically guided the subject to the next junctionSmall map in the bottom right corner of the display</p> <p>ExperimentResults</p> <p>Using VRHigh classification accuracy (low error rate) can be achieved.</p> <p>Subjects felt more natural in VE compared with BCI experiments with standard feedback</p> <p>Each subject preferred the Cave experiments to the HMD and both were favored over BCI session on a desktop PC</p> <p>Motivation seems to improve the BCI performance, but too much excitement might also distract the subject</p> <p>Despite distraction from auditory and moving visual stimuli in VE, motor imagery and its classification in the ongoing EEG is still possibleConclusion</p> <p>Thank you !</p>


View more >