23
Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

  • Upload
    ursala

  • View
    37

  • Download
    0

Embed Size (px)

DESCRIPTION

Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds. Introduction. Before a BCI can be used for control purposes, several training sessions are necessary Operant conditioning Feed back, real-time changes to the user Machine learning - PowerPoint PPT Presentation

Citation preview

Page 1: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Chapter 23

Combining BCI and Virtual Real-ity

: Scouting Virtual Worlds

Page 2: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Before a BCI can be used for control pur-poses, several training sessions are neces-sary

◦ Operant conditioning Feed back, real-time changes to the user

◦ Machine learning Adaptive algorithms to detect brain patterns

Introduction

Page 3: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

A technology which allows a user to interact with a computer-simulated environment.

VR (Virtual Reality)

Page 4: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

VE (Virtual Environment)◦ Allow users

to be shielded from the outside world to be able to focus on the required mental task

VR (Virtual Reality)

Page 5: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Then, Why VR for BCI?

Feedback

Visualization vs. VE feedback

Ron Angevin et al. (2004)◦ Control group (standard BCI

feedback) reacted faster◦ VR group achieved less error

Page 6: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Basic principle◦ detection & classification of motor-imagery re-

lated EEG patterns

Sensorimotor rhythms are analyzed◦ C3, Cz and Cz

Graz-BCI

Page 7: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

VE◦ To let a user become immersed in a 3D scene

Cave◦ Multiprojection stereo-based head-tracked VE sys-

tem

Graz-BCI + VE

Page 8: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

3D virtual environment1. Creation of a 3D model of the scene

3D modeling Software Packages Performer Maya

2. Generation of a VR-application that controls and animates the modeled scene

Virtual Research V8 HMD 640 X 480 pixels, refresh rate 60Hz

Vrjuggler + single back-projected wall + shutter glasses

Cave-like system

Graz-BCI + VE

Page 9: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

BCI experiments◦ Require a subject in a sitting position

No positional information had to be considered

◦ Rotational information from the tracking system was ignored Rotation should be controlled by the BCI

Graz-BCI + VE

Page 10: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Study 1Roation in a VE by Left- and Right-Hand Motor Imagery

Page 11: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Imagination of left and right hand movement

Subjects◦ 2 male(23, 26 years old), 1 female(28 years old)◦ 7 months

Feedback conditions1. A standard horizontal bar graph on a desktop monitor2. Virtual conference room presented with an HMD3. Virtual pub populated with animated avatars (includ-

ing music and chatter of the avatars)

Experiment

Page 12: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

The order of feedback conditions◦ Bar graph → HMD → Cave → HMD → bar graph

Instruction◦ To imagine left or right hand movements depending

on an acoustic cue (single or double beep)

Control◦ Either the length and the orientation of the horizontal

bar graph (in case of the standard BCI feedback)◦ Rotation angle and direction within VR

Experiment

Page 13: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

During experiments◦ Cue at second 3◦ Feedback for 4s◦ Screen update (including rotation) 24 times/s◦ One run 40 trials

Experiment

Page 14: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Results

Page 15: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

No difference between HMD and Cave

Performed well with VR than bar graph

Results

Page 16: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Study 2Moving Forward in a Virtual Street by Foot Mortor Im-agery

Page 17: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Imagination in the virtual street◦ Right hand movement: to stop◦ Foot movement to move (constant speed)◦ Walking distance is scored

CAM (Cumulative achieved mileage)

Male 23, 28 and 30 years old.

Experiment

Page 18: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Results

Page 19: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Study 3, 4Scouting through a Virtual Apartment

Asynchronous freeSpace Experi-ments

Page 20: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Virtual apartment on a single back-projected stereoscopic wall

Subject could decide freely where to go along predefined pathways (through the corridors or rooms)◦ Turn right, left, or straight)

System automatically guided the subject to the next junction◦ Small map in the bottom right corner of the display

Experiment

Page 21: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Results

Page 22: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Using VR◦ High classification accuracy (low error rate) can be achieved.

Subjects felt more natural in VE compared with BCI experiments with standard feedback

Each subject preferred the Cave experiments to the HMD and both were favored over BCI session on a desktop PC

Motivation seems to improve the BCI performance, but too much excitement might also distract the subject

Despite distraction from auditory and moving visual stimuli in VE, motor imagery and its classification in the ongoing EEG is still possible

Conclusion

Page 23: Chapter 23 Combining BCI and Virtual Reality : Scouting Virtual Worlds

Thank you !