PowerPoint Presentation
This is Your Brain on Virtual RealityLynda Joy Gerry21st of June
2016
1
Agenda:1. Spatial Processing and the Brain2. Agency and the
Brain3. Avatar Effects4. First-Person POV Embodied Experiences5. Ex
Nihilo: Stimulating Creativity and Empathy6. Future Directions:
Social Interactions in VR2
Spatial Processing and Wayfinding in Virtual Environments3
Boldly Going Where No Rat Has Gone Before (Mayank Mehta, 2008,
UCLA)4
VR For Rats5
Virtual Reality Affects Firing Patterns of Brains GPS
CellsBackground: Selective spatial firing of neurons in the
hippocampus
Research Question: Do virtual environments activate a mental map
in the same way as the real world?
Findings: VR world stimulates response more like a pedometer the
rats brains were tracking how many steps they were taking, rather
than incorporating multisensory cues to integrate into a mental
map6
Measures of Presence and Neuronal Activation7
Presence: Definition Presence is the subjective experience of
being transiently unaware of actions and cognitions linked to the
real environment while concurrently perceiving oneself as situated
in and perceiving action possibilities associated with the virtual
environment (VE) (Wirth et al., 2007)8
High Vs. Low Presence Ratings9
Baumgartner et al., 2008
Presence and The BrainThere are individual differences in how
easily drawn into the virtual environment a person feels
(Ijsselsteijn and Riva, 2003; Wirth et al., 2007)
Children and adults differ in the quality and speed with which
presence is evoked by VEs (Schaik et al., 2004)
Individual differences in susceptibility towards presence might
be modulated by the Prefrontal cortex, which is linked to executive
functions and inhibitory control over emotions and behavior (Aron
et al., 2004; Garavan et al., 1999; Koechlin et al., 2003;
Ridderinkhof et al., 2004)10
The Amphibian SCUBA Diving Simulator (Dhruv Jain, MIT Media
Lab)11
11
Body Ownership, Agency, and Avatar Effects12
Is that hand my hand?13
Body Ownership Illusions14
Body Ownership Illusions and AgencyBody Ownership Illusions:
Paired visual-tactile stimulation (Rubber Hand Illusion) or paired
visual-motor stimulation Tracked and rendered full-body motion in
fully-immersive virtual environments creates sense of AgencyBottom
Line: Under specific multisensory conditions, we can experience
artificial body parts, fake bodies, or virtual bodies as our own
body parts or whole body.15
16
Counseling Oneself: Embodying Sigmund Freud Avatar17
Osimo, Pizarro, Spanlang, & Slater (2015)
Avatar Effects ExamplesEmbodying a body of a different race
reduces implicit racial biases (Groom, Bailenson, and Nass,
2009)Embodying a tall avatar increases self-confidence in
negotiation tasks (Yee and Bailenson, 2007)Embodying an attractive
avatar increases self-disclosure(Yee and Bailenson, 2007)Embodying
a stressful posture in a virtual body can increase stress even
though physical body is not in that posture/position(Bergstrm,
Kilteni, and Slater, 2013)Embodying a child avatar body causes
subjects to overestimate object sizes(Banakou et al., 2013)18
Ahn, Bailenson, & Park (2014)19
Ahn, Bailenson, & Park (2014) Pro-Environmental
Self-EfficacyEnvironmental locus of control is the perception that
ones behaviors directly impacts the wellbeing of the
environment.
This is one of the most powerful drivers for environmental
behaviors.20
Core Problem: Knowledge-to-Action Gapthe time-delayed, abstract,
and often statistical nature of the risks of global warming does
not evoke strong visceral reactions. Weber (2006)21
Promoting Helping Behavior in VR22
Ahn et al., 2015
Benefits of AvatarsControlled environment + StimuliAbility to
have live interaction with your own and another persons avatarWhy
use avatars when you can use real people?23
Agency and the Brain24
Medial Prefrontal Cortex and Agency (David et al., 2006)Distinct
neural substrates underlying representations of the self and
othersThe medial prefrontal cortex is crucial for a neural bases of
the self and agencyPerspective taking and agency represent
independent constituents of self-consciousness25
FPP Hand Movement in VEs (Adamovich et al., 2009)26
Time-variant increase in activation in the left insular cortex
during observation of avatar hand
Imitation of avatar showed activation in angular gyrus and
extrastriate body area
Virtual hand avatar serves as disembodied training tools in
observation condition and as embodied extensions of subjects own
body in the imitation condition
27FPP Hand Movement in VEs (Adamovich et al., 2009)
Background: First-Person Point-of-View Embodied Experiences in
VR28
Think about it: there is no experience you have had that you are
not the absolute center of. The world as you experience it is there
in front of YOU or behind YOU, to the left or right of YOU, on YOUR
TV or YOUR monitor. And so on. Other peoples thoughts and feelings
have to be communicated to you somehow, but your own are so
immediate, urgent, real.- David Foster Wallace, This is Water (2005
Kenyon Commencement Speech)29
Machine to Be Another: Live Stream Perspective Swapping30
The Machine to Be Another (MIT Media Lab)31
THE MACHINE TO BE ANOTHER:An Empathy Machine?32Empathy is
created the moment we try to put ourselves in another persons
shoes. Aspen Baker, TED Radio Hour
The machine to be another is designed to stimulate empathy
through embodied interaction between individuals. (Bertrand et al.,
2014)
To what extent can we experience and understand what it is like
to be someone else?
Simo Ekholm: Meeting Myself from Another Persons
Perspective33
Makropol: Art House VR using POV storytelling34
Skammakrogen: How it Was Made35
How to Combine These Experiences?Be Another Labs use live stream
into the DK2Benefit: Real-Time InteractionCon: Low visual
resolution + not stereoOriginal plan: Live Stream in Stereo Low
visual resolution + DelayUsed Stereo and Pre-Recorded Footage with
Binaural audio36
Painter Project: Stereo Footage with Binaural Audio37
Ex Nihilo: Exploring the Stimulation of Creativity and Empathy
through Embodied Experiences in Virtual Reality38
Gallagher et al. (2015) Neurophenomenology of Awe and
Wonder39
Video of Experimental Task40
Video of Experimental Task41
42
43
44
45
Introducing Ex NihiloIn The Republic, Plato asks, "Will we say,
of a painter, that he makes something?" and answers, "Certainly
not, he merely imitates."46
Imitated Paintings by Subjects47
Subject QuoteIt almost feels like collectively discovering
something that was in her head all along. The experience allowed
this completely different way of connecting to someone that doesnt
rely on facial expressions or anything.48
Measuring Social Interaction in VEs49
Measuring Social Interaction in VR How do you measure two
peoples ability to communicate effectively?What does communicating
effectively mean?Suzanne Dikker shared semantic understanding and
neural synchrony50
Perceptual Crossing (Auvary et al., 2009)51
Perceptual Crossing (Auvary et al.,2009)Task: Participants
recognize one another while avoiding shadow and static
objectsMutual tactile interaction (trackball mouse and haptic
feedback)Static object = too stable and predictable to be
humanShadow = too unstable since it moves but does not respondOther
subjects avatar can react with communicative intent52
Froese and Di Paolo (2011)Hypothesized that the lack of personal
recognition of the other in the perceptual crossing paradigm was
due to lack of a genuinely social experimental taskThe mark of the
social is the co-regulation of mutual interaction53
Primary Intersubjectivity in Infants2-month old infants get
distressed when shown a pre-recorded video of an interaction with
their mother, as opposed to a live streamed video. Simple
recognition of a pattern of movements is not sufficient, and
instead social action perception requires an understanding of how
other's movements are related to our own.Infants display movement
synchrony with adult speech (Condon & Sander, 1974)Still face
studies (Toda & Fogel. 1993; Tronick, Als, Adamson., Wise.
& Brazelton, 1979)54
Thank [email protected]
