Upload
tranminh
View
219
Download
1
Embed Size (px)
Citation preview
Understanding OthersFrom Dots to Robots
University of California San Diego
Department of Cognitive Science
Cognitive Neuroscience and Neuropsychology Lab
http://www.cogsci.ucsd.edu/~asaygin
.Ayse P. SAYGIN, PhD
Understanding Others
Theory3rd person (“other”) actions: Often visually perceived1st person (“self”) actions: Rarely visually perceivedYet we are able to know what we perceive (Barresi & Moore,
1996, BBS)
NeuroscienceVisual hypothesis – Based on a visual analysis of the elements
(body parts, objects, motion, etc). No sensorimotorinvolvement is required
Simulation hypothesis – Analysis-by-synthesis. We map thevisual representation onto our own sensorimotorrepresentations
Perception and Action
perception
motor planning and executiondecision-making and other
“executive functions”
Mirror NeuronsFrontal area F5 in the
macaque (Rizzolatti lab, ca1996)
Later also found in parietalcortex
Neuroimaging in humans: Themirror neuron system Inferior frontal cortex Inferior parietal cortex Superior temporal sulcus (STS)
Other findings
Mirror neurons are not specific to vision
Auditory mirror neurons in monkey
Kohler et al., 2002, Science
Theory
An individual can understand others’ actions bymapping the visual representation of the observedaction onto his/her sensorimotor representation ofthe same action, thus using his/her own embodiedexperience of the world.
“An action is understood when its observationcauses the motor system of the observer to‘resonate’ ” (Rizzolatti, Fogassi, & Gallese,2001).
Motion-defined actions
Given that point-light biological motionfigures give rise to vivid action percepts,are they also processed in action-relatedareas - e.g. premotor/IFG?
Motion-defined actions
Given that point-light biological motionfigures give rise to vivid action percepts,are they also processed in action-relatedareas - e.g. premotor/IFG?
Or are they mainly processed in motion-sensitive areas?
Functional MRINMR technology
BOLD signal: Measures thehaemodynamic response relatedto neural activity (sort of…)
Oxy/deoxy hemoglobin
Excellent spatial, poortemporal resolution
MRI
One high resolution(anatomical) image
~2sCondition 1
Condition 2
fMRI
Many low resolution(functional) images
HRF
The Scanner
Magnetic: Static Magnetic FieldCoils
Resonance: Radiofrequency Coil
Imaging: Gradient Field Coils
Shimming Coils
Data transfer and storagecomputers
Physiological monitoring, stimulusdisplay, and behavioralrecording hardware
How It Works
Magnetic: Put subject instrong magnetic field
Resonance: Transmit radiowaves, turn off transmitter,receive radio waves emittedby subject’s brain (the MRsignal)
Imaging: Modulate the strengthof the magnetic fieldslightly over space
Bo
B0 is the scanner’smain field
fMRI StudySaygin AP, Wilson SM, Hagler DJ Jr, Bates E, Sereno MI.(2004) J. Neurosci.
Dale, Fischl & Sereno, 1999; Fischl, Sereno & Dale, 1999Fischl et al, 1999; Hagler, Saygin & Sereno, 2006
Biological Motion Vs. Scrambled
Lateral Temporal CortexpSTS, V5/MT+, (EBA, LOC) / BA 37, 39, 22
Inferior Frontal CortexInf. Frontal Sulcus and Precentral Sulcus / BA 44, 45, 6
V5/MT+
STSSTS
PreC
IFSPreC
IFS
From Saygin et al, 2004 J Neurosci
Possible Cross-SpeciesDifference
Human mirror system (frontal) responds tosimplified representations such as point-light actions.
Macaque mirror neurons do not respond to pointlights, or even respond to videos…
Monkey -> Human: Abstract representation ofactions are also OK?
fMRI Results
Biological motion activates premotor/inferior frontalcortex.
Indeed, IF and premotor areas are just as selectiveas pSTS.
Cross-Species Differences
Human mirror system (frontal) responds tosimplified representations such as point-light actions.
Macaque mirror neurons do not respond topoint lights, or even respond to videos…
Monkey -> Human: Abstract representation ofactions are also OK?
Language and Mirror Neurons
Inferior frontal gyrus - Broca’s area
Although language regions tend to be moreinferior and anterior, there is someoverlap
Mirror neurons - origins of language?
Motor theory of speechperception
Liberman et al. 1967
Auditory signals too variable
Motor representations are used inperception of speech
Nice idea but there was no evidence– Categorical perception: /da/ or /ta/ notbetween
– But also chinchillas, birds, macaques (e.g.,Kuhl & Miller, 1975)
Listening to and producing speech
UCLA/UCSD Study
Listen to monosyllables/pa/ /gi/Produce monosyllables/pa/ /gi/
Overlap found.Note: This is a moresuperior area thanBroca’s area.
Wilson, Saygin, Sereno &Iacoboni et al., 2004, NatureNeurosci.
Language Semantics
Hauk, et al., 2004, Neuron
Foot action wordsHand action wordsMouth action wordsSomatotopy
Empathy for pain
Singer et al., 2004, Science
Not in primary sensory cortex butin affective pain processingareas.
Neuropsychological Study
47 LHD patients, 13 RHD patients, 18 age-matched controls
– Normal or corrected to normal vision
– Patients >1 year post onset of stroke
– Unselected lesion site - but single infarct,unilateral lesion
– No other neurological conditions
Saygin AP (2007) Brain
Detect point-light biological motion in noise, 2AFC
Behavioral measure: Number of noise dots at 82%accuracy
Adaptive estimation with QUEST (Watson & Pelli, 1983)
Stimuli and Task
Behavioural Results
Both LHD and RHD patients significantly impaired.
Controls > LHD p<0.0001Controls > RHD p<0.01RHD, LHD n.s. p=0.7
From Saygin, 2007 Brain
VLSM: Voxel-Based LesionSymptom Mapping
Bates E, Wilson SM, Saygin AP, Dick F, Sereno MI, Knight R, Dronkers NF(2003). Nature Neurosci.
cf. Borovsky, Saygin, Bates & Dronkers, 2007, NeuropscyhologiaDronkers, Wilkins, Van Valin, Redfern & Jaeger, 2005, Cognition
Saygin, Wilson, Dronkers & Bates, 2004, NeuropsychologiaWilson & Saygin, 2003, J. Cogn. Neurosci.
+ Behaviour
A Given Voxel
Each patient’s lesioneither includes orexcludes voxel
Patient mPatient n
.
.
.
Patient 3Patient 4
Patient 2Patient 1
VoxelLESIONED
VoxelINTACT
A Given Voxel
Each patient’s lesioneither includes orexcludes voxel
Each patient hasbehavioural measure(s)
11.917.1
.
.
.
19.421.6
14.812.9
VoxelLESIONED
VoxelINTACT
A Given Voxel
Each patient’s lesioneither includes orexcludes voxel
Each patient hasbehavioural measure(s)
Compare Intact andLesioned and get astatistic (eg, t, p)
11.917.1
.
.
.
19.421.6
14.812.9
VoxelLESIONED
VoxelINTACT
Biological Motion: Lesion Map
Map of the t-statistic at each voxel
Large region in temporal and parietal cortex (BA 21, 22, 37, 39, 40)
Smaller area in inferior frontal gyrus (BA 44, 45, 6)
From Saygin, 2007 Brain
ANCOVA map: Frontal and posterior lesion sites are independentfrom each other
Lesion Independence
From Saygin, 2007 Brain
Motion cues sufficient to drive premotor areas
These regions are both involved in and necessaryfor biological motion perception
Humanoid Robots
Robotic agents: Can perform recognisable actionsbut do not have true biological motion
Appearance can be more or less human-like
An opportunity to test selectivity of the actionperception system for human movement and/orhuman appearance
Also relevant to robotics
Interactive robots: Retail,healthcare, education…
But what kind of robots shouldbe made?
Wall-E, 2008, Pixar
Humanoid Robots
Conflicting results re: robot actorse.g., Gazzola et al 2007; Kilner et al 2003; Oberman et al 2007;
Press et al 2005, 2007, Tai et al 2004
We used state of the art robotsPlus manipulated the appearance of the robots
The Uncanny Valley
Mori, 1970
Humanlikeness is notalways “good”
Framework - not based onexperimental data
Stimuli
Actions by:
Human (master of Repliee-Q2)
Android (Repliee-Q2)
Robot (made from Android)
Grasping, wiping, picking up, nodding, waving, yawning…
All video-ed using same camera, objects, background at IntelligentRobotics Lab, Osaka, Japan
Appearance and Motion
Nonhuman appearance
Nonhuman motion
Human appearance
Nonhuman motion
Human appearance
Human motion
Appearance and Motion
Nonhuman appearance
Nonhuman motion
Human appearance
Nonhuman motion
Human appearance
Human motion
Appearance and Motion
Nonhuman appearance
Nonhuman motion
MATCH
between appearanceand motion
Human appearance
Nonhuman motion
MISMATCH
between appearanceand motion
Human appearance
Human motion
MATCH
between appearanceand motion
Repetition SuppressionNeuronal adaptation - less response to repeated feature
Reduced fMRI response in regions coding repeated property
Repetition SuppressionNeuronal adaptation - less response to repeated feature
Reduced fMRI response in regions coding repeated property
Here same or different action (cf. Dinstein et al, 2007; Hamilton &Grafton, 2006, 2008; Lestou et al, 2008)
Same
2000 msec 2000 msec350 msec
Different
2000 msec2000 msec 350 msec
Same < Different = Repetition Suppression
Agent (3) x Repetition (2)
Human RepAndroid RepRobot RepREPEATED
Human NewAndroid NewRobot NewNEW
Interaction?
Parietal CortexINTERACTION (A>H A>R) in Bilateral aIPS
aIPS (BA 7, 40)
2
aIPS (BA 7, 40)
4
pIPS (BA 19)
3
pIPS (BA 19)
1
2
1
3
4
Appearance and Motion
Temporal cortex by appearance (likely EBA)
Parietal cortex by mismatch in appearance and motion
No evidence for selectivity for human appearance orbiological movement in frontal and parietal
It’s not biological movement or appearance per se…
Interaction/integration of appearance and movement
Uncanny valley: Processing conflict in the actionperception network
Predictive Coding
Hierarchical system. Braintries to minimize predictionerror
Human and Robot-Match between appearance andmovement-Correct model can be selectedfrom appearance
Android-Appears human-Attempt to use human model-Kinematics not as expected-Prediction error
Future Work
This is just a start…
Stimuli with different appearance andmovement (e.g., animation)
Methods to study dynamics of the network(EEG, MEG, TMS)
Turing Test (Turing, 1950, Saygin et al, 2000) -Can a computer pass for a human?
Total Turing Test (Harnad, 1989) - Must be arobot
A Neural Turing Test?Is the brain response to robot indistinguishable
from that for a real human?
Future Work
Elizabeth BATES, San Diego, CA
Thierry CHAMINADE, Marseille, France
Jon DRIVER, London, UK
Nina DRONKERS, Martinez/Davis, CA
Chris FRITH, London, UK
Donald HAGLER, San Diego, CA
Hiroshi ISHIGURO, Osaka, Japan
James KILNER, London, UK
Marty SERENO, London, UK & San Diego, CA