Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
A computational Approach to Behavior
Pietro Perona California Institute of Technology
26 March 2017 AAAI Symposium on
``Computational Principles of Natural and Artificial Intelligence’’ Stanford University
California gull vs. alkali fly (Larus californicus vs Ephydra hians)
Mono Lake, CA 37°58’50.6”N, 119°06'33.4"W
environment
development, regulation
decision, control
perception learning
natl. selection gene expression
genes
behavior
brain+ body
environment
construction
decision, control
perception learning
analysis of performance
design
behavior
HW+ SW
What is `behavior’?
6
Zidane in a dish
Courtship in Drosophila
Marla B. SokolowskiNature Reviews Genetics 2, 879-892
(2001)
Phenomenology: many scales
1 frame
1 second
1 minute
1 hour
1 day
10 pix 100 pix 1000 pix
mov
emes
actio
nsac
tiviti
es
step reach glance
walk
hike
have drink
dine flirt
court
write `A’open door
Parametrization
[Eyjolfsdottir, 2014]
Ethograms
[Dankert et al., Nature Methods, April 2009]
by hand it would have taken (20+24+10)*1/3 * 3 * 7 = 400h
Levels of enlightenment
• Description
• Phenomena
• Mechanisms
FINDING AGGRESSION NEURONS IN THE FLY
X
warmth-sensitiveion channel
hyperaggressive?
LIBRARIES OF Drosophila STRAINS WITH DIFFERENT GENETICALLY LABELED POPULATIONS
OF NEURONS
⨁⨁
⨁⨁
⨁⨁
27oC
X=ion channel
gene
Analysis by synthesis
[Braitenberg, 1984]
images, trajectories
pose, movemes, actions, activities, objects, scenes
plans, goals, behavior, relationships ...
group-level goals and plans
individual goals and plans
motor programs
sensor-based control
IMAGING,TRACKING
RECOGNITION
COGNITION
SOCIAL NETWORK
PREFRONTAL CORTEX
MOTOR CORTEX, BASAL GANGLIA etc.
SPINAL CORD
SENSO
RYR
EPRESEN
TATIO
NA
LM
OTO
RPL
AN
NIN
GAction Perception
World
INDIVIDUAL
interaction, cooperation, competition
SOCIAL INTERACTION
Levels of analysis• Description
• Phenomena
• Grammar
• Purpose
• Mechanisms
• Performance
• Ontogeny
• Phylogeny
From phenomena to mechanisms:
a simple example
[Branson et al. Nature Methods, Jun. ’09]
[Branson et al. Nature Methods, Jun. ’09]
‘T-Stops’
‘X-Stops’
Wang et al. 2003.
An engineers’ solution
Try implementing it with 105 neurons
http://www.news.wisc.edu/newsphotos/images/fruit_fly_research03_6801.JPG
Regressive motion
↵�
�
[Zabala ’12]
Experiment
[Chalupka ’15]
Model
Regressive Looming
Levels of enlightenment• Description - fly trajectories
• Phenomena - stops, chases, … (actions)
• Grammar - T-stops,X-stops, (interactions)
• Purpose - Avoid predation and collisions
• Mechanisms - regressive motion(implem.?)
• Performance - Mobility,safety
• Ontogeny - Learning??
• Phylogeny - ??
Mechanism inference
[Eyjolfsdottir 2016,2017]
Fly-centric features
v: sensory inputx: motor control
len
w-len
w-ang
fwd
sideyaw
flieswalls
reproduce
eat fight
escapeh
v
x
h1i^
h2i+1^h2i
^
h1i+1^
xi+1^
h2i-1^
h1i-1^
xi-1^ xi
^
generative
Model architecture
Simulation:- prediction as input- compute vi on the fly
… …
h1i+1
h2i+1
vi-1 xi-1
vi xi
vi+1 xi+1
h1i
h2i
h1i-1
h2i-1
yi
yi^discriminative
h2i+1^
h1i+1^
xi+1^
vi xi
h1i
h2i
yi
yi^
yi
yi^
[Eyjolfsdottir 2016,2017]
Sanity check: learning on a synthetic fly
Control laws:1) walk forward with small randomness
2) rotate when close to boundary(away from higher visibility)
3) walk towards obstacle, when visible
4) extend wing when half way towards it
5) rotate when close to obstacle(alternate between left and right)
6) repeat from 1)
obstacle visible
Can the model learn generative control laws?
SynthFly Simulation
0 100 200 300 400 500 600 700 800 900 1000-2
-1.5
-1
-0.5
0
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900 10000
200
400
600
800
0 100 200 300 400 500 600 700 800 900 1000-1
-0.5
0
0.5
1175-1
-0.5
0
0.5
1
-1
-0.5
0
0.5
1
-2
-1
0
1
2
0 100 200 300 400 500 600 700 800 900 10000
200
400
600
800
0 100 200 300 400 500 600 700 800 900 1000-2
-1.5
-1
-0.5
0
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900 10000
200
400
600
800
0 100 200 300 400 500 600 700 800 900 1000-1
-0.5
0
0.5
1175
0 100 200 300 400 500 600 700 800 900 1000-2
-1.5
-1
-0.5
0
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900 10000
200
400
600
800
0 100 200 300 400 500 600 700 800 900 1000-1
-0.5
0
0.5
1175
0 100 200 300 400 500 600 700 800 900 1000-1.5
-1
-0.5
0
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900 10000
100
200
300
400
500
600
700
0 100 200 300 400 500 600 700 800 900 1000-1
-0.5
0
0.5
191
r r rl l l l l l l
0 100 200 300 400 500 600 700 800 900 1000-1.5
-1
-0.5
0
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900 10000
100
200
300
400
500
600
700
0 100 200 300 400 500 600 700 800 900 1000-1
-0.5
0
0.5
191
r r rl l l l l l l
0 100 200 300 400 500 600 700 800 900 1000-1.5
-1
-0.5
0
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900 10000
100
200
300
400
500
600
700
0 100 200 300 400 500 600 700 800 900 1000-1
-0.5
0
0.5
1175
0 100 200 300 400 500 600 700 800 900 1000-1.5
-1
-0.5
0
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900 10000
100
200
300
400
500
600
700
0 100 200 300 400 500 600 700 800 900 1000-1
-0.5
0
0.5
191
r r rl l l l l l l
0r r rl l l l l l l
100
0
0
0
1
-1
-1
1
distance to object
r l r l r l r l r l r l r l r l r
left/right wing angle
hidden unit 91
hidden unit 175
synthetic fly simulation
time timeturn
Interesting units
Classification
Discovery
Discovery: FlyBowl
tSNE dimensionality reductionfemale male right wing extension left wing extension
hli: hidden states of unsupervised model
Discovery: IAM-OnDB
stroke length writer identity
Prediction
Simulation
Simulation: FlyBowl
fly visionchamber vision
simulated real
Simulation: FlyBowl
Simulation - handwriting
“e”
“s”
“r”
generated text character injectionsoriginal text “activate”
Summary• Brains, behavior and intelligence
• Description
• Phenomena
• Mechanisms
• Relationship with AI?
Collaborators
Kristin Branson
HeikoDankert
David Anderson Michael Dickinson
+Carlos Gonzalez
Shay OhayonRoian Egnor
Michael MaireJohn BenderTim LebestkyAlice Robie
Erik Hoopfer
Xavi Burgos
PiotrDollar
EyrúnEyjólfsdóttir
CristinaSegalin
Grant van Horn