Modelling the Division of Labor:
A Spiking Neuron Net Approach
Michele Sebag
TAOJoint work with Sylvain Chevallier, Helene Paugam-Moisy
SocPAR 2010
Framework: Swarm Robotics
Swarm-bot (2001-2005) Swarm Foraging, UWE
Symbrion IP, 2008-2013; http://symbrion.org/
Swarm Robotics: Why and What
WHAT
I Simple agentssimple micro-motives for macro-behaviors
I No pacemakersdecentralized, distributed, randomized systems
I More is different
An alternative to complex robots
I Inexpensive → Many → Reliable
I The “invisible hand“(Hayek’s inheritage ?)
Swarms: HOW
PrinciplesLocal information I` → estimates global quantities I
Local information → individual behaviour b(I`)Aggregate b(I`) = Behaviour[I ]
Examples
I Sounds & clusters of birds and frogs; Melhuish 99
I Bees & air-conditioning of the hive Auman 08
From observing to designing emergenceMain Issues
I Communication feasibility, cost
I Convergence individual and collective safety
I Reality Gap in simulation vs in-situ
I Bootstrapping how to prime the pump
Swarms: HOW
PrinciplesLocal information I` → estimates global quantities I
Local information → individual behaviour b(I`)Aggregate b(I`) = Behaviour[I ]
Examples
I Sounds & clusters of birds and frogs; Melhuish 99
I Bees & air-conditioning of the hive Auman 08
From observing to designing emergenceMain Issues
I Communication feasibility, cost
I Convergence individual and collective safety
I Reality Gap in simulation vs in-situ
I Bootstrapping how to prime the pump
This talk focuses on
Division of labor
Synchronization
How do social agents proceed to synchronize their activities?
Overview
I Swarm Robotics
I Biological / Artificial modelsI SpikeAnts
I Spiking NeuronsI Network Architecture
I Analysis
I Discussion and Perspectives
Biological/Artificial models
BatteryMotors
Software
The hardware perspective
Division of laborthe social stomach ?
Biological/Artificial models
BatteryMotors
Software
The hardware perspectiveDivision of labor
the social stomach ?
Biological/Artificial models, 2
J. Halloy et al., 2010
Social stomach: Macro-modelling
X Foraging robots β rate of energy stocking
S Stocking robots µ rate of energy consumption
Y Other robots (θ + Xt) rate of recruitment∂X∂t = (θ + Xt)(N − X − S)− βX∂S∂t = βX − µS
Y: empty robots
X: foraging robotsrate of recruitment
rate of stocking
S: stocking robots
rate of consumption
Division of labor, 2
Winfield & Liu 08Finding food/resting
I Finding food delivers energy
I Searching costs energy
I bumping into other robots costs energy
Goal
I Allocate time between search and resthttp://www.brl.uwe.ac.uk/projects/swarm/index.html
Adaptive Foraging in Swarm Robotic Systems, 2
Probabilistic Finite State Machine
Design
I Find transition probabilities
I Rest and Search thresholdsI Input:
I internal cues (food retrieved)I environment cues (bumping into other
robots)I social cues (success/failure of relatives)
Controller design in Swarm Robotics
Constraints design of emergence
I Spatially distributed
I Decentralized (no pacemaker)
I Asynchronous
Available information
I Cues from relatives
I Internal time (hunger-like)
Can it be avoided ?
I Random generator probabilistic model
I Sophisticated skills counting ability
Overview
I Swarm Robotics
I Biological / Artificial modelsI SpikeAnts
I Spiking NeuronsI Network Architecture
I Analysis
I Discussion and Perspectives
Spiking neurons vs std neurons
Standard neurons
I Directed graph G = (E ,V) and weights W
I An activation function: (linear, sigmoid, RBF)
ei (t + 1) =← f (∑j
wijej(t))
Spiking neurons Hodgkin Huxley 52
I Internal state (membrane potential)
I Activation function ≡ differential equation
∂e(t)∂t = f (e(t),Excitations, Inhibitions) if e(t) < ϑ
else fires a spike and e(t) is set to Vreset
Spiking neurons, 2
What is new
I An asynchronous process
I What matters is the dynamics of the input
Modelling/studying dynamics
I Synchrony in cell assembliesHebb, 49
I Complete synchronyMirollo, 90
I Transient synchronyHopfield, 01
I Order-chaos phase transitionSchrauwen, 08
I PolychronizationIzhikevich, 06
I Rhythmic oscillationsBrunel, 03
Synchronization
In biological systems
I Fireflies
I Cricket chirping
I Pacemaker cells of the heart
I Neural cells
Questions
I Why (synchronized patterns are more efficient ?)
I How ?
ClaimEmergence/Dynamics results from individual interactions
Cole 91, Gordon 92
Division of labor among foraging ants
PrincipleThe ant goes foraging
iff she does not see sufficiently many ants foraging
Related problems
I The Dying seminar Schelling 1978
I The El Farol bar Arthur 1994
The Dying Seminar
Schelling, 1978; Nadal et al. 2009
Individual Utility Function
I N scientists are asked to go to a seminar:
I ... scientist i will go if #attendees > n(i)
The Dying Seminar
Schelling, 1978; Nadal et al. 2009
Individual Utility Function
I N scientists are asked to go to a seminar:
I ... scientist i will go if #attendees > n(i)
The Dying Seminar
Schelling, 1978; Nadal et al. 2009
Individual Utility Function
I N scientists are asked to go to a seminar:
I ... scientist i will go if #attendees > n(i)
The El Farol bar
Arthur 1994
Individual Utility Function 100 scientists
I The best option is to go to El Farol bar
I ... if not too many people go to the bar... < 60
I otherwise, better stay at home...
Devising a policy
I Random draw: go to the bar with probability .6
I Find rules to predict the attendance, based on the history
Division of labor among foraging ants
PrincipleThe ant goes foraging
iff she does not see sufficiently many ants foraging
Related problems
I The Dying seminar Schelling 1978
I The El Farol bar Arthur 1994
Differences
I Not an imitation game survival of the colony
I No synchronization
The foraging colony
A 4 state agent model
I S leep
I Observe
I Forage
I General Interest
S O
F
G
Ant policy
1. If I don’t see “sufficiently many” foraging ants,I go foraging (then sleeping)
2. Otherwise, I go for General Interest tasks
3. After any task, back to Observation
The ant model: Two spiking neurons
Passive neuron Leaky Integrate-and-Fire (LIF)
dVp
dt = −λ(Vp(t)− Vrest) + Iexc(t), if Vp < ϑelse fires a spike and Vp is set to V p
reset
Excitation: signal of working ants
Active neuron Quadratic Integrate-and-Fire (QIF)
dVa
dt = −λ(Va(t)− Vrest)(Va(t)− Vthres) + Iinh(t) + Iclock(t), if Va < ϑelse fires a spike and Va is set to V a
reset
Inhibition: signal of working ants
Excitation: internal clock
Bistable:
{> Vrest bursting regime< Vrest goto V a
reset
The ant model: Two spiking neurons, foll’d
During the observation state,Decision making: Competition of active and passive neuron
I if Active wins, goto F (and emits spikes, sent to neighborants)
I If Passive wins, goto GI if none wins before tO, goto F .
wins= emits a spikeOther states
I Passive and Active neurons are not excited/inhibited.
Microscopic Scale
Sleep state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Microscopic Scale
Observation state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Microscopic Scale
Foraging state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Microscopic Scale
Sleep state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Microscopic Scale
Observation state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Microscopic Scale
General Interest state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Microscopic Scale
Observation state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Microscopic Scale
Foraging state
PA
0
0.5
1
1.5
0 20 40 60 80 100 120
Active neuronPassive neuron
ϑ
Time (ms)
Macroscopic Scale
Ant colony
I M ants = M (active, passive) neurons
I A spiking neuron network
I Sparsely connected (connectivity ρ)
What happens ?n(t): number of foraging ants at time t
Foraging effort F =∑t
n(t)
Parameters of the model
Parameter type Symbol Description Value (units)
Neural λ Membrane relaxation constant 0.1 mV−1
Vrest Resting potential 0.0 mVϑ Spike firing threshold 1.0 mV
V preset Passive neuron reset potential -0.1 mV
Vthres Active neuron bifurcation threshold 0.5 mVV a
reset Active neuron reset potential 0.55 mVIclock Active neuron constant input current 0.1 mVw Synaptic weight 0.01 mV−1
Agent tF Foraging duration 47.1 mstO Maximum observation duration 10.5 mstS Sleeping duration 45.7 mstG General I. duration 16.7 ms
Population ρ Connection probability 0.3 %M Population size 150 agents
Initializationevery ant sleeps and wakes up after U[0, 2tS ]
Foraging effort: Sensitivity analysis
Average on 10 independent runs times 100,000 time steps
200
400
600
800
1000
0 200 400 600 800 1000200
300
400
500
600
700
0 0.2 0.4 0.6 0.8 1
FF
M ρ
0
500
1000
1500
0 0.05 0.1 0.15 0.2
Fw
200
240
280
0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95
F
V areset
vs population size M, connectivity ρ,active neuron reset potential V a
reset and synaptic weight w .
Influence of connectivity ρ
ρ = 0.1
ρ = 0.2
0
10
20
30
40
50
60
0 500 1000 1500 2000 2500 3000
nF
(t)
01020304050607080
0 500 1000 1500 2000 2500 3000
nF
(t)
t
Emergence of workshift as ρ increases.
Influence of population size M
0102030405060708090
0 500 1000 1500 2000 2500 3000
nF
(t)
300 agents
050
100150200250300350
0 500 1000 1500 2000 2500 3000
nF
(t)
t
1000 agents
Variance of workshift size increases with M
Macroscopic study, foll’d
First indicator: Foraging effort FI Behaves as expected
I high variance in some regions.
Second indicator: Entropy of synchronization HI Consider n(t) number of foraging agents at t
I Discard orphan time steps t s.t. n(t − 1) 6= n(t) 6= n(t + 1)
I LetN = {n(t), t = 1 . . .T , n(t) = n(t + 1) or n(t) = n(t − 1)}
I Let pn ∝ |{t, n(t) = n, n ∈ N}|H = −
∑n∈N
pn log pn
Three different regimes
0
10
20
30
40
50
60
70
80
0 500 1000 1500 2000
n F(t
)
Simulated time t
0
50
100
150
200
250
0 500 1000 1500 2000
n F(t
)Simulated time t
0100200300400500600700800900
1000
0 500 1000 1500 2000
n F(t
)
Simulated time t
Asynchronous Synchronous aperiodic Synchronous periodicA B C
0
10
20
30
40
50
60
70
80
0 10 20 30 40 50 60 70 80
n F(t
+1)
nF(t)
50
100
150
200
250
50 100 150 200 250
n F(t
+1)
nF(t)
200
400
600
800
1000
200 400 600 800 1000
n F(t
+1)
nF(t)
H = 0 High Log2
Three different regimes
0
10
20
30
40
50
60
70
80
0 500 1000 1500 2000
n F(t
)
Simulated time t
0
50
100
150
200
250
0 500 1000 1500 2000
n F(t
)
Simulated time t
0100200300400500600700800900
1000
0 500 1000 1500 2000
n F(t
)
Simulated time t
Asynchronous Synchronous aperiodic Synchronous periodicA B C
0
50
100
150
200
0 500 1000 1500 2000Simulated time t
0
50
100
150
200
0 500 1000 1500 2000Simulated time t
0
50
100
150
200
0 500 1000 1500 2000Simulated time t
Raster plot: Active = red, passive = blue
SpikeAnts: Emergent synchronization
Control parameters
I Sociability ρ√M
I Receptivity w|ϑ−Vrest|
Phase diagram
C
B
AC
B
A
0 0.2 0.4 0.6 0.8
0.05
0.1
0.15
0.2
Rec
epti
vity
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 0.2 0.4 0.6 0.8
0.05
0.1
0.15
0.2
Rec
epti
vity
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
H(m
ean)
H(s
tand
ard
devi
atio
n)
Sociability Sociability
SpikeAnts: Emergent synchronization
Control parameters
I Sociability ρ√M
I Receptivity w|ϑ−Vrest|
Phase diagram
C
B
AC
B
A
0 0.2 0.4 0.6 0.8
0.05
0.1
0.15
0.2
Rec
epti
vity
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 0.2 0.4 0.6 0.8
0.05
0.1
0.15
0.2
Rec
epti
vity
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
H(m
ean)
H(s
tand
ard
devi
atio
n)
Sociability Sociability
SpikeAnts: A representative run
at the triple point
B A B A B C
0
50
100
150
200
250
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
n F(t
)
t
Stable regime: synchronous periodic.
Overview
I Swarm Robotics
I Biological / Artificial modelsI SpikeAnts
I Spiking NeuronsI Network Architecture
I Analysis
I Discussion and Perspectives
SpikeAnts
The model
I Frugal, deterministic model
I Biological plausibility / no counting abilities
I Accounts for the emergence of synchronization
Further extensions
I Comparisons with probabilistic models
I Stochastic parameters
Further extensions
Reconsidering excitation/inhibitionFrom SpikeAnts to an Ising model
The environment handling perturbationsWhat can be learned/optimized within SpikeAnts ?
Going realImplementing SpikeAnts
Thanks
I Sylvain Chevallier, Helene Paugam Moisy TAO, LRI
I Jose Halloy, Jean-Louis Deneubourg VUB
I Symbrion IP
More in NIPS 2010.