Embed Size (px)
DESCRIPTION
Decentralised Coordination of Continuously Valued Control Parameters using the Max-Sum Algorithm. Ruben Stranders , Alessandro Farinelli , Alex Rogers, Nick Jennings School of Electronics and Computer Science University of Southampton {rs06r, af2, acr , nrj }@ ecs.soton.ac.uk. - PowerPoint PPT Presentation
Citation preview
Decentralised Coordination of Continuously Valued Control Parameters using the
Max-Sum Algorithm
Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick JenningsSchool of Electronics and Computer Science
University of Southampton{rs06r, af2, acr, nrj}@ecs.soton.ac.uk
2
This presentation focuses on the use of Max-Sum in coordination problems with continuous parameters
From Discrete to Continuous
Max-Sum for Decentralised Coordination
Empirical Evaluation
3
Max-Sum is a powerful algorithm for solving DCOPs
Complete Algorithms
DPOPOptAPOADOPT
Communication Cost
Iterative AlgorithmsBest Response (BR)
Distributed Stochastic Algorithm (DSA)
Fictitious Play (FP)
Max-SumAlgorithm
Optimality
Max-Sum solves the social welfare maximisation problem in a decentralised way
Agents
Max-Sum solves the social welfare maximisation problem in a decentralised way
1x
2x
3x
4x
5x
6x
7x8x
Control Parameters
Max-Sum solves the social welfare maximisation problem in a decentralised way
1U
2U
3U
4U
5U
6U
7U8U
Utility Functions
Max-Sum solves the social welfare maximisation problem in a decentralised way
)( 33 xU
Localised Interaction
},,,{ 54313 xxxxx
Max-Sum solves the social welfare maximisation problem in a decentralised way
Agents
M
iiiU
1
)(maxarg xx
Social welfare:
The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph
Variable nodes Function nodes
1x
2x
3x
1U
2U
3U
Agent 1Agent 2
Agent 3
Max-Sum solves the social welfare maximisation problem by message passing
1x
2x
3x
1U
2U
3U
Variable nodes Function nodes
Agent 1Agent 2
Agent 3
Max-Sum solves the social welfare maximisation problem by message passing
jiadjk
iikiji xrxq\)(
)()(
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
From variable i to function j
From function j to variable i
Until now, Max-Sum was only defined for discretely valued variables
Graph Colouring)( iji xq )( iij xr
However, many problems are inherently continuous. Heading
andVelocity
Unattended Ground Sensor
ActivationTime
Autonomous Ground Robot
Thermostat
Preferred RoomTemperature
So, we extended the Max-Sum algorithm to operate in continuous action spaces
Discrete Continuous
We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)
We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)
)( iji xq )( iij xr
“Continuous” Graph Colouring
A CPLF is defined by a domain partitioning followed by value assignment
A CPLF is defined by a domain partitioning followed by value assignment
A CPLF is defined by a domain partitioning followed by value assignment
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
To make Max-Sum work on CPLFs, we need to define key two operations on them
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
To make Max-Sum work on CPLFs, we need to define key two operations on them
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i1. Addition
of two CPLFs
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
To make Max-Sum work on CPLFs, we need to define key two operations on them
jiadjk
iikiji xrxq\)(
)()(From variable i to function j
From function j to variable i
2. Marginal Maximisation to a single variable
Addition of two CPLFs involves merging their domains, and then summing their values
1x
2x
1x
2x
1f 2f
Addition of two CPLFs involves merging their domains, and then summing their values
1x
2x
1x
2x
1x
2x
1. Merge domains
1f 2f
21 ff
Addition of two CPLFs involves merging their domains, and then summing their values
Addition of two CPLFs involves merging their domains, and then summing their values
2. Sum Values
Marginal maximisation is the operation of finding the maximum value of a function, if we fix all but one variable
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
From function j to variable i:
Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
)(max)(\ jjii fxgj
xx
Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable
ijadjk
kjkjjiiij xqUxrj \)(\
)()(max)( xx
)(max)(\ jii fxgj
xx
),(max)( 2112
xxfxgx
Example: bivariate function:
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
),( 1 fxProject onto
axis
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Project onto
axis),( 1 fx
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Project onto
axis),( 1 fx
Result of projection
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Extract UpperEnvelope
Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction
Extract UpperEnvelope
),(max)( 2112
xxfxgx
)( 1xg
We empirically evaluated this algorithm in a wide-area surveillance scenario
Dense deployment of sensors to detect activity within an urban environment.
Unattended GroundSensor
Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors
time
duty cycleDiscrete
time
duty cycle
time
duty cycle
Discretised time
Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors
time
duty cycleDiscrete Continuous
time
duty cycle
time
duty cycle
time
duty cycle
time
duty cycle
time
duty cycle
Continuous Max-Sum outperforms Discrete Max-Sum by up to 10%
0 5 10 15 20 25 300.75
0.80
0.85
0.90
0.95
1.00
ContinuousDiscrete
Discretisation
Solu
tion
Qua
lity
(as f
racti
on o
f opti
mal
)
Average Solution Quality over 25 Iterations
Tota
l Mes
sage
Size
Continuous Max-Sum leads to more effective use of communication resources than Discrete Max-Sum
0 5 10 15 20 25 300
5000
10000
15000
20000
ContinuousDiscrete
Discretisation
Total number of values exchanged between agents
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
1. No artificial discretisation
time
time
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
1. No artificial discretisation Continuous
Discrete
2. Better solutions
time
timeSo
lutio
n Q
ualit
y
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
time
time1. No artificial discretisation Continuous
Discrete
2. Better solutions ContinuousDiscrete
3. Effective communication
Solu
tion
Qua
lity
Mes
sage
Size
For future work, we wish to extend the algorithm to arbitrary continuous functions
For example, using Gaussian Processes
In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum
time
time1. No artificial discretisation Continuous
Discrete
2. Better solutions ContinuousDiscrete
3. Effective communication
Solu
tion
Qua
lity
Mes
sage
Size
Questions?
