Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Problem IntroductionDynamic Programming Formulation
Project
Williams et.al. – Approximate DynamicProgramming for Communication-Constrained
Sensor Network ManagementAn Experimental Review and Visual Tool
Jonatan Schroeder
Project Outline – CPSC532MUniversity of British Columbia
Mar 17, 2008
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Outline
1 Problem Introduction
2 Dynamic Programming Formulation
3 Project
Based on: J. L. Williams, J. W. Fisher III, and A. S. Willsky. Approximatedynamic programming for communication-constrained sensor networkmanagement. IEEE Transactions on Signal Processing, 55(8):4300–4311,August 2007.
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Sensor Networks
Spatially distributed autonomous devicesSensor to monitor:
temperature, sound, pressurepollutantsexternal agentsbattlefield surveillance
Wireless communicationLimited energy
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Example
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Example
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Example
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Example
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Example
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Example
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Example
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
The Problem
Identify the state (position, velocity) of the objectProbability Distribution Function (pdf)Estimate the object’s next state
Subset of sensors and a leader sensor
Objectives:Maximize the information estimation performanceMinimize the communication cost
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
The Problem
Identify the state (position, velocity) of the objectProbability Distribution Function (pdf)Estimate the object’s next state
Subset of sensors and a leader sensorObjectives:
Maximize the information estimation performanceMinimize the communication cost
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Dynamic Programming Approach
Objective function: estimation performance orcommunication cost
Lagrangian relaxation
State: object actual positionPolicy: leader and sensor subset selectionRolling discrete horizon
In each step, N steps are calculated, and one is taken
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Object Dynamics
Object state: xk =
pxvxpyvy
Object dynamics: xk+1 = Fxk + wk
F =
1 T 0 00 1 0 00 0 1 T0 0 0 1
wk is a white Gaussian noise process
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Measurement Model
Measurement model: zsk = h(xk , s) + vs
k
h(xk , s) =a
‖Lxk − ys‖22 + b
vsk is a white Gaussian noise process
z0:k is the history of all measurements {z0, z1, . . . , zk}Estimation: in progress
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Communication
Cost (per bit) of direct communication: C̃ij ∝∥∥y i − y j
∥∥22
Cost considering path {i0, i1, . . . , inij}:
Cij =
nij∑k=1
C̃ik−1ik
Transmission of probabilistic model: Bp bitsTransmission of measurements: Bm bits
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Constrained Dynamic Programming
Conditional belief state: Xk , p (xk |z0:k−1) ∈ P(X )
Control: uk = (lk ,Sk ), lk ∈ S and Sk ⊂ SControl policy – time k : µk (Xk , lk−1)
Control policy set: πk = {µk , . . . , µk+N−1}
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Optimization Problem
minπ
E
[k+N−1∑
i=k
g (Xi , li−1, µi (Xi , li−1))
]
s.t.E
[k+N−1∑
i=k
G (Xi , li−1, µi (Xi , li−1))
]≤ M
Constraint addressed through Lagrangian relaxation:
Lk (Xk , lk−1, πk , λ) = E
[k+N−1∑
i=k
g (Xi , li−1, µi (Xi , li−1))
+ λ
(k+N−1∑
i=k
G (Xi , li−1, µi (Xi , li−1))−M
)]
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Optimization Problem
minπ
E
[k+N−1∑
i=k
g (Xi , li−1, µi (Xi , li−1))
]
s.t.E
[k+N−1∑
i=k
G (Xi , li−1, µi (Xi , li−1))
]≤ M
Constraint addressed through Lagrangian relaxation:
Lk (Xk , lk−1, πk , λ) = E
[k+N−1∑
i=k
g (Xi , li−1, µi (Xi , li−1))
+ λ
(k+N−1∑
i=k
G (Xi , li−1, µi (Xi , li−1))−M
)]Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Dynamic Programming Function
Lk (Xk , lk−1, πk , λ) = E
[k+N−1∑
i=k
g (Xi , li−1, µi (Xi , li−1))
+ λ
(k+N−1∑
i=k
G (Xi , li−1, µi (Xi , li−1))−M
)]
JDk (Xk , lk−1, λ) = min
πkLk (Xk , lk−1, πk , λ)
JLk (Xk , lk−1) = max
λ≥0JD
k (Xk , lk−1, λ)
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Dynamic Programming Function
Lk (Xk , lk−1, πk , λ) = E
[k+N−1∑
i=k
g (Xi , li−1, µi (Xi , li−1))
+ λ
(k+N−1∑
i=k
G (Xi , li−1, µi (Xi , li−1))−M
)]
JDk (Xk , lk−1, λ) = min
πkLk (Xk , lk−1, πk , λ)
JLk (Xk , lk−1) = max
λ≥0JD
k (Xk , lk−1, λ)
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Complexity Evaluation
Ns sensorsNp samples for each policyN steps (horizon)Ns2Ns possible controlsNs2NsNp new total samplesO(NN
s 2NsNNNp )
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Linearized Gaussian Approximation
Considering the smoothness of zSkk
Function for entropy evaluation is simplifiedO(NN
s 2NsN)
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Greedy Sensor Subnet Selection
Each stage is divided in substagesA new problem is defined, algebraically equivalent to theoriginal problemSensors that increase the cost in a substage are discardedSensors are limited to a small neighbourhoodWorst-case complexity: O(NN3
s )
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Dynamic Programming Problem Implementation
Implementation of the problem described in this paperBoth communication contraint (CC) and informationconstraint (IC) approachesRepeat simulation resultsObtain extended resulsTool for visualizing:
Agents positionObject stateLeader and subset selection
Jonatan Schroeder Approximate DP for Sensor Network Management
Problem IntroductionDynamic Programming Formulation
Project
Williams et.al. – Approximate DynamicProgramming for Communication-Constrained
Sensor Network ManagementAn Experimental Review and Visual Tool
Jonatan Schroeder
Project Outline – CPSC532MUniversity of British Columbia
Mar 17, 2008
Jonatan Schroeder Approximate DP for Sensor Network Management