EVENT-DRIVEN CONTROL, COMMUNICATION, AND OPTIMIZATION · Optimization EVENT-DRIVEN Control in Distributed Wireless Systems ... Perceived event density (data sources) over given region

C. G. Cassandras Division of Systems Engineering

and Dept. of Electrical and Computer Engineering and Center for Information and Systems Engineering

Boston University

Christos G. Cassandras CODES Lab. - Boston University

EVENT-DRIVEN CONTROL,

COMMUNICATION, AND OPTIMIZATION

TIME-DRIVEN v EVENT-DRIVEN CONTROL


REFERENCE PLANT CONTROLLER

INPUT

-

+

SENSOR MEASURED OUTPUT

OUTPUT ERROR

REFERENCE PLANT CONTROLLER

INPUT

-

+

SENSOR MEASURED OUTPUT

OUTPUT ERROR

EVENT: g(STATE) ≤ 0

EVENT-DRIVEN CONTROL: Act only when needed (or on TIMEOUT) - not based on a clock

OUTLINE


Reasons for EVENT-DRIVEN Control, Communication, and Optimization

EVENT-DRIVEN Control in Distributed Wireless Systems

EVENT-DRIVEN Sensitivity Analysis for Hybrid Systems

REASONS FOR EVENT-DRIVEN MODELS, CONTROL, OPTIMIZATION


Many systems are naturally Discrete Event Systems (DES) (e.g., Internet) → all state transitions are event-driven Most of the rest are Hybrid Systems (HS) → some state transitions are event-driven Many systems are distributed → components interact asynchronously (through events) Time-driven sampling inherently inefficient (“open loop” sampling)

REASONS FOR EVENT-DRIVEN MODELS, CONTROL, OPTIMIZATION


Many systems are stochastic → actions needed in response to random events Event-driven methods provide significant advantages in computation and estimation quality System performance is often more sensitive to event-driven components than to time-driven components

Many systems are wirelessly networked → energy constrained → time-driven communication consumes significant energy UNNECESSARILY!

CYBER-PHYSICAL SYSTEMS

Christos G. Cassandras CISE - CODES Lab. - Boston University

INTERNET

CYBER

PHYSICAL

Data collection: relatively easy…

Control: a challenge…

STATES

s1

s2 s3 s4

TIME t

TIME-DRIVEN SYSTEM

STATES

TIME t

STATE SPACE: X = ℜ

DYNAMICS: ( ) ,x f x t=

EVENT-DRIVEN SYSTEM

STATE SPACE: X s s s s= 1 2 3 4, , ,

DYNAMICS:

( )exfx ,'=t2

e2

x(t)

t3 t4 t5

e3 e4 e5 EVENTS

x(t)

t1

e1


TIME-DRIVEN v EVENT-DRIVEN SYSTEMS

SYNCHRONOUS v ASYNCHRONOUS BEHAVIOR


Wasted clock ticks

More wasted clock ticks

Even more wasted clock ticks …

INCREASING TIME GRANULARITY

Indistinguishable events

x + y x y

x y

TIME

Time-driven (synchronous) implementation: - Sum repeatedly evaluated unnecessarily - When evaluation is actually needed, it is done at the wrong times !

TIME

t1 t2

SYNCHRONOUS v ASYNCHRONOUS COMPUTATION


SELECTED REFERENCES - EVENT-DRIVEN CONTROL, COMMUNICATION, ESTIMATION, OPTIMIZATION


- Astrom, K.J., and B. M. Bernhardsson, “Comparison of Riemann and Lebesgue sampling for first order stochastic systems,” Proc. 41st Conf. Decision and Control, pp. 2011–2016, 2002. - T. Shima, S. Rasmussen, and P. Chandler, “UAV Team Decision and Control using Efficient Collaborative Estimation,” ASME J. of Dynamic Systems, Measurement, and Control, vol. 129, no. 5, pp. 609–619, 2007.

- Heemels, W. P. M. H., J. H. Sandee, and P. P. J. van den Bosch, “Analysis of event-driven controllers for linear systems,” Intl. J. Control, 81, pp. 571–590, 2008. - P. Tabuada, “Event-triggered real-time scheduling of stabilizing control tasks,” IEEE Trans. Autom. Control, vol. 52, pp. 1680–1685, 2007. - J. H. Sandee, W. P. M. H. Heemels, S. B. F. Hulsenboom, and P. P. J. van den Bosch, “Analysis and experimental validation of a sensor-based event-driven controller,” Proc. American Control Conf., pp. 2867–2874, 2007. - J. Lunze and D. Lehmann, “A state-feedback approach to event-based control,” Automatica, 46, pp. 211–215, 2010. - P. Wan and M. D. Lemmon, “Event triggered distributed optimization in sensor networks,” Proc. of 8th ACM/IEEE Intl. Conf. on Information Processing in Sensor Networks, 2009. - Zhong, M., and Cassandras, C.G., “Asynchronous Distributed Optimization with Event-Driven Communication”, IEEE Trans. on Automatic Control, AC-55, 12, pp. 2735-2750, 2010.

EVENT-DRIVEN CONTROL

IN DISTRIBUTED (USUALLY WIRELESS)

SYSTEMS


MOTIVATIONAL PROBLEM: COVERAGE CONTROL

Deploy sensors to maximize “event” detection probability – unknown event locations – event sources may be mobile – sensors may be mobile

Perceived event density (data sources) over given region (mission space)

0

5

10

02

46

8100

10

20

30

40

50

Ω

R(x) (Hz/m2)

? ? ? ? ? ?

? ? ?

• Meguerdichian et al, IEEE INFOCOM, 2001 • Cortes et al, IEEE Trans. on Robotics and Automation, 2004 • Cassandras and Li, Eur. J. of Control, 2005 • Ganguli et al, American Control Conf., 2006 • Hussein and Stipanovic, American Control Conf., 2007 • Hokayem et al, American Control Conf., 2007

OPTIMAL COVERAGE IN A MAZE


Zhong and Cassandras, 2008

http://www.bu.edu/codes/research/distributed-control/

COVERAGE: PROBLEM FORMULATION

Sensing attenuation: pi(x, si) monotonically decreasing in di(x) ≡ ||x - si||

Data source at x emits signal with energy E

N mobile sensors, each located at si∈R2

Signal observed by sensor node i (at si )

SENSING MODEL: ]),(|by Detected[),( iii sxAiPsxp ≡

( A(x) = data source emits at x )


0

5

10

02

46

8100

10

20

30

40

50

Ω

R(x) (Hz/m2)

? ? ? ? ? ?

? ? ?

Joint detection prob. assuming sensor independence ( s = [s1,…,sN] : node locations)

[ ]∏=

−−=N

iii sxpxP

1

),(11),( s

OBJECTIVE: Determine locations s = [s1,…,sN] to maximize total Detection Probability:

),()(max dxxPxR∫Ω

ss

COVERAGE: PROBLEM FORMULATION


Perceived event density

Event sensing probability

CONTINUED DISTRIBUTED COOPERATIVE SCHEME

Set

[ ] dxxpxRssHN

iiN ∫ ∏

Ω =

−−=1

1 )(11)(),,(


Maximize H(s1,…,sN) by forcing nodes to move using gradient information:

[ ] dxxdxs

xdxpxpxR

sH

k

k

k

kN

kiii

k∫ ∏Ω ≠=

−∂∂

−=∂∂

)()()()(1)(

,1

ki

kki

ki s

Hss∂∂

+=+ β1Desired displacement = V·∆t

Cassandras and Li, EJC, 2005 Zhong and Cassandras, IEEE TAC, 2011

CONTINUED DISTRIBUTED COOPERATIVE SCHEME

… has to be autonomously evaluated by each node so as to determine how to move to next position:

CONTINUED

Use truncated pi(x) ⇒ Ω replaced by node neighborhood Ωi

Discretize pi(x) using a local grid


[ ] dxxdxs

xdxpxpxR

sH

k

k

k

kN

kiii

k∫ ∏Ω ≠=

−∂∂

−=∂∂

)()()()(1)(

,1

ki

kki

ki s

Hss∂∂

+=+ β1

DISTRIBUTED COOPERATIVE OPTIMIZATION


i

Nss

sts

ssHN

each on sconstraint ..

),,(min 1,,1

1

1

on sconstraint ..

),,(min1

sts

ssH Ns

N

Ns

sts

ssHN

on sconstraint ..

),,(min 1

…

N system components (processors, agents, vehicles, nodes), one common objective:

DISTRIBUTED COOPERATIVE OPTIMIZATION


i

Controllable state si, i = 1,…,ni

))(()()1( kdksks iiii sα+=+

Step Size

Update Direction, usually ))(())(( kHkd ii ss −∇=

i

Ns

sts

ssHi

on sconstraint ..

),,(min 1

i requires knowledge of all s1,…,sN

Inter-node communication

SYNCHRONIZED (TIME-DRIVEN) COOPERATION


1

2

3

COMMUNICATE + UPDATE

Drawbacks: Excessive communication (critical in wireless settings!) Faster nodes have to wait for slower ones Clock synchronization infeasible Bandwidth limitations Security risks

ASYNCHRONOUS COOPERATION


1

2

3

Nodes not synchronized, delayed information used

Bertsekas and Tsitsiklis, 1997

Update frequency for each node is bounded + technical conditions

⇒ ))(()()1( kdksks iiii sα+=+

converges

ASYNCHRONOUS (EVENT-DRIVEN) COOPERATION


2

3

UPDATE COMMUNICATE

UPDATE at i : locally determined, arbitrary (possibly periodic) COMMUNICATE from i : only when absolutely necessary

1

WHEN SHOULD A NODE COMMUNICATE?


Node state at any time t : xi(t)

Node state at tk : si(k)

⇒ si(k) = xi(tk)

j

i tk

)(kjτEstimate examples:

))(()( kxks jj

ij τ= Most recent value

( )))(()())(()( kxdktkxks jjji

j

jkj

jij ταττ ⋅⋅

∆−

+= Linear prediction

: node j state estimated by node i )(ksijAT UPDATE TIME tk :



AT ANY TIME t :

If node i knows how j estimates its state, then it can evaluate )(tx ji

Node i uses • its own true state, xi(t) • the estimate that j uses, )(tx j

i

… and evaluates an ERROR FUNCTION ( ))(),( txtxg jii

Error Function examples: 21)()( ,)()( txtxtxtx j

iij

ii −−

: node i state estimated by node j )(tx ji


i i

j

)(txi

δi


Node i communicates its state to node j only when it detects that its true state xi(t) deviates from j’ estimate of it so that

)(tx ji

( ) ij

ii txtxg δ≥)(),(

( ))(),( txtxg jiiCompare ERROR FUNCTION to THRESHOLD δi

⇒ Event-Driven Control


CONVERGENCE

Asynchronous distributed state update process at each i: ))(()()1( kdksks i

iii s⋅+=+ α Estimates of other nodes, evaluated by node i

THEOREM: Under certain conditions, there exist positive constants α and Kδ such that

0))((lim =∇∞→

kHk

s

INTERPRETATION: Event-driven cooperation achievable with minimal communication requirements ⇒ energy savings

Zhong and Cassandras, IEEE TAC, 2010

−=

otherwise)1(update sends if)(()(

kkkdKk

i

ii

i δδ δ s


COONVERGENCE WHEN DELAYS ARE PRESENT

Red curve:

Black curve:

ij0τ t

0

( )kiδ

( )jii xxg ~,

( )jii xxg ,

ij3τij

1τ ij2τ ij

1σ ij2σ ij

3σ ij4σij

4τ

( )jii xxg ,

( )kiδ

0tij

3τij2τij

1τij0τ

Error function trajectory with NO DELAY

DELAY


COONVERGENCE WHEN DELAYS ARE PRESENT

ASSUMPTION: There exists a non-negative integer D such that if a message is sent before tk-D from node i to node j, it will be received before tk. INTERPRETATION: at most D state update events can occur between a node sending a message and all destination nodes receiving this message.

Add a boundedness assumption:

THEOREM: Under certain conditions, there exist positive constants α and Kδ such that

0))((lim =∇∞→

kHk

s

NOTE: The requirements on α and Kδ depend on D and they are tighter.

Zhong and Cassandras, IEEE TAC, 2010

SYNCHRONOUS v ASYNCHRONOUS OPTIMAL COVERAGE PERFORMANCE


SYNCHRONOUS v ASYNCHRONOUS: No. of communication events for a deployment problem with obstacles

SYNCHRONOUS v ASYNCHRONOUS: Achieving optimality in a problem with obstacles

Energy savings + Extended lifetime

DEMO: OPTIMAL DISTRIBUTED DEPLOYMENT WITH OBSTACLES – SIMULATED AND REAL



DEMO: REACTING TO EVENT DETECTION

Important to note:

There is no external control causing this behavior. Algorithm includes tracking functionality automatically

BOSTON UNIVERSITY TEST BEDS

SMARTS Kickoff Meeting Christos G. Cassandras CISE - CODES Lab. - Boston University

EVENT-DRIVEN SENSITIVITY

ANALYSIS

REAL-TIME STOCHASTIC OPTIMIZATION

CONTROL/DECISION (Parameterized by θ) SYSTEM PERFORMANCE

NOISE


)]([ θLE

)]([max θθ

LEΘ∈

GOAL:

L(θ) GRADIENT ESTIMATOR

)(1 nnnn L θηθθ ∇+=+

)(θL∇ x(t)

DIFFICULTIES: - E[L(θ)] NOT available in closed form - not easy to evaluate - may not be a good estimate of

)(θL∇)(θL∇ )]([ θLE∇

REAL-TIME STOCHASTIC OPTIMIZATION FOR DES: INFINITESIMAL PERTURBATION ANALYSIS (IPA)

CONTROL/DECISION (Parameterized by θ)

Discrete Event System (DES)

PERFORMANCE

L(θ) IPA

NOISE

)(1 nnnn L θηθθ ∇+=+

Sample path

For many (but NOT all) DES: - Unbiased estimators - General distributions - Simple on-line implementation

[Ho and Cao, 1991], [Glasserman, 1991], [Cassandras, 1993, 2008]

)]([ θLE


)(θL∇

x(t) Model

x(t)

REAL-TIME STOCHASTIC OPTIMIZATION: HYBRID SYSTEMS

CONTROL/DECISION (Parameterized by θ)

HYBRID SYSTEM

PERFORMANCE

L(θ) IPA )(1 nnnn L θηθθ ∇+=+

A general framework for an IPA theory in Hybrid Systems?

Sample path


)]([ θLE

)(θL∇

NOISE

x(t)

Performance metric (objective function):

PERFORMANCE OPTIMIZATION AND IPA

( ) ( )[ ]TxLETxJ ),0,(;),0,(; θθθθ =

( ) ∑ ∫=

+

=N

kk

k

k

dttxLL0

1

),,(τ

τ

θθ

( ) ( ) ( )θθττ

θθ

∂∂

=′∂

∂=′ k

ktxtx ,,

NOTATION:

( )θθθ

dTxdJ ),0,(;

IPA goal: - Obtain unbiased estimates of , normally

- Then: θθηθθ

ddL n

nnn)(

1 +=+

( )θθ

ddL


HYBRID AUTOMATA


HYBRID AUTOMATA

Q: set of discrete states (modes) X: set of continuous states (normally Rn) E: set of events U: set of admissible controls

),,,,,,,,,,( 00 xqguardInvfUEXQGh ρφ=

f : vector field, φ : discrete state transition function,

XUXQf →××:QEXQ →××:φ

Inv: set defining an invariant condition (domain), guard: set defining a guard condition, ρ : reset function,

q0: initial discrete state x0: initial continuous state

XQInv ×⊆XQQguard ××⊆

XEXQQ →×××:ρ

HYBRID AUTOMATA


HYBRID AUTOMATA Unreliable machine with timeouts

T=τ

00

0

==

τx 1

1

==

τ ux

][ T<τ

α

00

2

==

τx

β, Kx ≥

γ

0'0'

==

τx

0'0'

==

τx

IDLE BUSY DOWN

x(t) : physical state of part in machine τ(t): clock

α : START, β : STOP, γ : REPAIR

Invariant

Guard Reset

THE IPA CALCULUS

IPA: THREE FUNDAMENTAL EQUATIONS

1. Continuity at events:

Take d/dθ :

)()( −+ = kk xx ττ

kkkkkkk ffxx ')]()([)(')(' 1 τττττ +−−

−+ −+=

If no continuity, use reset condition ⇒ θ

δυρτd

xqqdx k),,,,()('

′=+


System dynamics over (τk(θ), τk+1(θ)]: ),,( txfx k θ=

( ) ( ) ( )θθττ

θθ

∂∂

=′∂

∂=′ k

ktxtx ,,

NOTATION:


Solve over (τk(θ), τk+1(θ)]: θ∂

∂+

∂∂

=)()(')()(' tftx

xtf

dttdx kk

′+∫

∂∂∫=′ ∫ +∂

∂−

∂∂ t

k

duxuf

kdu

xuf

k

v

k

kt

k

k

xdvevfetxτ

τθ

ττ )()()()()(

initial condition from 1 above


2. Take d/dθ of system dynamics over (τk(θ), τk+1(θ)]: ),,( txfx k θ=

θ∂∂

+∂

∂=

)()(')()(' tftxxtf

dttdx kk

NOTE: If there are no events (pure time-driven system), IPA reduces to this equation

3. Get depending on the event type:


kτ ′

- Exogenous event: By definition,

0=′kτ

0)),,(( =θτθ kk xg- Endogenous event: occurs when

′

∂∂

+∂∂

∂∂

−=′ −−

− )()(1

kkkk xxggf

xg τ

θττ

- Induced events:

)()( 1+

−

′

∂∂

−=′ kkkk

k yt

y τττ


Ignoring resets and induced events:


kkkkkkk ffxx ')]()([)(')(' 1 τττττ ⋅−+= +−−

−+

+∫

∂∂∫=′ ∫ +∂

∂−

∂∂ t

k

duxuf

kdu

xuf

k

v

k

kt

k

k

xdvevfetxτ

τθ

ττ )(')()()()(

0=′kτ

′

∂∂

+∂∂

∂∂

−=′ −−

− )()(1

kkkk xxggf

xg τ

θττor

1. 2.

3.


2 1

3

)(' −kx τ

( ) ( )

( )θθττ

θθ

∂∂

=′

∂∂

=′

kk

txtx ,Recall:

Cassandras et al, Europ. J. Control, 2010

IPA PROPERTIES

Back to performance metric: ( ) ∑ ∫=

+

=N

kk

k

k

dttxLL0

1

),,(τ

τ

θθ

( ) ( )θ

θθ∂

∂=′ txLtxL k

k,,,,NOTATION:

Then: ( ) ∑ ∫=

++

′+⋅′−⋅′=

+N

kkkkkkkk

k

k

dttxLLLd

dL0

11

1

),,()()(τ

τ

θττττθθ

What happens at event times

What happens between event times


IPA PROPERTIES: ROBUSTNESS

THEOREM 1: If either 1,2 holds, then dL(θ)/dθ depends only on information available at event times τk: 1. L(x,θ,t) is independent of t over [τk(θ), τk+1(θ)] for all k

2. L(x,θ,t) is only a function of x and for all t over [τk(θ), τk+1(θ)]:

0=∂∂

=∂∂

=∂∂

θkkk f

dtd

xf

dtd

xL

dtd

IMPLICATION: - Performance sensitivities can be obtained from information limited to event times, which is easily observed - No need to track system in between events !

( ) ∑ ∫=

++

′+⋅′−⋅′=

+N

kkkkkkkk

k

k

dttxLLLd

dL0

11

1

),,()()(τ

τ

θττττθθ


[Yao and Cassandras, 2010]

IPA PROPERTIES : ROBUSTNESS

EXAMPLE WHERE THEOREM 1 APPLIES (simple tracking problem):


k

k

k

kk

k

k

ddufa

xf

θθ=

∂∂

=∂∂

⇒ , Nk

twutxax

dtgtxE

kkkkkk

T

,,1 )()()( s.t.

)]()([min0

,

=++=

−∫

θ

φφθ 1 =

∂∂

⇒xL

NOTE: THEOREM 1 provides sufficient conditions only. IPA still depends on info. limited to event times if for “nice” functions uk(θk,t), e.g., bkθt

Nktwtutxax kkkkkk

,,1)(),()(

=++= θ

IPA PROPERTIES: DECOMPOSABILITY

THEOREM 2: Suppose an endogenous event occurs at τk with switching function g(x,θ). If , then is independent of fk−1.

If, in addition, then

IMPLICATION: Performance sensitivities are often reset to 0 ⇒ sample path can be conveniently decomposed

0)( =+kkf τ

0=θd

dg 0)( =′ +kx τ

)( +′ kx τ


IPA PROPERTIES

EVENTS

Evaluating requires full knowledge of w and f values (obvious) );( θtx

However, may be independent of w and f values (NOT obvious) θ

θdtdx );(

It often depends only on: - event times τk - possibly )( 1

−+kf τ

);,,,( θtwuxfx =

τk τk+1


IPA PROPERTIES

- No need for a detailed model (captured by fk) to describe state behavior in between events - This explains why simple abstractions of a complex stochastic system can be adequate to perform sensitivity analysis and optimization, as long as event times are accurately observed and local system behavior at these event times can also be measured. - This is true in abstractions of DES as HS since: Common performance metrics (e.g., workload) satisfy THEOREM 1

In many cases:


THE CLASSIC SCHEDULING PROBLEM: cµ-RULE


Problem:

cµ-rule: Always serve the non-empty queue with highest ciµi value

.,,1,0 , )(1min0

1,,1)(NicdttxcE

T i

T N

iiiNtu

=>

∫ ∑

=∈

x1(t)

x2(t)

xN(t)

µ1

µ2

µN

SERVICE RATES

ARRIVAL PROCESSES

… CONTROLLER:

,,1)( Ntu ∈

NOTE: cµ rule is an (almost) static control policy!

OPTIMALITY OF cµ-RULE


• Deterministic model Smith, 1956

• Classical Queueing Theory: - M/G/1 system - Cox and Smith,1961 - Discrete time, general arrivals, geometrically distributed service - Baras et al., 1985; Buyukkoc et al., 1985 - Discrete time, service times with increasing/decreasing failure rates – Hirayama et al., 1989

• Fluid models: - Deterministic - Chen and Yao, 1993; Avram et al., 1995 - Fluid limits (heavy traffic) – Kingman, 1961; Whitt, 1968; Harrison, 1968; Mieghem, 1995

STOCHASTIC FLOW MODEL FOR SCHEDULING


1);();(

2

2

1

1 ≤+µ

θµ

θ tutu

State dynamics:

−≥=

== + otherwise);()()()(,0)(0)();(

θαα

θtut

ttutxdt

tdxtfnn

nnnnn

>−

=

−=

0)())(1(

0)())(1(),(min)(

21

12

21

122

2

txtu

txtuttu

µµ

µµα

>=

=0)()(0)()(),(min

)(11

1111 txt

txtttu

θµθµα

]1 ,0[)( ∈tθ

);(1 θtx

);(2 θtx

)(1 tα

)(2 tα

1µ

2µ

);(1 θtu

);(2 θtu

Capacity Constraint:

nn tu µθ ≤≤ );(0

RANDOM PROCESSES

IPA FOR LINEAR HOLDING COSTS


dttxctxcT

QT

)]()([1)( 220 11∫ +=θ Sample function:

NOTE: Result independent of inflow rate process αn(t) ⇒ Universality of cµ-rule !

THEOREM: If c1µ1 > c2µ2 , then 0)( <′ θQ

=

>= 0)()(

0)(1)(

11

1

1*

txttx

tµ

αθ cµ-rule is optimal

Kebarighotbi and Cassandras, J. DEDS, 2011

Proof: Use IPA CALCULUS to determine and show it is < 0 )(θQ′

CONCLUSIONS


Seek to combine TIME-DRIVEN with EVENT-DRIVEN Control, Communication, and Optimization and exploit their relative advantages and disadvantages

EVENT-DRIVEN Control in Distributed Wireless Systems:

- Act only when necessary (when specific events occur)

EVENT-DRIVEN Sensitivity Analysis for Hybrid System

- Sensitivities depend mostly on events and are robust with respect to noise

THANK YOU

Documents

EVENT-DRIVEN CONTROL, COMMUNICATION, AND OPTIMIZATION · Optimization EVENT-DRIVEN Control in Distributed Wireless Systems ... Perceived event density (data sources) over given region