Fernando Paganini

Universidad ORT Uruguay

Population dynamics in networks:

from queues to PDEs and control,

from P2P to deferrable power loads.

Universidad ORT Uruguay

IMA, Sept 2015.

Joint work with

Andrés Ferragut

Universidad ORT Uruguay

Outline

1. Networking research: between queues and fluids.

2. M/G- Processor Sharing queues, point process

model and its PDE fluid counterpart.

3. Dynamics of peer-to-peer networks.

4. Dispatching deferrable power loads.

Mathematics in networking - a historical view• In the beginning we had queueing theory…

– Erlang, circa 1910, dimensioning telephone switches.

• Statistical multiplexing inspired packet networks…

– Kleinrock, 60s – 70s, Internet pioneer and queueing theorist.

• But queueing theory reached its limits in networking:

– Limited results in network case, dubious assumptions. – Limited results in network case, dubious assumptions.

• Turn to macroscopic fluid flow models:

– Bertsekas/Gallager in late ’70s, Kelly et al in ’90s.

– Cover elastic traffic, apply to general networks.

– Early ’00s, Network Utility Maximization for multiple layers.

• Still, Internet jobs remain statistically multiplexed.

– Queuing theory has a comeback.

– But fluid models appear also at this scale.

Classical M/M/1 queues

λPoisson ( ) arrivals

Equivalently: exponential job size,

constant service rate.

( )A t

t

( )X t

exp( ) service times µ

Markov chain, state

is queue occupancy

1

( ) (1 ), 0.

.Stable (ergodic) if

Invariant distribution is geometric:

xx x

λρ

µ

π ρ ρ

= <

= − ≥

x-1 x

λ

µ

exp( )λt

t

Networks of Queues and Data networks

• Queueing theory concepts influential in the early days of data networks: statistical multiplexing motivates “packet switching”.

• But networks of M/M/1 queues are not easy to analyze. The steady-state distribution only known under narrow assumptions (Jackson networks).

• Besides, packet networks do not match model assumptions: sending times not Poisson, packet length not exponential.

Going from “molecular” to fluid models.

( ),

, ( ) 0 .

Real valued following ODE

saturated to

x t

dxx t

dtλ µ= ≥−

Arrivals: rate pkt s/secλ

( )x t

Departures: pkt s/secµ

( )A t

t

tλ

( )( ) lim , (0) .

Relationship with Markov model for M/M/1 queue:

scaling limit where m

m

mX

X mtx t m

m→∞= =

t

• Beyond the M/M/1 assumptions, the fluid “tank” is a natural macroscopic first-principles model. Extends to varying rates.

• Allows for a simpler transition to the network case.

• Leads to way to study feedback control of rates.

Fluid view of network congestion control.

( )i

r t

feedback

• Congestion signals from scarce resources drive input rates.

,max ( ) .

CONCAVE LINK CAPACITYUTILITYCFUNCTION

Decentralized control is primal or dual solution of convex program

subject to

Network Utility Maximization model (Kelly et al).

r i i ili li i

U r R r c l≤ ∀∑ ∑�����

ONSTRAINTS

���������

The network at the scale of connections

• The previous fluid models treat connections as permanent.

• But connections (flows) arrive, are served and leave, there is

also statistical multiplexing at the scale of flows.

• Back to queueing theory! Distinguishing features:

1. Service discipline: all flows present are served

simultaneously, sharing the network capacity.

2. Job sizes: exponential distribution is too limiting.

Heavy tails observed for Internet files: many short

transfers, far fewer long connections.

Outline

1. Networking research: between queues and fluids.

2. M/G- Processor Sharing queues, point process

model and its PDE fluid counterpart.

3. Dynamics of peer-to-peer networks.

4. Dispatching deferrable power loads.

Processor Sharing: service capacity (bps) shared between

jobs present. Rate per job: . Special case of NUM . c

r

c

x =

i

x �c

)

)

(

Poisson( arrivals of file transfer jobs

Random file size, general distrib

(Si

ution.

ze ).G

λ

σ σ= >P

i

i

σ

( )G σ

The M/G/1-Processor Sharing Queue.

x-1 x

λ

cµ

’

Roberts and Massoulié 98 proposed this model for

TCP traffic, studied it in the case of exponential file sizes.

i

.Markov chain, stable if cλρµ

= <

jobs present. Rate per job: . Special case of NUM . c

rx

x =

Can we handle a more realistic model of file sizes? i

M/G/1-PS Queue.

But # of jobs is not the entire state: residual work matters.i

�

Compact representation of the state

(Gromoll et al. '02, P.Robert '03): σ

Good news (see Kelly '78):

PS queue is in steady state, the population of

jobs has the sam ge

i

om

nsen

(

sitiv

e distribution as in exponenti

e:

al e) cas .ρ

i

(Gromoll et al. '02, P.Robert '03):

Point measure on R , each point is a job, position is residual work.+

σxσ

3σ2σ1σ0

( ).

0.

Arrivals: a new point mass appears, following distribution

Service motion to the left, speed Depart when reaching .

G

c x

σ

σ= =

/ /The model applies also to the queue:

here, each job present has a firm capacity for itself.

Points move at constant spe d

e .

M G ∞

⇒

• At the scale of jobs, stochastic models are very natural.

• However, again they are hard to solve except for simple cases.

• For a more general bandwidth sharing, (e.g. given by NUM), it is not easy to find steady-state distributions.

• Turn again to fluid models:

• Job quantities become real numbers

Fluid models for M/G-PS Queues

• Job quantities become real numbers

• Differential equations replace Markov processes.

• How to account for residual work? Fluid distribution on R+

σ

( , )f t σ

Arriving "mass", follows file-size distribution

( )Service "advection", rate r σ=

Departures

For the point mass process,

the complementary cumulative

distribution is a decreasing,

piecewise constant stochastic process

( , )t σΦ

σnσ3σ2σ1σ

( )X t

( )x t

( , )F t σ( , )

representin

Fluid version:

g population o

deterministic

real-valued f

f jobs

unction F t σ

PDE using cumulative distributions

( , )F t σ

σrdt

with more residual work than . σ

( ) , )

:

(( , )

( , ) ( , )( ) ( , , )

ARRIVALS PROPAGATIONDynamics:

rate per job.

G dt F t rdt

r

F t dt

F t F tG r F t

t

λ σ σσ

σ σλ σ σσ

+ +

⇒

+ =

∂ ∂= +∂ ∂

���� ����

[ ’ , ] :P -Tang-Ferragut-Andrew '12 first use of PDE to prove a

stabil bandwidthit sy co harinjecture ng with for Internet general f ile sizes.

IEEE TAC

Outline

1. Networking research: between queues and fluids.

2. M/G- Processor Sharing queues, point process

model and its PDE fluid counterpart.

3. Dynamics of peer-to-peer networks.

4. Dispatching deferrable power loads.

BitTorrent-like peer-to-peer networks

S S

Liµ

• Content = set of pieces.

• Peers:

– Seeders own entire content, willing to disseminate it.

– Leechers downloading pieces, also contribute byuploading pieces they have.

L L L LL

LL

p2p capacity scales with demand

iµuploading pieces they have.

• Information on who owns which pieces, passed around to achieve efficient exchange.

• “Tit-for-tat” rules give incentives do cooperate.

iµ pWe a

er pssume up

eer is tload ban

he bottldwidth

eneck.

PEERS (~hundreds)

leechers seeders

PIECES

Population dynamics of a swarm of peers

Arrive Trans Depart

Filesharing PIECES

(~thousands)

sharing

Kesidis et al .07, Massou

[Yang-DeVe

lié-Vojnov

ciana '04, Qiu-Srikant '04]:

[ Zhu-Hajek '12ic '08, ]:

pop

Coarse model leech,

ulation state per sub

seed.

De

set of pi

tailed mod

eces. Huge

el

dimens

x yi

i

ion, limited results.

.

Our intermediate model: track profile of population as a function

of residual work σi

Leecher dynamics as an M/G-PS queue

( ) ( )[0,1)1Covers deterministic sizes:

(peers want all the file, start with no content).

G σ σ=

S S

L L L LL

LL

µ

µ

µσ

( ) ( )G Sσ σ= >P

( )[ ], 1.

seeking

download of ra

Leechers arrive as Poiss ,

ndom size me

n

n

o

aS S

λ=E�

i

"1 M/G/"Leecher dynamics is an Processor Sharing queue.−∞ +

0

0 0

( ).

( )

Assumptions (empirically validated for peers with

use of totaEfficient

Processor shari

l u

ng

pload capacity

: rate per leecher

homo

.

geneous ):

,

upR x y

x y yr

x x

µµ

µ µµ

= +

+= = +

i

i

0( ) ( )Populations: leechers. Take for simplicity seeder .s x t y t y≡

From stochastic to fluid model

( )0

0

( ) , 0(

. .)!

teady-state distribution of peers

present in the stochastic queu

Exploiting

e:

insensitivity, we find the s

x y

x C xx y

ρ λπ ρ ρ

µ

+ = ≥ = +

i

( ) .( ) [Pechinkin'83, Zachary '07].

Moreover, in steady state, residual jobs are independently drawn

from ccdf For instance,

demands yielddeterministic u a download profile in enifo

rm q

G G s dsσ

σ∞

= ∫

i

uilibrium.

( ) 0

( , , )

( , ) ( , )

More information from fluid model:

r F t

x yF t F tG

t xσ

σ σλ σ µσ

+∂ ∂= +

∂ ∂�����

( , )F t σ

σ

( )x t

0

( )

,

( ) ( ).

( , ) ( ) ( )

Note: in the case of exponentially distributed jobs,

the PDE admits a solution where satisfies

the ODE This is the model in [Qiu-Srikant '04]

.x y

G e

e

t x

F t x t x t

σ

σ

σ

λ µ

σ

−

−

=

= − +

=

0

0

( )

(0, ) ( )

.Fixed seeders

Consider a

with no leecher arrivals.

Initial population of lee , wi

transient sc

th partial cchers onten

ena

.

r

t

io

y t y

x F σ σ≡

= Φ

0( , ) ( , )x yF t F t

t x

σ σµ

σ+∂ ∂

=∂ ∂

Transient analysis with PDE model

Compare with detailed packet

simulations (in ns2) of BitTorrent11 ( ).

( )

Theorem: transient time to empty

system is T dy

σσ

µ σΦ

=Φ +∫

0 0

0

0

(0)( ) ( ),

1' log 1

Coarser ODE model:

Predicts completion time

Pessi. mistic !

x y x

x

y

t x x

T

µ

µ

== − +

= +

0

0 0

00

.( )

1 1.

system is

Bound:

y

T dy

xT

x

σµ σ

µ µ+

=Φ +

≤ ≤

∫

Including arrivals: equilibrium and stability.

0( ) ( ,0);( , ) ( , )

;0 ( ,1).

x t F tx yF t F t

F tt x

σ σλ µ

σ

=+∂ ∂= +

≡∂ ∂

( )[0,1)

0

*

( ) 1 .

( ) ( ) (1 )

Assume seeders alone cannot cope with demand:

PDE h

For concreteness, work with d

as a unique equilibriu

etermi

.

nistic j

m:

ob sizes:

.

y

Y

G

F

σ σ

ρ

σ ρ σ

=

>

= − −*

x*

0

*

( ) ( ) (1 ).

x

YF σ ρ σ= − −�����

Nonlinear PDE. Is the equilibrium globally stable?

σ1

x

Corresponds to a uniform density in residual work.

Consistent with queueing results.

Monotone dynamical systems [Hirsch-Smith ’06]

,

' ' ' ' ( )

: , . (0) ( ), 0

(0)

.

.

Banach space ordering defined by closed convex cone

.

Semiflow on Maps

monotone or order preserving if

t

t

Y Y

y y y y Y y y y y Int Y

X X X Y x x t t

x

+

+ +≤ − ∈ − ∈Φ →

⇔ ⇔⊂ → ≥

Φ ≤

�

i

i

i '(0) ( ) '( )

(0) '(0) ( ) '( ) .0

.

strongly monotone if o f

r t

x x t x t

x x x t x t t

⇒ ≤Φ < ⇒ >i �

(Strong) order preservation rules out "recurrent" dynamics. i

,

,

Proposition

Let be an ordered Banach space, and a strongly

monotone semiflow on with orbits of compact closure.

If is open, invariant under and contains a single

equilibrium poin

t all

t

t

Y

X Y

X

p

Φ

⊂Φ

trajectories in converge to . X p

(Strong) order preservation rules out "recurrent" dynamics.

For instance, there can be no stable limit cycles.

i

i

Is our transport dynamics monotone?

( ) ( )Nat

Corresponds to the cone of positive functions.

ural ordering in functions of :

.F F F Fσ σ σ σ≥ ⇔ ≥ ∀� �

( , ) ( , )( , )

F t F tr F

t

σ σλ σ

σ∂ ∂

= +∂ ∂

( )Order is preserved if

is decreasing:

r F

.( , ) ( , )

is decreasing:

F F r F r Fσ σ σ≥ ⇒ ≤ ∀� �

But, precisely invoking the theorem is more technical.

In particular, compactness. Turn to a finite-dimensional version.

0 .( )

( ) Works in particular for processor sharing:x y

r Fx

µ +=

Proposition: r g∂ ∂

0 0.2 0.4 0.6 0.8 1

0( ) ( )x t z t=1( )z t

2( )z t

( )jz t

1( )M

z t−

A spatially discretized dynamics

{ }0 1 1

1

( )

: : 0

, 0, ,

.

1.( )( )

with component-wise orderin

Stat

g.

e spa ce M

M

j j jj

jg z

z z z z

j Mz Mr z z zλ

−

+

⊂

= ≥ ≥ ≥ ≥

= … −= + −

���������

RZ

Z

( )0

( ) - ( ) - ( ): under , if the equilibrium is unique,

it is globally attractive. Covers in particular the case

Theorem

.x y

rx

i ii iii

µ +=

0

1 0

0

,

( ) )., , ( ) 0

0, ( )

( )(

( )

( ))

If is Metzler ( for any

If in additi

then

is irreducon then

If traject

ible

ories ar

e bounded

.

.

j

ij

k

j

j j j

j

r gj k A z a i j

zi

ii

i

zr

A zz

r z z zii zκµ+

∂ ∂≤ ∀ = ≥ ≠

∂ ∂∂

<∂− >∑

Extensions, 1,..., .

( , ) ( , )( , ) ,

is not the same for all . Depends on piece exchange rules.

Preceding theory extends to the

case of "propo

1) upload rateHeter og classeneo e us s

i

i

i ii i

r i

i n

F t F tr F

t

µσ σλ σ

σ

=∂ ∂

= +∂ ∂

i

i0

0( )rtional reciprocity": i i

j

yr F

x

µµ= +

∑j

x∑

( , ) ( )

2) generated by terminating leechers.

State is given by and ; dynamics no longer monotone.

Local analysis around equilibrium. Can show stability using

the small-ga

Dynamics in seeder

n

s,

i

F t y tσi

i

theorem, or a Lyapunov functional.

Can study the noise response studied through linearization,

classical filtering. Matches well with variance in experiments.

i

Outline

1. Networking research: between queues and fluids.

2. M/G- Processor Sharing queues, point process

model and its PDE fluid counterpart.

3. Dynamics of peer-to-peer networks.

4. Dispatching deferrable power loads for frequency

regulation in the power grid.

Real time balancing in the power grid

sec min 5-60 min Hours, day ahead

primaryfreq control

secondary freqcontrol

Reserves markets

• Due to limited storage, electric demand and supply must

tightly match. Various markets are set up with this objective.

Forward markets

tightly match. Various markets are set up with this objective.

• At fast time-scale, balancing is a control engineering problem.

• Fast-responding generators are set up to provide “secondary

frequency regulation” by following a reference signal.

• Can a smarter control of demand help with regulation?

• Idea: exploit deferability of certain loads (e.g., electric

vehicles), schedule to track a desired consumption profile.

• Many references, in particular Poolla’s group in Berkeley.

Deferrable loads as a controlled queue

load

aggregator

( )Poisson load arrivals λ served loads depart

service fraction for load

(fraction of time turned on,

or fraction of nominal power)

: k

u k

:

For the th arrival:

required energy.

service time at

: k

kk

k

Q

Q

pσ

−

=or fraction of nominal power)0

:

nominal power

(spare tlaxit

.

ime).y

k

kl

p

0 :nominal power, assumed

common to all loads.

p

Control decision: choice of for each of loads present

Objective: control aggregate power consumption to a

desired reference. Initially, suppose constant reference.

Constraint: respect

.

deadl

ku•

•

• ines as much as possible.

Point process representation

σ

l •

•

•

•

•

••

• ( ,1 )u u− −

•

[ ] [ ]2( , ) .

0

. ;

Arrivals: a new point mass appears, following the joint

distribution of Let

Service time and laxity are consumed according to fraction

Departures when reachi

.

ng

k k k kl l L

u

σ σ τ

σ

+∈ = =

=

R E E

i

i

i .

0. Misses deadline if it crosses line l =i

Equal sharing: uniform service fraction

[ ]0

,

( )!

Sojourn time:

Poisson arrivals an M/G/ queue,

with stationary Poisson distribution

, w

.ith

Serve all loads present at .

k k

n

kT Tu

N nn u

u

u p

e ρ

τ

ρ λτρ

σ

−

⇒ = =

= =

∞

=

→

P

E

σ

l •

•

••

•

••

•

!n u

[ ]

[ ] [ ]0 0

.

!

, .

Mean # of loads in system:

Consumed power is mean

Independent of Actually this is expected, matches mean demand

.

k

Nu

p Nup p p p Q

u

λτρ

λτ λ

= =

= = =• =

•

E E

E

[ ] [ ] 0. .Closer to constaVar Var nt power as N p up p uρ= ⇒ =• ↓

. .1

Probability of missing deadline: Deteriorates asl

u uu

σ > ↓ − • P

Least-Laxity-First Scheduling Serve fraction of loads

present, at nominal power.

Pick those with smallest l .

axity

u

σ

l •

••

•

•

•

•Fraction

u

( )tθProposition: If

population as in previous case,

exp( ),

k

N

σ µ

λτ

∼

.Poisson Same mean and variance.u

λτ

[ ]

*

*

*

( ).

( ) .

[ ]: , 0

[ ] [ ]

[ ]: , 0

[ ] [ ]

Missed deadlines can be studied through the frontier process

In the large scale limit,

If deadline miss 0

If

.

deadlin

.

k

k k

k

k k

t

t

Eu

E E

Eu

E E

θλ θ θ

ση θ

σ

ση θ

σ

→ ∞ →

> = > ⇒+

< = < ⇒

•

+⇒•

→⇒ P

P

�

�[ ] 1e miss .→

Simulation studies

Standard deviation of power

Least-laxity first.

u η>

Fraction of misseddeadlines.

η

u η<

Alternatives for firm deadlines

σ

l •

••

•

•

•

•( ,1 )u u− −

σ

l ••

•

•

( ,1 )k k

u u− −

"Exact scheduling":

Tailor service level so each

load leaves exactly at deadline.ku

"Laxity expiring scheduling":

Fixed service fraction

for loads with laxity, serve

at full power when it expir

es.

u

Analysis of power variance:

change of variables turns it

into an M/G/ queue.∞

Analysis of power variance:

results available for

exponential jobs/laxities.

Fluid models for service deferral

( ) ( )

Single class, soft deadlines

n t u t

( , , .)

Turn to fluid population models.

The most complete representation would be a PDE in

Start with ODE in load populations, valid in exponential case.

l tσi

i

( ),

What if we want to track a non-constant power reference?

Requires a controlled queue operating outside equilib m. riuu t

Firm deadlines, "laxity expiring"

method, two classes of loads

�

0 .

( ) ( )( ) ( )

( ) ( ) ( )noise

n t u t

t v t

p t p n t u t

n λτ

= − +

=

*

*.Equilibrium: n

u

λτ=

1

2

0

1 1( ) ( ) ( )(1 ( )) ( )

( )

.

( )

1 1( )(1 ( )) ( ) ( )

( ) ( ) ( ) ( )

method, two classes of loads

n t u t n t u t v tL

t

t

n t u t m t v tL

m

p t p n t u t m t

n λτ

τ

− − −

= − −

= +

= +

+

* *

0, independent ofIn both cases: p p uλτ=

2 *

* * * *

* * (1 )

(1 ) (1 ).Equil:

L u

Lu u Lu un m

λτ λτ

τ τ

−

+ − + −= =

Control: tracking a regulation signal

P

v

( )u tδ

( )p tδ C

( )

Referen

ce

r tδ

state

−

( )e t

*( ):

: linearized plant around an operating point.

Controller to track of mean zero (since is fixed).

P

C r t pδ

Results with real

regulation signal

from PJM operator.

2Design: feedforward + state feedback, optimal control.−H

• In various instances, network performance is dictated by the dynamics of populations (jobs, peers, energy loads,…).

• Relevant stochastic queues: M/G – processor sharing, M/G/∞. Point process state. Stationary distribution can be sometimes be found through insensitivity.

• Fluid differential equation models have a wider applicability. To capture general job sizes, a transport PDE is required.

Conclusions

To capture general job sizes, a transport PDE is required.

• Control theory tools apply to analysis (Lyapunov, small gain, monotone systems) or synthesis (H2 regulator, etc.)

• Future work:

– Deferrable energy loads: other policies, decentralized implementation and incentives.

– Processor-sharing in cloud computing systems.

• PDE model and Internet stability conjecture:

– F. Paganini, A.Tang., A. Ferragut, L. Andrew, “Network Stability under Alpha Fair Bandwidth Allocation with General File Size Distribution”, IEEE Trans. on Automatic Control, Vol 57(3), pp. 579-591, 2012.

• Peer-to-peer dynamics:

– A. Ferragut, F. Paganini, “PDE models for population and download progress in P2P networks”, IEEE Trans. on Control of Nwk Sys, 2015.

– “A. Ferragut, F. Paganini, “Queueing analysis of peer-to-peer swarms:

References

– “A. Ferragut, F. Paganini, “Queueing analysis of peer-to-peer swarms: stationary distributions and their scaling limits” Perf. Evaluation, 2015.

– F. Paganini, A. Ferragut, “Monotonicity and global stability in download dynamics of content-sharing networks” Proc. CDC 2014, Los Angeles.

• Deferrable power loads.

– F. Bliman, A. Ferragut, F. Paganini , “Controlling aggregates of deferrable loads for power system regulation”, Proc. ACC 2015, Chicago.

– A. Ferragut, F. Paganini , “Queueing analysis of service deferrals for load management in power systems”, Proc. Allerton Conference 2015.

Thank you!

Questions?Questions?