Numerical Methods for Stochastic Networks

Peter W. GlynnInstitute for Computational and Mathematical Engineering

Management Science and Engineering Stanford University

Based on joint work with Jose Blanchet, Henry Lam, Denis Saure, and Assaf Zeevi

Presented at Stochastic Networks Conference, Cambridge, UKMarch 23, 2010

Stochastic models:

Descriptive

Prescriptive

Predictive

Today’s Talk:

• an LP based algorithm for computing the stationary distribution of RBM (Saure / Zeevi)

• a Lyapunov bound for stationary expectations (Zeevi)

• Rare-event simulation for many-server queues (Blanchet / Lam)

Computing Steady-State Distributions for Markov Chains

One Approach

An LP Alternative

( ) ( )n

nx K

x h x c

( ) ("tightness")inf ( )c c

n n

nx K x K

cx

h x

• Where does constraint

come from?

• We assume that we can obtain a computable bound on

i.e.

• This will come from Lyapunov bounds (later).

( ) ( )n

nx K

x h x c

( )h XE( )h X c E

Application to RBM

Reflected Brownian Motion (RBM):

For some stochastic models, the LP algorithm is particularly natural and powerful

dR

( :1 , )ijb i j d

d ( ) d d ( ) d ( )X t t B t R Y t

jY 0jX

2

1 , 1

1, =

2

d d

i ij j iji i j ii i j i

b Rx x x x

DL

1int ( )

int ( )

min

s/t ( )( ) ( )( ) u

1, 0, 0

| |d

k jk

dk

d

k i k i i k jkj x Fx

k k jkx

u

p f x f x

p p

R

R

L D

(d )( )( )d

x f x c

R L

(d )( )( ).j

j jFx f x D

j

Theorem: This algorithm converges as , in the sense that

as .

,m n

n

n

j

n

Numerical Results

Smoothed marginal distribution estimates for the two-dimensional diffusion. The dotted line is computed via Monte Carlo simulation, and the solid line represents the algorithm estimates based on n = 50 and m = 4, incorporating smoothness constraints.

• - valued Markov process with cadlag paths

• We say if there exists such that

is a –local martingale for each

( ( ) : 0)X X t t S

( )g AD k

0( ) ( ( )) ( ( ))d

tM t g X t k X s s

xP x S

0( ( )) ( ) ( ( ))d

h

x xg X h g x k X s s E E

Computing Bounds on Stationary Expectations

• Main Theorem:

Suppose is non–negative and satisfies

Then,

( )g AD

sup( )( ) .x S

Ag x

(d )( )( ) 0.S

x Ag x

Diffusion Upper Bound

• Suppose is and satisfies for , where and

where . Then,

d ( ) ( ( ))d ( ( ))d ( )X t X t t X t B t

0g 2C ( )( ) ( )g x h x c Ldx R 0h

2

1 ,

1( ) ( )

2

d

i iji i ji i j

x b xx x x

L

T( ) ( ) ( )b x x x

.h c

Many-Server Loss Systems

Many-Server Asymptotic Regime

Simplify our model (temporarily):

• using slotted time• eliminate Markov modulation• discrete service time distributions with finite

support

Consider equilibrium fraction of customers lost in the network.

Key Idea:

Many server loss systems behave identically

to infinite-server systems up to the time of

the first loss.

Step 1:

Step 2:

Step 3:

Step 3:

Step 3:

Crude Monte Carlo : 3.7 days

I.S. : A few seconds

Network Extension

• Estimate loss at a particular station

• If the most likely path to overflow a given station does not involve upstream stations, previous algorithm if efficient

• If an upstream station does hit its capacity constraint, we have “constrained Poisson statistics” that need to be sampled

Questions?