Steady-State Statistical Analysis By Dr. Jason Merrick

Steady-State Statistical Analysis

By Dr. Jason Merrick

Simulation with Arena - Steady-state Output AnalysisC7/2

Warm Up and Run Length

• Most models start empty and idle– Empty: No entities present at time 0– Idle: All resources idle at time 0– In a terminating simulation this is OK if realistic– In a steady-state simulation, though, this can bias the

output for a while after startup• Bias can go either way

• Usually downward (results are biased low) in queueing-type models that eventually get congested

• Depending on model, parameters, and run length, the bias can be very severe


Warm Up and Run Length (cont’d.)

• The period up to 1500 minutes is less congested

• Thus average output measures will be biased down

• How can we get rid of this bias?


Intelligent Initial Conditions

• Collect data– Observe an actual state of the real system that has been

running for a reasonable period of time– Use this state as the initial conditions– Not possible if system does not exist or you are modifying

the system

• Use another model– Queuing models, inventory models etc.– Give steady-state results under more restrictive

assumptions than simulation– Use these results as initial conditions


Warm-up

• Define some time tW until which no statistics are collected

– Suppose mW observations are collected up to time tW

– Suppose m observations are collected after time tW

• The idea is that Ymw+1,…,Ym+mw are drawn from the

“steady state” distribution, while Ym,…,Ymw are

from a different warm-up distribution– So truncating the warm-up observations removes the bias

WWW mmmm YYYY ,.....,,,.... 11


Determining Warm-up Times

• Ensemble averages

– The average across replications of the first, second, third, … observations

– Each ensemble average is an iid sample from the distribution of that observation

– Put t-distribution confidence interval around each average

– See when the ensemble averages settle down

222

21

21

21

222221

111211

,,,

,,,

,,,

,,,

,,,

m

m

nnmnn

m

m

sss

YYY

YYYY

YYYY

YYYY

Series Averages

EnsembleAverages



-20

-15

-10

-5

0

5

10

15

20

25

1 11 21 31 41 51 61 71 81 91

Observations

-8

-6

-4

-2

0

2

4

6

8

1 11 21 31 41 51 61 71 81 91

Ensemble Averages



-100

-50

0

50

100

150

1 11 21 31 41 51 61 71 81 91

Observations

-40

-30

-20

-10

0

10

20

30

40

50

60

1 11 21 31 41 51 61 71 81 91

Ensemble Averages


Truncated Replications

• If you can identify appropriate warm-up and run-length times, just make replications as for terminating simulations– Only difference: Specify Warm-Up Period in Simulate

module

nmmnmn

mmm

mmm

YYY

YYY

YYY

nWW

WW

WW

,,,

,,

,,

,1,

2,21,2

1,11,1

2

1

Y

2s


Batching in a Single Run

• If model warms up very slowly, truncated replications can be costly– Have to “pay” warm-up on each replication– Throw away

W

W

W

mnn

m

m

YY

YY

YY

,1,

,21,2

,11,1

,,,

,,

,,


Batching in a Single Run

• Alternative: Just one R E A L L Y long run– Only have to “pay” warm-up once

– Problem: Have only one “replication” and you need more than that to form a variance estimate (the basic quantity needed for statistical analysis)• Big no-no: Use the individual points within the run as “data” for

variance estimate

• Usually correlated (not indep.), variance estimate biased

mmmm WWWYYYYY ,...,,,, 121

throw away sample


Batching in a Single Run (cont’d.)

• Break each output record from the run into a few large batches ckmkmkmkmkmmm WWWWWWW

YYYYYYYY ,...,,,..,,,...,,,..., 122111

Batch 1 Batch 2Warm-up…...

k

iimW

Yk

Y1

1

1

k

iikmW

Yk

Y1

2

1

k

iikjmj W

Yk

Y1

)1(

1


• The batch means will not actually be independent– The idea is to reduce the correlation to a level that it will not introduce a

significant bias in the estimate of the standard deviation

– The individual observations in the series are correlated with the previous observations

– The correlogram shows that the correlation reduces the higher the lag– So if the batch size is long enough then most of the observations

making up batch 1 will be approximately independent of those making up batch 2

– Only the observations near the end of the batches will be correlated


kmkmkmm WWWWYYYY 211 ,..,,,...,

Batch 1 Batch 2

1Y 2Y



• Rules of thumb– Schmeiser (1982) found that for a given run length, there was little

benefit in more than 30 batches– However, less than 10 batches was too few

– There may well be correlation between all lags, looking at the lag 1 correlation is usually enough to ascertain independence

– Auto-correlation estimates are not very good for sample sizes like 30– Use smaller batches (say c > 100) and if the independence test is

passed then the bigger batches will be fine

cYYY ,,, 21


Examining # of Batches

• Consider the Simple Processing System– t = 1,000,000 minutes, increase number of batches

15

16

17

18

19

20

8 10 15 20 25 30 35 100

# Batches

Av

era

ge

Flo

wti

me

CI too small CI doesn’t change much


Examining Run Length

• Consider the Simple Processing System– n = 25 replications, increase the run length

0

5

10

15

20

25

10 100 1000 10000 100000

Run Length

Av

era

ge

Flo

wti

me


• Consider the Simple Processing System– t = 15 minutes, increase the number of replications

Examining # of Replications

0

1

2

3

4

5

6

7

0 50 100 150 200 250

# replications

Av

era

ge

Flo

wti

me

Documents

Steady-State Statistical Analysis By Dr. Jason Merrick