Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Presen
Efficient Simulation of Network Performance by
Importance Sampling
Poul E. Heegaard
Norwegian University of Science and Technology Trondheim, Norway
Outline
• MotivationSYSTEM SIMULATION MODEL
9
2
45
6
781
• Main contributions
• Rare event simulation
• Importance sampling
• Adaptive parameter biasing
• Network simulations
• Closing remarks
tation of thesis, 1998-06-09 1
NTNU - Dept. of Telematics
Presen
Motivation
Task: evaluation of system which are characterized by
- being large and complex
- being distributed with tight logical couplings
- have strict quality of service
SYSTEM SIMULATION MODEL
9
2
45
6
78
1
Evaluation means:
- analytic (efficient, but needs oversimplification)
- simulation (flexible, but inefficient)
- measurements (efficient, but expensive and inflexible)
tation of thesis, 1998-06-09 2
NTNU - Dept. of Telematics
Presen
Motivation
=> Simulation is flexible means for system evaluation,
=> Speed-up is required,
=> Importance sampling is efficient,
=> Optimal/good parameters are essential.
1e-09
2e-09
3e-09
4e-09
5e-09
6e-09
Optimal change of measure
prop
erty
of
inte
rest
Simulation results
Exact, = 4.90e-09
Error bars
Insensitive to changes in f*(s)
change of measure
This thesis:
Can importance sampling be applied in network simulations?
tation of thesis, 1998-06-09 3
NTNU - Dept. of Telematics
Presen
Why is network simulation with rare events a problem?
3
5
4
6
2
110
9
23
4
5
6
78
1
0.1
1
10
100
1000
1e-07 1e-06 1e-05 0.0001 0.001 0.01 0.1 1
Spe
ed-
up g
ain
[lo
g]
Exact blocking [log]
Domain of interest (<1e-06)!
users
link failures
Restrictive quality of service requirements
- model size and complexity
- direct simulation of properties
is very inefficientdependent on rare events
tation of thesis, 1998-06-09 4
NTNU - Dept. of Telematics
Presen
Main contributions
- Adaptive biasing for importance sampling in perfor-mance simulations of networks
- Flexible modelling framework
- Network simulation for feasibility demonstrations
1e-11
1e-10
1e-09
1e-08
1e-07
0 1 2 3 4 5 6 7 8 9 10generators
simulation results
exact results
bloc
king
pro
babi
litie
s
with error bars
- Heuristics importance sampling experiments
- Combination of speed-up techniques
- Comparisons of rare event techniques
- Application of importance sampling to MPEG and ATM
tation of thesis, 1998-06-09 5
NTNU - Dept. of Telematics
Presen
Speed-up techniques
SYSTEM SIMULATION MODEL
9
2
45
6
781
PARALLEL AND DISTRIBUTED SIMULATION
Parallel and distributed
1
1 N
N
2 N
1
Parallel independent replicated simulation
SPEED-UPS?
less than N
close to N
HYBRID TECHNIQUES COMBINES ANALYTIC SOLUTIONS WITH SIMULAT- decomposition in time or space
- conditional sampling
T1 T2T3 T4 T5
X1
X0 | X1=x
x-1xx+1
RARE EVENT PROVOKING TECHNIQUEVARIANCE MINIMIZATION EXPLOITS
-0.5
0
0.5
1
1.5
2
2.5
0 2 4 6 8 10 12 14 16
X a
nd Y
YX
KNOWN OR INTRODUCED CORRELATION
- control variable
RESTART with 4 levels
Original distribution
1
10-2
10-4
10-6
10-8
5 10 15 20
Impo
rtan
ce s
amp
ling
CHANGE THE SAMPLING DISTRIBUTIO
- antithetic variates - common random
simulation (PADS)
number
tation of thesis, 1998-06-09 6
NTNU - Dept. of Telematics
Presen
Rare event simulation techniques
Objective:
Change the underlying sampling distribution to provoke rare events of interest to occur more often.
Known approaches:
- RESTART
- Importance sampling
tation of thesis, 1998-06-09 7
NTNU - Dept. of Telematics
Presen
5
1
5
2
5
3
5
Importance sampling
2 4 6 8 10 12 14 x
f(X)=original model
y
f*(X)=stressed model Rare event:Pf g X( ) 1=( ) 1«
Importance sampling:Pf g X( ) 1=( ) P
fg X( ) 1=( )«
Observation: g X( ) I x y( )= , e.g.:
-> overflow of MPEG cells in a multiplexer
-> blocked calls
Relation: Ef g X( )( ) Ef g X( ) X( )( )= where
X( )f X( )f X( )------------= the likelihood ratio between f X( ) and f X( )
Estimator: X
1n--- Xi( ) g Xi( )
i 1=
n
=
where Xi are samples from f x( )
tation of thesis, 1998-06-09 8
NTNU - Dept. of Telematics
Presen
Importance sampling heuristics (1)
- Use likelihood ratio as indication of goodness of simu-lation results
- no analytic solution available
- no direct simulation results
- use knowledge of E L( ) 1=
- Two observations from experiments:
(i) L 1 and rel.error L( ) 1« => ̂IS is good if its relative error rel.error ̂IS( ) 1«
(ii) L 1« or rel.error L( ) 1 :
=> ̂IS is poor even if the rel.error ̂IS( ) 1« .
tation of thesis, 1998-06-09 9
NTNU - Dept. of Telematics
Presen
Importance sampling heuristics (2)
- The sampling distribution is heavy tailed under too strong biasing (infinite variance)
R
running mean of IS
true value E[]
^
tation of thesis, 1998-06-09 10
NTNU - Dept. of Telematics
Presen
Modelling framework
3
5
4
6
2
110
9
23
4
5
6
78
1
user
link failures
Model feasibilities- different resource requirements- different quality of service requirements- pre-emptive priorities- alternative routing on overload and failures
Objectives: assessment of- blocking probability- rerouting probability- disconnection probability- consequence of pre-emption- consequence of failures
priority=1
priority=2
primary route user type B
secondary route user type Aprimary route
type A
usertype B
user type A
tation of thesis, 1998-06-09 11
NTNU - Dept. of Telematics
Presen
Mapping to state space model (1)
3
5
4
6
2
110
9
23
4
5
6
78
1
2
1
2
61
5
10
3
Resource Generator
1
2
pool 103
2
tation of thesis, 1998-06-09 12
NTNU - Dept. of Telematics
Presen
Mapping to state space model (2)
blocking pool 10 =>
generator 2
blocking pool 3
blocking pool 2
gene
rato
r 1
Resource Generator
1
2
pool 103
2
ARGET SUBSPACE 10
0,0 0,1 0,2 0,3 0,4
1,0 1,1 1,2 1,3 1,4
2,0 2,1 2,2 2,3
3,0 3,1 3,2
current state
tation of thesis, 1998-06-09 13
NTNU - Dept. of Telematics
Presen
#entities gen. 1,#entities gen. 2
arrival of entity from gen. 2
departure of entity from gen. 2
TARGET SUBSPACE(resource constraint,
boundary, barrier
STATE
EVENTS
State space model
tation of thesis, 1998-06-09 14
NTNU - Dept. of Telematics
Presen
Importance sampling in network simulations
Challenges
STATE SPACE MODEL
3
10
2
BARRIER
Gen
1
Gen 1
Gen 2
3
210
Gen 2
• Multidimensional state models
• Balanced dimensioning, i.e. no bottlenecks
Previous solutions to change of measure
• Identify a bottleneck
3
10
2
Gen
1
Identify the bottleneckand change the measureaccording to this • Drift towards the bottle-
neck barrier
• Fixed change of measure
• If no single bottleneck => inefficient solution!
tation of thesis, 1998-06-09 15
NTNU - Dept. of Telematics
Presen
Adaptive change of measure
3
10
2
Gen
1
3
10
2
Gen
1
3 2 10
importance
3
10
2
Gen
1
3 2 10
importance
3 2 10
importance
Choose a barrier directionat random, and change the measure toward this.
Approached barrier 3:Choose the barrier directionagain according to the newrelative importance estimates
Approached barrier 10:Choose the barrier directionagain according to the newrelative importance estimates
A new, adaptive approach
• Towards the most important barrier at current state.
• State dependent change.
• A good estimate on the “current importance of all barrier” is required.
Algorithm
At each state:
(i) Estimate the currentimportance of all barriers
(ii) Choose a direction
(iii) Induce drift in the chosen direction.
Step (i)-(iii) are repeated for every state.
tation of thesis, 1998-06-09 16
NTNU - Dept. of Telematics
Presen
Estimation of target importance
- Requirements:
1. Sufficiently accurate
2. Robust
3. Efficient
- Simplification:
- only the relative importance is of interest
- use the greatest importance contribution, Hj1
˜( )
=> must identify the sub-path from current state ˜
toa state in the target sub-space
˜j .
tation of thesis, 1998-06-09 17
NTNU - Dept. of Telematics
Presen
Efficient search for the sub-path
- find the sub-path with the largest contribution to Hj ˜
( )
- exploit the Markov properties
0,0 1,0 2,0
0,1
0,2 1,2 2.2 3,2
3,12,1
4,2
5,0
5,1
0,0 1,0 2,0 2,1 3,1 3,2 4,2 5,2
5,2
x = 0 1 2 4 5 7 8 9
x ck1j– xx ck2j–
k1˜
x ck1j– 1˜
k1+( )
k2˜
x ck2j– 1˜
k2+( )
k1˜
x ck1j– ( )
k2˜
x ck2j– ( )
number of resources allocated
tation of thesis, 1998-06-09 18
NTNU - Dept. of Telematics
Presen
Network example 1
- No priority nor alternative routing
5
3
4
6
2
1
10
3
9
412
2
7
1
711
8 5
86
5
1e-11
1e-10
1e-09
1e-08
1e-07
0 1 2 3 4 5 6 7 8 9 10
generators
simulation results with error bars
exact results
blo
cki
ng
pro
bab
ilit
ies
- Compared with exact results
- Simulation more efficient than numerical calculations
tation of thesis, 1998-06-09 19
NTNU - Dept. of Telematics
Presen
Network example 2
- Improving the quality of service by rerouting
3
5
4
6
2
110
9
2
3
4
5
6
78
1
blo
ck
ing
pro
ba
bil
itie
s
Upper bounds of blocking
1e-10
1e-09
1e-08
1e-07
1e-06
1e-05
1e-04
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Simulated blocking probability of generator 5
(a) With primary route only
generators
- Simulation results close to rough blocking estimates
- Mean likelihood ratio close to 1 with low relative error
- Significant speedup over direct simulation observed
tation of thesis, 1998-06-09 20
NTNU - Dept. of Telematics
Presen
Network example 3
- Disturbing low priority traffic
1e-07
1e-06
1e-05
1e-04
1e-03
bloc
king
pro
babi
lity CASE 3.1:
low priority
CASE 3.2:mixed with
CASE 3.3:mixed with
high priority traffic high priority trafficand exposed to
gen.23 gen.31 gen.23 gen.23gen.31 gen.31
traffic only
link failures
- Best results for generator 23 and 31 in accordance to the biasing setup of importance sampling
- No speed-up compared to direct simulation (loss prob-ability in order of 10 4– .
- The mean likelihood ratio is 0.742 for case 3.3 => overbiased?
tation of thesis, 1998-06-09 21
NTNU - Dept. of Telematics
Presen
Closing remarks
- importance sampling in performance simulation of tel-ecom networks with:
- balanced utilisation of resources
- users with different quality of service requirements
- preemptive priority and rerouting
- link and node failures
- new, adaptive importance sampling biasing proposed
- flexible modelling framework applied
- feasibility demonstrated
- heuristics for importance sampling experiments
- combination of speed-up techniques
- importance sampling in other applications
- further development of fundament required before inclusion of rare event techniques in simulation tool
tation of thesis, 1998-06-09 22
NTNU - Dept. of Telematics