1

10 May 2004 PADS04 1

Formalization and Strictness of Simulation Event Orderings

Teo Yong Meng1,2 and Bhakti Onggo2

1Singapore-Massachusetts Institute of Technology Alliance2Department of Computer ScienceNational University of Singapore

email: teoym@comp.nus.edu.sgurl: www.comp.nus.edu.sg/~teoym

10 May 2004 PADS04 2

Related Work

Berry and Jefferson (1985) – critical path analysisBagrodia et al. (1991) – Space-Time algorithmBarriga et al. (1995) – incremental benchmarkJha et al. (1996) – ideal simulation protocolBalakrishnan et al. (1997) – workloadFerscha et al. (1997) – NMAP, three layers (model, protocol, platform)Liu et al. (1999) – Dartmouth Scalable Simulation Framework (DaSSF)……..

2

10 May 2004 PADS04 3

Previous Work1. B.S.S. Onggo and Y.M. Teo, Performance Trade-off in Distributed

Simulation, Proceedings of the 6th IEEE International Workshop on Distributed Simulation and Real Time Applications, pp. 77-84, IEEE Computer Society Press, Texas, USA, October 2002.

2. Y.M. Teo, B.S.S Onggo and S.C. Tay, Effect of Event Orderings on Memory Requirement in Parallel Simulation, Proceedings of the 9th International Symposium on Modelling, Analysis and Simulation ofComputer and Telecommunication Systems, pp. 41-48, IEEE Computer Society Press, Cincinnati, USA, August 2001.

3. H. Wang, Y.M. Teo and S.C. Tay, An Analytic Method for Predicting Simulation Parallelism, Proceedings of the 33rd Annual Simulation Symposium, pp. 211-218, IEEE Computer Society Press, Washington D.C., USA, April 2000.

www.comp.nus.edu.sg/~teoym/recent-publications.htm

10 May 2004 PADS04 4

Outline

IntroductionWhy? - MotivationWhat? - Performance FrameworkHow?

Formalization – Event OrderingsCharacterization – Not coveredStrictness of Event OrderingsExperimental Results

Summary

3

10 May 2004 PADS04 5

Motivation

“The World”

A Model

Sequential /Parallel Simulator

SC1

SC2

SC3 SC4 SC6

SC5

LP1 LP3 LP4

LP2

LP6

LP5

PP1 PP2

Phys

ical

Sy

stem

Si

mul

atio

n M

odel

Si

mul

ator

PhysicalSystem

SimulationModel

Simulator /Implementation

10 May 2004 PADS04 6

Parallel Simulation Performance – Layered Approach

SC1

SC2

SC3 SC4 SC6

SC5

LP1 LP3 LP4

LP2

LP6

LP5

PP1 PP2

Phys

ical

Sy

stem

Si

mul

atio

n M

odel

Si

mul

ator

ProblemParallelism

ModelParallelism

EffectiveParallelism

event orderings,

synchronization

protocols, etc.

event level parallelism

4

10 May 2004 PADS04 7

Proposed Framework = Formalization + Characterization

Physical System

Simulation Model

Simulator

Event Orderings

Perf

Perf

Perf

Πprob

Mprob

Πord

Mord

Πsync

Msync, Mtot, Mshr, Mdst

Norm

alization

Perf

strictness (ςR), stricter

Com

parison

Poset

Perf. = performance measure

10 May 2004 PADS04 8

Formalization using Poset

Simulation Model

x ⇒ y, iff (x.ts < y.ts) or (x.ts = y.ts and priority(x) < priority(y)) (Total)

x ⇒ y, iff x.ts < y.ts (Timestamp)

x ⇒ y, if (y.pred = x) or (y.ante = x) or (x.ts + W < y.ts) (Time-interval)

x ⇒ y, if (y.pred = x) or (y.ante = x) (Partial)

x ⇒ y, if (y.pred = x) or (y.ante = x) or ( ⎣x.ts/W⎦ < ⎣y.ts/W⎦ )

x ⇒ y, if (y.pred = x) or (x.ts + la < y.ts)

x ⇒ y, if (y.pred = x) or (x.ts + la < y.ts) or ( ⎣x.ts/W⎦ <⎣y.ts/W⎦ )

x ⇒ y, if y.ante = x

PhysicalSystem

x ⇒ y, iffx.ts < y.ts

Simulator

Sequential

TW Protocol

BTW Protocol

CMB Protocol

BL Protocol

Unsynchronized

5

10 May 2004 PADS04 9

Characterization Time, Space & Strictness

Layers Time Space Strictness

Physical System prob

prob

DE

=Π

∑= <<

=m

ii

Dt

prob tQMprob

1 0)(max

Simulation

Model ordord

DE

=Π ∑= <<

=m

ii

Dt

ord tLMord

1 0)(max

Simulator sync

sync

DE

=Π

∑= <<

=n

ii

Dt

sync tBMsync

1 0)(max

syncordnorm

probtot MMMM ++=

||)}||

||||||(||{max

,

1,,0

ti

m

ititiDt

shr

B

LQM

+

+= ∑=

<≤

||)}||||||

||(||{max

,)1(,)1(

1,)1(

1 0

tjkitjki

k

jtjki

n

i Dt

dst

BL

QM

+−+−

=+−

=<≤

++

= ∑∑

tot

R

SS

=ς

10 May 2004 PADS04 10

Formalization of Simulation Event Orderings

6

10 May 2004 PADS04 11

Related Work (1)

Memory ConsistencyMemory operation orderings - sequential consistency (Lamport, 1979), Weak Ordering (Dubois, 1990), etc.

Broadcast CommunicationMessage orderings - FIFO order, causal order, total order (Hadzilacos and Toueg, 1993), etcImplementation: Birman et al. (1987), Gambhire et al. (2000)

10 May 2004 PADS04 12

Related Work (2)

HLA/Time Management causal order, timestamp, … (Fujimoto and Weatherly, 1996), AT/ATC (Fujimoto, 1999), Causal receive (Zhou et al., 2002)

Discrete-event simulationSimulation event orderings - partial event order, time-interval, timestamp, total event order, etc.Implementation / Simulator: Sequential (Banks et al., 2000), Time Warp (Jefferson, 1985), CMB (Chandy et al., 1979), Bounded Lag (Lubachesky, 1989), Bounded Time Warp (Turner, 1992), etc.

7

10 May 2004 PADS04 13

Partially Ordered Set (Poset)

DefinitionA poset is a tuple (S, R) where S is a set and R is a partial order on the set S. R must be anti-reflexive, anti-symmetric and transitive.

Types of Orderings:Partial order (Dushnik, 1941)Total order (Dushnik, 1941)Interval order (Fishburn, 1988)Tolerance order (Bogart, 1995)Semi-order (Pirlot, 1997)Split semi-order (Fishburn, 1999)……………

10 May 2004 PADS04 14

Simulation Event Orderings

Events x and y are comparable if x must be ordered before y or vice-versa. Otherwise, they are concurrent (non-comparable).

DefinitionA simulation event ordering is a tuple (E, SR) where E is a set of events and SR is a set of comparable events based on event order R. Event order R must be anti-reflexive, anti-symmetric, and transitive.

8

10 May 2004 PADS04 15

Event Order: Physical System

A physical system is viewed as a network of service centers.

Events in a physical system occur in a physical time order.

DefinitionLet x be an event in a physical system and x.tthe physical time when event x happens. The event order in a physical system dictates that for all x and y (where x ≠ y), x is ordered before y if and only if x.t < y.t.

10 May 2004 PADS04 16

Event Order: Simulation Model

Virtual time paradigmeach service center is modeled as a logical process (LP),an event in the physical system is modeled as a simulation event, and the physical time when it occurs is modeled as a simulation time (or timestamp)

Events can be executed/simulated in different orderings:

Total event order - event x is ordered before event y in iff (x.ts < y.ts) or (x.ts = y.ts and priority(x) < priority(y)).Partial event order - event x is ordered before event y in if (y.pred = x) or (y.ante = x). ………

9

10 May 2004 PADS04 17

Event Order: ImplementationA (sequential, parallel, distributed) simulator maintains a specific event ordering during runtime.

Sequential SimulatorEvents are sorted in a timestamp order using a Future Event ListEvent with the smallest timestamp is executed, in case of a tie, event with higher priority will be executed →total event order

Time Warp SimulatorRollback ensures event with the smallest timestamp in an LP is committed first (y.pred = x).Aggressive cancellation by sending ant-messages ensure that an antecedent will be committed first (y.ante = x).

10 May 2004 PADS04 18

Formalization based on Event Orderings - Summary

S1

S2

S3

S4

01a 2

2a 41d 6

6a 92d 12

6d

55a 7

3d 87a 10

9a 1111a 13

12a 1413a

44a 7

4d 1010a 13

10d

108d

Timestamp

43a

0 2 4 6 8 10

Serv

ice

Cen

ter

S1

S2

S3

S4

a)

b)

01a 2

2a 41d 6

6a 92d 12

6d

55a 7

3d 87a 10

9a 1111a 13

12a 1413a

44a 7

4d 1010a 13

10d

108d

43a

LP

LP

LP

LP

01a 2

2a 41d 6

6a 92d 12

6d

Timestep0 1 2 3 4 5 6 7

Logi

cal P

roce

ss

43a

44a

74d 10

10a 1310d

88a8

8a 108d10

8d

1413a13

12a

1111a

109a8

7a

73d5

5a73d 8

7a

Problem/ Physical System

Simulation Model – Partial Event Order

Implementation – Time Warp

10

10 May 2004 PADS04 19

Formalization - Summary

Simulation Model

x ⇒ y, iff (x.ts < y.ts) or (x.ts = y.ts and priority(x) < priority(y)) (Total)

x ⇒ y, iff x.ts < y.ts (Timestamp)

x ⇒ y, if (y.pred = x) or (y.ante = x) or (x.ts + W < y.ts) (Time-interval)

x ⇒ y, if (y.pred = x) or (y.ante = x) (Partial)

x ⇒ y, if (y.pred = x) or (y.ante = x) or ( ⎣x.ts/W⎦ < ⎣y.ts/W⎦ )

x ⇒ y, if (y.pred = x) or (x.ts + la < y.ts)

x ⇒ y, if (y.pred = x) or (x.ts + la < y.ts) or ( ⎣x.ts/W⎦ <⎣y.ts/W⎦ )

x ⇒ y, if y.ante = x

PhysicalSystem

x ⇒ y, iffx.ts < y.ts

Simulator

Sequential

TW Protocol

BTW Protocol

CMB Protocol

BL Protocol

Unsynchronized

10 May 2004 PADS04 20

Strictness Analysis

11

10 May 2004 PADS04 21

Strictness of Event Orderings

Why?To compare event dependencies of different event orderingsTo quantify the degree of event dependenciesamong different event ordersTime independent

How?Stricter relationStrictness measure (ςR)

01a 2

2a 41d 6

6a 92d 12

6d

55a 7

3d 87a 10

9a 1111a 13

12a 1413a

44a 7

4d 1010a 13

10d

108d

43a

10 May 2004 PADS04 22

Stricter RelationDefinition.

Let (E, SR1) and (E, SR2) be two event orderings on the same set of events E. Event order R1 is stricter than R2 if for any E, SR2 ⊆ SR1. An event order R1 is incomparableto event order R2 if we can find two sets of events E1 and E2, such SR2 ⊆ SR1 is true for E1 but SR2 ⊆ SR1 is not true for E2.

Lemma 3.1. The properties of a stricter relation are:if R1 is stricter than R2 and R2 is stricter than R1, then R1 = R2 (anti-symmetric).if R1 is stricter than R2 and R2 is stricter than R3, then R1 is stricter than R3 (transitive).

12

10 May 2004 PADS04 23

Strictness

The strictness of an event order R ( ςR) is defined as

where and is the size of the set of comparable (or non-concurrent) events ordered by R and the total event order respectively.

Strictness of an event ordering ranges from 0 when SR = ∅, and 1 when R is the total event order.

tot

R

SS

RS totS

10 May 2004 PADS04 24

Strictness Example

Set of events E = { 103a , 11

4a , 115a , 12

2d , 136a , 13

3d , 147a ,

154d , 15

5d , 168a }, hence ||E|| = 10.

|| Stot || = ||E|| × (||E||-1) / 2 = 45

SR = {( 114a , 13

6a ), ( 136a , 15

4d ), ( 114a , 15

4d ), ( 103a , 12

2d ), ( 122d , 13

3d ),

( 133d , 16

8a ), ( 103a , 13

3d ), ( 103a , 16

8a ), ( 122d , 16

8a ), ( 122d , 13

6a ),

( 103a , 13

6a ), ( 103a , 15

4d ), ( 122d , 15

4d ), ( 133d , 14

7a ), ( 103a , 14

7a ),

( 122d , 14

7a ), ( 103a , 15

5d ), ( 122d , 15

5d ), ( 133d , 15

5d ), ( 115a , 14

7a ),

( 147a , 15

5d ), ( 155d , 16

8a ), ( 115a , 15

5d ), ( 115a , 16

8a ), ( 147a , 16

8a )},

hence || SR || = 25

ςR = || SR || / || Stot || = 0.56

13

10 May 2004 PADS04 25

Comparison of Strictness of Event Orderings

Tot : Total CMB : Chandy-Misra-Bryant Partial : Partial TS : Timestamp BL : Bounded Lag Unsync : Unsynchronized TI : Time-interval BTW : Bounded Time Warp

TOT

BL

TS

TI

BTW

CMB

Unsync

Causality Line Maintains Causality

Ignores Causality

Sequential Simulator

Parallel Simulators

Totally Unordered

Partial

10 May 2004 PADS04 26

Instrumentation

14

10 May 2004 PADS04 27

Benchmarks

Open system - MIN(n×n, ρ) [Teo&Tay95]Closed system - PHOLD(n×n, m) [Fujimoto90]

10 May 2004 PADS04 28

Strictness Analysis (1)

MIN (n, 0.8)

0.0

0.2

0.4

0.6

0.8

1.0

n=8 n=16 n=24 n=32

Problem Size

Str

ictn

ess

TotalTSTI(1)TI(2)CMBPartial

15

10 May 2004 PADS04 29

Strictness Analysis (2)

PHOLD (n, 4)

0.0

0.2

0.4

0.6

0.8

1.0

n=8 n=16 n=24 n=32

Problem Size

Str

ictn

ess

TotalTSTI(1)TI(2)CMBPartial

10 May 2004 PADS04 30

Strictness Analysis (3)

0.00.10.20.30.40.50.60.70.80.91.0

MIN(8x8, 0.8) PHOLD(8x8, 4)

Benchmarks

Stric

tnes

s Physical SystemCMB event orderCMB protocol (4PPs)CMB protocol (8PPs)

16

10 May 2004 PADS04 31

Summary

A unified simulation performance framework based on event orderings.

Formalization of event orderings based on poset

Performance characterizationtime (event parallelism)space (memory usage)event dependencies- stricter relation, strictness (ςR)

10 May 2004 PADS04 32

Ongoing Work

Parallel simulation -> performance & capability

Performance – scalability (fixed workload (Amdahl’s law ‘67), scaled workload (Gustafson’s Law ‘87), memory bounded (Sun & Ni ’93), ….)

Capability -> framework for understanding performance of grid-enabled simulation

17

10 May 2004 PADS04 33

Forthcoming …

“A Framework for Formalization and Characterization of Simulation Performance”,B.S.S. Onggo, PhD Thesis, Department of Computer Science, National University of Singapore, 2004.

10 May 2004 PADS04 34

Thank You

www.comp.nus.edu.sg/~teoym/recent-publications.htm