Introduction to Simulation

Andy WangCIS 5930-03

Computer SystemsPerformance Analysis

2

Simulations• Useful when the system is not available• Good for exploring a large parameter

space• However, simulations often fail

• Need both statistical and programming skills• Can take a long time

Common Mistakes• Inappropriate level of detail

– More details more development time more bugs more time to run

– More details require more knowledge of parameters, which may not be available

• E.g., requested disk sector– Better to start with a less detailed model

• Refine as needed

3

Common Mistakes

Common Mistakes• Improper language

– Simulation languages• Less time for development and statistical

analysis– General-purpose languages

• More portable• Potentially more efficient

4

Common Mistakes• Invalid models

– Need to be confirmed by analytical models, measurements, or intuition

• Improperly handled initial conditions– Should discard initial conditions– Not representative of the system behavior

• Too short simulations– Heavily dependent on initial conditions

5

Common Mistakes• Poor random number generators

– Safer to use well-known ones– Even well-known ones have problems

• Improper selection of seeds– Need to maintain independence among

random number streams– Bad idea to initialize all streams with the

same seed (e.g., zeros)

6

Other Causes of Simulation Analysis

Failure• Inadequate time estimate

– Underestimate the time and effort– Simulation generally takes the longest time

compared to modeling and measurement• Due to debugging and verification

• No achievable goal– Needs to be quantifiable

7

Other Causes of Simulation Analysis

Failure• Incomplete mix of essential skills

– Project leadership– Modeling and statistics– Programming– Knowledge of modeled system

• Inadequate level of user participation– Need periodic meetings with end users

8

Other Causes of Simulation Analysis

Failure• Obsolete or nonexistent documentation• Inability to manage the development of

a large, complex computer program– Needs to keep track of objectives,

requirements, data structures, and program estimates

• Mysterious results– May need more detailed models

9

Terminology• State variables: the variables whose

values define the state of the system – E.g., length of a job queue for a CPU

scheduler

• Event: a change in the system state

10

Terminology

Static and Dynamic Models

• Static model: time is not a variable– E.g., E = mc2

• Dynamic model: system state changes with time– CPU scheduling

11

Continuous and Discrete-time Model

Continuous-time model• System state is defined at

all times

Discrete-time model• System state is defined

only at instants in time

12

Time TimeTuesdays andThursdays

Time spent executing a job

Number of students attending this class

Continuous and Discrete-state Model

Continuous-state model• Use continuous state

variables

Discrete-state model• Use discrete state

variables

13

• Possible to have all four combinations of continuous/discrete time/state models

Time Time

Time spent executing a job

Number of jobs

Deterministic and Probabilistic Model

Deterministic model• Output of a model can be

predicted with certainty

Probabilistic model• Gives a different result for

the same input parameters

14

input input

outputoutput

Linear and Nonlinear Models

Linear model• Output parameters are

linearly correlated with input parameters

Nonlinear model• Otherwise

15

Stable and Unstable Models

Stable model• Settles down to a steady

state

Unstable model• Otherwise

16

Open and Closed ModelsOpen model• Input is external to the

model and is independent of the model

Close model• No external input

17

Computer System Models

• Generally – Continuous time– Discrete state– Probabilistic– Dynamic– Nonlinear

18

Selecting a Language for Simulation

• Simulation language• General-purpose language• Extension of general-purpose language• Simulation package

19

Selecting a Language for Simulation

Simulation Languages• Have built-in facilities

– Time advancing– Event scheduling– Entity manipulation– Random-variate generation– Statistical data collection– Report generation

• Examples: SIMULA, Maisie, ParSEC

20

General-purpose Languages

• C++, Java• No need to learn a new language• Simulation languages may not be

available• More portable• Can be optimized

21

Extensions of General-Purpose Languages

• Provide routines commonly required in simulation

• Examples: CSim, NS-2 (OTcl + C++)

22

Simulation Packages• Provide a library of data structures,

routines, algorithms• Significant time savings

– Can be done in one day• However, not flexible for unforeseen

scenarios

23

Types of Simulations• Emulation

– Hybrid simulation• Monte Carlo simulation• Trace-driven simulation• Discrete-event simulation

24

Types of Simulations

Emulation and Hybrid Simulation

• Emulation– A simulation using hardware/firmware

• Hybrid simulation– A simulation that combines simulation and

hardware– E.g., a 5-disk RAID

• One simulated disk• Four real disks

25

Monte Carlo Simulation• A type of static simulation• Models probabilistic phenomenon• Can be used to evaluate

nonprobabilistic expressions– E.g., use the average of estimates to

evaluate difficult integrals

26

Trace-Driven Simulation

• Trace: a time-ordered record of events on a real system– Needs to be as independent of the

underlying system as possible• Storage-level trace may be specific to the

cache replacement mechanisms above, the working set, the memory size, etc.

27

Advantages of Trace-Driven Simulation

• Credibility• Easy validation

– Just compare measured vs. simulated numbers

• Accurate workload– Preserves the correlation and interferences

effects

28

Advantages of Trace-Driven Simulation

• Less randomness– Deterministic input– Less variance– Fewer number of runs to get good

confidence• Fairer comparison (deterministic input)

– For different alternatives• Similarity to the actual implementation

29

Disadvantages of Trace-Driven Simulation

• Complexity– More detailed simulation to take realistic

trace inputs• Representativeness

– Trace from one system may not be representative of the workload on another system

– Can become obsolete quickly

30

Disadvantages of Trace-Driven Simulation

• Finiteness– A trace of a few minutes may not capture

enough activity• Single point of validation

– Algorithms optimized for one trace may not work for other traces

• Trade-off– Difficult to change workload characteristics

31

Discrete-Event Simulation

• Uses discrete-state model– May use continuous or discrete time values

32

Common Components• Event scheduler

– E.g., schedule event X at time T• Simulation clock • A time-advancing mechanism

– Unit time: Increments time by small increments

– Event-driven: Increments time automatically to the time of the next earliest event

33

Common Components• System state variables• Event routines (handlers)• Input routines

– E.g., number of repetitions• Report generator• Initialization routines

– Beginning of a simulation, iteration, repetition

34

Common Components• Trace routines (for debugging)

– Should have an on/off feature– Snapshot/continue from a snapshot

• Dynamic management• Main program

35

Event-Set Algorithms• How to track events

– Ordered linked list (< 20 events)– Indexed linked list (20 – 120 events)

• Calendar queue– Tree structure (> 120 events)

• E.g., heap

36

37

White Slide