Parallel and distributed Simulation

Parallel and Distributed Simulation

SS 2015

Adelinde Uhrmacher

Modeling and Simulation Research Group

University of Rostock

Background and Applications

Adelinde M. Uhrmacher University of Rostock

2

Introduction & Motivation

What is parallel / distributed simulation?

Why are we interested in this subject?

Virtual environments vs. analytic simulations

Historical Perspective

Literature: Richard Fujimoto: Parallel and distributed Simulation Systems, John Wiley, New York, 2000.

Transparencies are partly taken from a lecture given by Richard Fujimoto at the Georgia Institute of Technology and a tutorial at the Parallel, distributed Simulation Conference, Bologna, Italy, June 2000.


3


Simulation - a system that represents or emulates the behavior of another system over time; a computer simulation is one where the system doing the emulating is a computer program

Parallel (distributed) simulation refers to the technology concerned with executing computer simulations over computing systems containing multiple processors

Tightly coupled multiprocessor systems Workstations interconnected via a network (e.g., the Internet)


4


Reduced model execution time Up to n-fold reduction using n CPUs

Scalable performance Maintaining the same execution speed for bigger

models/virtual environments by using more CPUs

Geographically distributed users and/or resources (e.g., databases, specialized equipment) Co-location is expensive! May be impractical

Integrate simulations running on different platforms Network perspective rather than to export them to a

platform

Fault tolerance Not as easy as it might seem!


5

Enable Simulation of Big Models

Cell level simulation of an ATM (packet) network Simulate one hour of network operation Network with 1000 links 155 Mbits/second links @ 20% utilization 53 byte packets (cells) One simulator event per cell transmission (link) 500 k events / second simulator speed

150 hours for a single simulation run! Larger, more complex networks?

Next generation Internet: Million nodes Higher link bandwidths


6

Fast Execution: On-Line Decision Aids

Simulation tool is used for fast analysis of alternate courses of action in time critical situations

Initialize simulation from situation database Faster-than-real-time execution to evaluate effect of decisions

Applications: air traffic control, battle management Simulation results may be needed in only seconds

live data feeds

analysts and decision makers

forecasting tool (fast simulation)

situation database

interactive simulation

environment


7

Introduction & Motivation



Virtual environments vs. analytical simulations



8

Virtual Environments

Uses: training (e.g., military, medicine, emergency planning), entertainment, social interaction? Simulations are often used in virtual environments to create dynamic computer generated entities

Adversaries and helpers in video games Defense: Computer generated forces (CGF)

Automated forces Semi-automated forces

Physical phenomena Trajectory of projectiles Buildings "blowing up" Environmental effects

(e.g., rain washing out terrain)


9

System Analysis

"Classical" application of simulation; here, focus on "discrete event" simulation

Telecommunication networks Transportation systems Electronic systems (e.g., microelectronics, computer systems) Battlefield simulations (blue army vs. red army) Ecological systems Manufacturing systems Logistics Cell biological systems

Focus typically on planning, system design, and understanding


10

Virtual Environments vs. Analysis

Typical Characteristics

Virtual environment Analysis typical objective

execution pacing

human interaction

accuracy

quantitative analysis of complex systems

create realistic or enter- taining representation

as-fast-as- possible

if included, often external observer

real-time (paced, syn. Simulation-wall clock time)

integral to controlling entities

statistically correct results

human perception often plays a large role

A major issue

Validity, reproducibility plausibility, natural touch and feel


11

Principal Application Domains

Parallel Discrete Event Simulation (PDES)

Discrete event simulation to analyze systems Fast model execution (as-fast-as-possible) Produce same results as a sequential execution Typical applications

Telecommunication networks Computer systems Transportation systems Military strategy and tactics

Distributed Virtual Environments (DVEs)

Networked interactive, immersive environments Scalable, real-time performance Create virtual worlds that appear realistic Typical applications

Training Entertainment Social interaction


12


Chandy/Misra/Bryant algorithm

Time Warp algorithm early experimental

data

second generation algorithms

making it fast and easy to use

High Performance Computing Community

1975 1980 1985 1990 1995 2000

SIMulator NETworking (SIMNET) (1983-1990)

Distributed Interactive Simulation (DIS) Aggregate Level Simulation Protocol (ALSP)

(1990 - 1997ish)

High Level Architecture (1996 - today)

Defense Community

Adventure (Xerox PARC)

Dungeons and Dragons Board Games

Multi-User Dungeon (MUD) Games

Internet & Gaming Community

Multi-User Video Games

Approximative Multi-resolution approaches

Grid-Based approaches

Service-oriented approaches

Cloud computing

2005 2010


13

Parallel / Distributed Simulation Today

High Performance Computing Community After a slow start, the technology is beginning to be embraced because of efforts such as the High Level Architecture

Defense Community Technology has been fully embraced

Training War gaming Test & evaluation

Gaming Community Technology becoming heavily used Server-based systems Internet gaming


14

Summary

Several reasons to execute simulations over multiple computers

Performance Geographical distribution Easier integration of systems (interoperability), reuse

Virtual environment vs. system analysis Have different requirements, sometimes resulting in use of different approaches and techniques Developed from largely disjoint communities

Research and development communities High performance computing Defense Internet and gaming

Adelinde M. Uhrmacher University of Rostock Parallel and Distributed Simulation SS

2006

15

Outline

Hardware platforms

Parallel computers Multi-Core machines

Distributed computers

Simulation fundamentals

State Time Time flow mechanisms

Adelinde M. Uhrmacher University of Rostock Parallel and Distributed Simulation SS

2006

16

Parallel and Distributed Computers

Parallel computers (tightly coupled processors) Shared memory multiprocessors Distributed memory multicomputers Multi-core PCs

Distributed computers (loosely coupled processors) Networked workstations

Parallel Computers Distributed Computers

Comm. Latency (small messages)

A few to tens of microseconds

Hundreds of microseconds to seconds

Processors Machine room/office Building, city, global

Physical extent Homogeneous Often heterogeneous

Comm.Network Custom switch Commercial LAN / WAN


17

Shared Memory Multiprocessors

Programming model: shared variables; synchronization via locks

CPU

cache

memory . . .

CPU

cache

memory

Examples: SUN Enterprise, SGI Origin

{ shared int i;L Lock(L) i = i +1; Unlock(L) }

Processor 1

{shared int i;L Lock(L) i = i +1; Unlock(L) }

Processor 2

CPU

cache . . .

I/O devices

Interconnection network


Multi-CPU, Multi-Core PCs

While Moore's Law, which predicts exponentially increasing microelectronic circuit density over time, is expected to continue into the next decade, it is clear that further hardware performance increases will result primarily from multi-core circuits that rely on parallel processing rather than improved clock rates While the need to simulate larger, more complex systems will continue into the foreseeable future, our ability to simulate such systems with sequential software will be diminishing in the years ahead. Long a nice to have capability, parallel simulation will increasingly become a must have. However, many major hurdles remain before parallel execution can become widely exploited. Development of scalable parallel simulation software, easy to use synchronization techniques, effective load balancing and resource allocation approaches, automated fault tolerance, and time shared use of computing platforms each by itself presents a formidable challenge, yet must all become straightforward or automatic if widespread adoption of parallel simulation methods is to be successful. Richard Fujimoto, Modeling Simulation and Parallel Computation, the Future is now! Springsim 2007

18


19

Distributed Memory Multicomputers

Programming model: no shared variables; message passing

CPU

cache

Interconnection network

memory

. . . communications

controller

CPU

cache memory

communications controller

Examples: IBM SP, Intel Paragon

{int i; Send (2, &i, sizeof(int)) }

Processor 1 {int j; Receive (2, &j, sizeof(int)) }

Processor 2


20

Parallel and Distributed Simulation

Parallel simulation involves the execution of a single simulation program on a collection of tightly coupled processors (e.g., a shared memory multiprocessor).

M M M

P P P P

simulation model

parallel processor

Distributed simulation involves the execution of a single simulation program on a collection of loosely coupled processors (e.g., PCs interconnected by a LAN or WAN).

Replicated trials involves the execution of several, independent simulations concurrently on different processors


21

Outline

Hardware platforms

Parallel computers

Distributed computers

Simulation fundamentals

State Time Time flow mechanisms


22

Experiment

Job 1

Job n

Replication 1 (Job 1)

Replication m (Job 1)

Replication 1 (Job n)

Replication m (Job n)

Analysis (Rep 1 , Job 1)

Analysis (Rep m, Job 1)

Analysis (Rep 1 , Job n)

Analysis (Rep m, Job n)

Analysis Job 1

Analysis Job n

Analysis Experiment

Distributed / sequential simulation algorithm

(Remote / Distrib.) data sink w.

seq./par. access

Iterative / Upon completion of all

replications

Iterative / Upon completion of all

jobs

Iterative / Upon completion of

replication

Parallel jobs Parallel replications Parallel analysis Parallel job

analysis

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Feedback (jobs )

Multi CPU, Multi Core, Cluster, Grid, Cloud, ...

SOA, Web Services, RMI, MPI, ...

HLA, Data bases, ...

Distributed model

Cancellation (rep 1, job 1)

Partitioning / Load balancing Job distribution


23

Simulation Fundamentals

A simulation is an experiment with a model in this case performed on a computer. Please note that in the parallel distributed community often no clear separation between formally described model and algorithm that executes this model does exist.

Program variables (state variables) represent the current state of the physical system

Simulation program modifies state variables to model the evolution of the physical system over time


24

Time

physical system: the actual or imagined system being modeled simulation: a system that emulates the behavior of a physical system

physical time: time in the physical system Noon, December 31, 1999 to noon January 1, 2000

simulation time: representation of physical time within the simulation

Floating point values in interval [0.0, 24.0]

wallclock time: time during the execution of the simulation, usually output from a hardware clock

9:00 to 9:15 AM on September 10, 1999

12

6

11

9 3

10

8 7

1 2

4 5

main() { ... double clock; ... }

physical system simulation


25

Simulation Time

Simulation time is defined as a totally ordered set of values where each value represents an instant of time in the physical system being modeled.

For any two values of simulation time T1 representing instant P1, and T2 representing P2:

Correct ordering of time instants If T1 < T2, then P1 occurs before P2 9.0 represents 9 PM, 10.5 represents 10:30 PM

Correct representation of time durations T2 T1 = k (P2 P1) for some constant k 1.0 in simulation time represents 1 hour of physical

time


26

Paced vs. Unpaced Execution

Modes of execution As-fast-as-possible execution (unpaced): no fixed relationship necessarily exists between advanced in simulation time and advances in wallclock time

Real-time execution (paced): each advance in simulation time is paced to occur in synchrony with an equivalent advance in wallclock time

Scaled real-time execution (paced): each advance in simulation time is paced to occur in synchrony with S* an equivalent advance in wallclock time (e.g., 2x wallclock time)


27

Paced vs. Unpaced Execution

Modes of execution With unpaced we refer to As-fast-as-possible

We rather use paced than Real-time execution for an execution that advances in simulation time in synchrony with (K*) an equivalent advance in wallclock time

As Real-time simulation often refers to certain real-time constraints for executing the simulation (either in paced or unpaced mode)

Faster than real-time typically implies a simulation used for decision processes.


28

Simulation Taxonomy

Continuous simulation State changes occur continuously across time Typically, behavior described by differential equations

Discrete simulation State changes only occur at discrete time instants Times stepped: time advances by fixed time increments Event driven: time advances occur with irregular increments

computer simulation

discrete models continuous models

event driven time-stepped


29

Time Stepped vs. Event Stepped

Goal: compute state of system over simulation time

stat

e va

riab

les

simulation time

time stepped execution

stat

e va

riab

les

simulation time

event driven execution


30

The Stepped Execution (Paced)

While (simulation not completed) { Wait until (W2S(wallclock time) current simulation time) compute state of simulation at end of this time step advance simulation time to next time step }

Simulation Time = W2S(W) = T0 + S (W W0) W = wallclock time; S = scale factor W0 (T0) = wallclock (simulation) time at start of simulation

(assume simulation and wallclock time use same time units)


31

Summary

Hardware Platforms Tightly coupled multiprocessors: fast communication Networked workstations: larger message latencies

Important to distinguish among simulation time, wallclock time, and time in the physical system

Paced execution (e.g., immersive virtual environments) vs. unpaced execution (e.g., simulations to analyze systems)

Continuous and discrete simulation use different execution paradigms, time advance mechanisms

Time stepped vs. event stepped Here, focus on discrete event simulations


32

The next weeks

Basics of Parallel Discrete Event Simulation

DEVS Advanced Techniques

Conservative Techniques

Optimistic Techniques

Advanced Optimistic Techniques

Partitioning, Loadbalancing

GPU based executions

HLA (High Level Architecture)

Background and ApplicationsIntroduction & MotivationWhat is parallel / distributed simulation?Why are we interested in this subject?Enable Simulation of Big ModelsFast Execution: On-Line Decision AidsIntroduction & MotivationVirtual EnvironmentsSystem AnalysisVirtual Environments vs. AnalysisPrincipal Application DomainsHistorical PerspectiveParallel / Distributed Simulation TodaySummary Outline Parallel and Distributed ComputersShared Memory MultiprocessorsMulti-CPU, Multi-Core PCs Distributed Memory MulticomputersParallel and Distributed SimulationOutline Slide Number 22Simulation FundamentalsTime Simulation TimePaced vs. Unpaced ExecutionPaced vs. Unpaced ExecutionSimulation TaxonomyTime Stepped vs. Event SteppedThe Stepped Execution (Paced)Summary The next weeks

Documents

Parallel and distributed Simulation