Challenges of Distributed Simulation

CETES Tech NightJanuary 18, 2011

Presented by Bill Millspaugh

Challenges of Distributed

Simulation

Challenges of Distributed

Simulation

Presented by Bill Millspaugh

[email protected]

580-284-4427

Goal of DIS is to link:

• Systems built for separate purposes

• Technologies from different eras

• Products from separate vendors

• Platforms from various services

• Human in the loop simulators

• Live entities

Types of Simulations

Operations with real

equipment in the field

Systems & troops in

simulators fighting on

synthetic battlefields

Wargames, models,

analytic tools

Live

Virtual

Constructive

-

-

-

Distributed Interactive Simulation (DIS)

• Distributed

– Physically separated

– Multiple simulators

– Dissimilar devices

– Multiple computers

• Interactive

– Simulated players interact

with an exercise

– Players affect each other

• Shoot

• Kill

• Collide

• Communicate

– Interact with environment

• Terrain

• Culture

• Atmosphere

• Water & Ocean

DIS Supporting Infrastructure

• Interface standards

• Communication architectures

• Technical forums

• Management structures

Architecture Characteristics

• Autonomous simulation nodes

• Transmission of “ground truth” information

• Transmission of state change information

• Object/event simulation architecture

• Use of “dead reckoning” algorithms to

extrapolate entity states

Autonomous Nodes

• Events are broadcast, available to all

• Receiving nodes are responsible for calculating

the effects of an event on the objects it is

simulating

• Effects may include the generation of new

events

• Nodes can join or leave an exercise in progress

• Initiating nodes need not calculate what effects

or who is affected by their event/stimulus

Transmission of “Ground Truth”

• Each node transmits the absolute truth about the state of the object(s) it represents

• Receiving nodes are sole responsible for determining whether their objects can perceive an event and whether they are affected by it

• Degradation of information is performed by the receiving node in accordance with an appropriate model of sensor characteristics before it is presented to human crew members or automated crews

Object/Event Architecture

• Information about non-changing objects in

the virtual world is assumed to be known

to all simulations and need not be

transmitted

• Dynamic objects keep each other informed

of their movements and the events that

they cause through the transmission of

Protocol Data Units (PDUs)

Common Environment

• Each simulation creates and maintains its

own copy of the common environment

• Terrain

• Coordinate System

• Infrastructure (roads, bridges, buildings…)

• Weather

• Dead reckoning

• Effectiveness calculations

Information Exchanged

• Simulation management

• Entity state information

• Fire events

• Detonation events

• Radio communication events

• Electronic emissions

Simulation Communications

• Information is exchanged over the network in packets called Protocol Data Units (PDUs)

• These formatted data units are exchanged between networked simulations to convey messages about entities and events

• PDUs provide data concerning simulated entity states, the types of entity interactions that take place in a DIS exercise, and they provide data for the management and control of a DIS exercise.

PDU Families

• Entity information interaction

• Warfare

• Logistics

• Simulation management

• Distributed emission regeneration

• Radio communications

• Entity management

• Minefield

• Synthetic environment

• Simulation management with reliability

• Live entity

• Non-real time protocol

Information in a PDU

• Example: Entity State

– Header: Protocol version, Exercise ID, Type

PDU, Time stamp, PDU length

– Body: Description of entity, Location, Velocity,

Orientation, Dead reckoning used,

Appearance, Description of articulated parts

and capabilities of the entity

Entity State PDU

Entity State PDU (cont)

Example Entity Enumerations

kind dom country cat subcat spec extra description

1 1 45 5 6 0 0 China_107mm_RKT

1 1 45 10 4 0 0 OPFOR_60mmMTR_TY31

1 1 78 2 6 0 0 Recon_Vehicle_HJ62C_Armored

1 1 222 0 1 11 0 Truck_Bridge_PMP_Engineer

1 1 222 1 2 7 0 OPFOR_T72BK

1 1 222 10 13 0 0 Mortar_2S12_120mm_Towed

1 1 222 28 4 2 2 ADA_Launcher_SA10

1 1 222 28 4 3 2 Radar_ADA_FLAPLID

1 1 225 2 1 8 0 M2A3_FIST

1 1 225 2 5 2 0 LAV_25_Recon

1 1 225 2 5 3 0 LAV_AT

1 1 225 2 5 6 0 US_HMMWV_PROPHET

1 1 225 2 5 28 0 US_Stryker_FIST

1 1 225 2 5 29 0 US_Stryker_Eng

1 1 225 2 5 31 0 US_Stryker_ATGM_Vehicle

Example Ammunition Enumerations

kind dom country cat subcat spec description

2 1 106 1 2 0 Aspide: HE-Frag

2 1 206 2 1 4 35mm gun: AHEAD

2 1 222 1 19 2 SA-7

2 1 222 1 30 0 SA-18

2 1 222 1 31 0 SA-19

2 1 225 1 2 3 AIM-120C (AMRAAM)

2 1 225 1 15 3 FIM-92 Stinger RMP

2 1 225 1 16 1 THAAD MISSILE

2 2 222 2 2 2 30MM/APDS

2 2 222 2 2 2 30mm APDS

2 2 222 2 2 4 30MM/HE-FRAG

2 2 222 2 2 4 30mm FRAG-HE

2 2 222 2 2 4 30mm HEI-T

2 2 225 1 2 4 25mm Bushmaster II

2 9 225 1 20 4 GBU-24

World Coordinate System

Entity Coordinate System

Euler Angles

• Entity Location is specified as the position

of the origin of the entity coordinate

system in world coordinates

• Orientation is specified using three angles

that describe successive rotations required

to transform from the world coordinate

system to entity coordinate system.

• These angles are called Euler angles.

First Rotation

Second Rotation

Third Rotation

Coordinate Transformations

• World coordinates

• Latitude/Longitude

• Latitude/Longitude

• UTM

Latitude/Longitude

Universal Transverse

Mercator (UTM)

Military Grid Reference

System (MGRS)Game Coordinates

Dead Reckoning

= DR Position

= True Position

= True/DR Coincide

Veh A

Veh B

Initial

state

Constant

course &

speed

True & DR

diverge,

threshold

not

exceeded

Threshold

exceeded,

B sends

update

A gets msg,

corrects his

position of B,

starts new DR

Notes: Scheme also applies to orientation and position

of articulated parts.

Smoothing algorithms remove jerky motion.

DIS Standards

• IEEE Std 1278.1-1995 (R2002) –Application Protocols

• IEEE Std 1278.1a – Supplement to IEEE Std 1278.1-1995

• IEEE Std 1278.2 1995 (R2002) – Communication Services and Profiles

• IEEE Std 1278.3-1996 (R2002) – Exercise Management and Feedback

• IEEE Std 1278.4-1997 (R2002) – Verification, Validation and Accreditation

• Simulation Interoperability Standards Organization (SISO) Ref-010-2006 – Enumeration and Bit Encoded Values for use with Protocols for Distributed Interactive Simulation Applications (Accompanies 1278.1 and 1278.1a)

Some Distributed Simulation

Applications

Link Geographically Separated Live, Virtual, and Constructive Entities

In Shared Joint & CoalitionSynthetic Theater Environments

F-16

AWACS

F-15

F-15E

F-16

F-16

F-16

AWACS

F-15

F-15

RQ-1

F-16

F-15E

F-16

F-16

F-16

F-117

B-1

B-52

B-1

F-16

RJ

B-2

F-16

AWACS

B-52

A-10

F-15

F-15

F-16

F-15

F-16

F-15

F-15

JSTARS

F-16

F-15EA-10

F-15

F-22

F-15

F-15E

F-16

F-16

OPERATIONAL FUNDED LIMITED DMT UNFUNDED

Distributed Mission Operations

32 AOC

Simulator/Jet + DMO + Network

Functional Concept Integrating Experiment 2010 (FCIE 10)

Ft Sill, OK

Colorado Springs, CO (SMDC)

Hurlburt, FL

Ft Benning, GA

Ft Bragg GA

Ft Rucker, AL

Ft Leonard Wood, MO

Ft Monroe, VA

Ft Gordon, GA

Huntsville, AL

Langley AFB, VA

MBL – Fort Benning AMBL - Fort Rucker

SIGCEN - Fort Gordon

SMDC - Redstone

Models, Tools and Systems

FCIE Portal Server

CPOF

Scalable Network Technologies

Comm Effects Server (SNT-CES)

Network Visualization Tool (NVT)

Call Manager

Adobe Connect Server

OpenFire

FireSim Interface

OneSAF

Representation:

TCN

ESB


OneSAF

FireSim Interface

AMDWS

TAIS (10.*)

CPOF

ForceXXI Battle Command,

Brigade and Below (FBCB2)

Representation:

Maneuver – IBCT,SBCT

Host Nation Trucks


EADSIM

JEMS

GCCS-A

FireSimXXI Interface

Representation:

National Assets

National ISR

Comms & Comms Relay

Platforms


Advanced Tactical Combat

Model (ATCOM)

NVT

OH58D Virtual Simulator

GCS-GUSS

TAIS (10.*)

Aviation Mission Planning

Station (AMPS)

CPOF

FireSim Interface

Representation:

Aviation Brigade

ADA - Radar

RWA

UAS

RWA


Fires Simulation XXI (FireSim XXI)

Reconfigurable Tactical Operations Simulator

(RTOS)

Network Visualization Toolkit (NVT)

Advanced Field Artillery Tactical Data System

(AFATDS)

Air Missile Defense Workstation (AMDWS)

Tactical Airspace Integration System (TAIS 10.*)

Total Battle Space Awareness (TBSA)

Joint Deep Operations Coordination System

(JADOCS)

Tactical air Control Party Close Air Support

System (TACP-CAS)

TBMCS (clients)

Forward Area Air defense Engagement

Operations (FAAD-EO)

Air Defense System Integration (ADSI(14.*))

Command Post of Future (CPOF)

CPOF Server

Battle Command Server (BCS)

Extended Air Defense Simulation (EADSIM)

Ground Based Laser (GBL) - Boeing


Representation:

Artillery

MPAD

ADA – non-radar

Maneuver

TBM & CM

Artillery ( +120mm Mortars)

UAV(Class 1 UAV &TUAV)

ADA

Hospital Elements, HQ Icon

Sustainment BDE Elements

Host Nation BDE Elements?

Civilians

FCIE 10 MODELS & SIMULATIONS

USAF - Hurlburt(505th)


Air Warfare Simulation(AWSIM)

Theater Battle Management

Core System (TBMCS)

Joint Deep Operations

Coordination System(JADOCS)

AFSERS

GCCS?

GIAC

ACE-IOS

Representation:

Fixed Wing(SOF,USAF)

Marine Air

Civilian Aircraft

Fires BL - Fort Sill

MSBL - Ft Leonard WoodModels, Tools and Systems

OneSAF

CPOF

JBC2S

BETSS-C

FireSim Interface

Representation:

MEB

Civilians

Maneuver


FireSim Interface

Navy Ship

EMGUN

USN – Ft Sill

USMC - Ft SillModels, Tools and Systems

FireSim Interface

MEU

SMDC-BL

Colorado SpringsModels, Tools and Systems


SOF – Ft BraggModels, Tools and Systems

FireSim Interface

CPOF

Representation:

SOF

USAF - LangleyModels, Tools and Systems

Call Manager

Adobe Connect Server

OpenFire

Representation: BCBL-LeavenworthModels, Tools and Systems

CPOF

CPOF Server

Persistent Major Issues

Issues• Lack of correlation of common environment

• Terrain data correlation– Multiple terrain database architectures (Open Flight, Objective

One SAF, Compact Terrain Data Base, Evans and Southerland,…)

– Both visual and underlying simulation databases required

– Visual databases differ from simulation to simulation

– Trees and vegetation are added differently for visualization and simulation

– Lack of models for calculating blast damage to buildings and terrain

– Visualization of building damage is very primitive

– Need models to generate rubbling of buildings

– No mechanism/standard exists to communicate terrain and infrastructure damage in one simulation to others in the network

– Micro terrain is implementation dependent

Issues (cont)

• Terrain database generation is labor intensive

• Weather is ad hoc

• Each simulation handles smoke and dust differently

• Visual dust and smoke effects are probably not correlated to internal smoke and dust simulated effects

• Need automated voice recognition in high noise environments

• Need natural language understanding

• Need artificial intelligence to control computer generated forces

• Tactical messaging to simulation units for command and control (tactical messages are difficult to decode and keep changing)

Issues (cont)

• Need good crowd modeling

• Need open source stealth viewing devices

for virtual environment

• Need good interface control documents

with exercise control information and

enumeration databases (Good exercises

are a result of good management)

Skills Needed for Modeling and

Simulation in a Distributed

Environment

Skills Required

• Mathematics

– Calculus

– Matrix algebra

– Vector mathematics

– Geometry/trigonometry

– Statistics

• Random sampling techniques

• Random number generation

Skills (cont)

• Physics/mechanics/dynamics

– Light

– Optics

– Momentum

– Kinetic energy

– Projectile equations

• Computer science

– Object oriented design/programming

– Sorting

– Searching

– Hashing

– Data/information structures

• Lists– Stacks, queues

– Single/doubly linked lists

• Trees– Binary tree representation

– Traversing binary trees

– M-way trees

• Multilinked structures

• Dynamic storage allocation

Skills (cont)

• Networking

– General network troubleshooting

– Switch programming capabilities

– Router programming capabilities

– Understanding of IPv4

• Subnetting

• How broadcast/unicast/multicast networks work

• Gateway and routing configuration

• DNS standards

Skills (cont)

Questions?

Technology

Challenges of Distributed Simulation