JSF Modeling Information Management JSF Modeling Information Management NDIA Systems Engineering Conference San Diego, California October 2003 Ref: JSF03-03-0649

JSF Modeling Information ManagementJSF Modeling Information ManagementNDIA Systems Engineering Conference

San Diego, CaliforniaOctober 2003

Ref: JSF03-03-0649

Jim HollenbachSimulation Strategies, Inc.

[email protected], 703.360.3902JSFPO Technical Lead

Modeling Information Sources Action Team

W. Henson GravesLockheed Martin Aeronautics Company

[email protected], 817.777.1856LM Aero Technical Lead

Modeling Information Sources Action Team

DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

mailto:[email protected]

mailto:[email protected]

2

OperationalPerformance

F-16CF-22

JSF-AF

F/A-18E/FAV-8B JSF-N

JSF-MC Potential Budget Range

Historical Avg TAC AIR Share of Funding(per CBO Study)

CBO High-End Unit Cost Estimates* -F/A-18E/F, F-22, JSF

TACAIR Procurement Funding Requirements

Data Sources: DoD budget reports, LMTAS estimates using DoD budgetdata derived cost factors

* CBO Testimony on Modernizing Tactical Air craft,Airland Subcommittee, SASC,10 March 1999

Billionsof

FY99$

2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 2020

0

0

2

4

6

8

10

12

14

16

18

Support

Test and Evaluation

Manufacturing

Engineering Design

Complete,Complete,AuthoritativeAuthoritative

Product Product Development Development

DataDataRequirements Analysis

Simulation Based Acquisition (SBA):Shared Information is the Hub of IPPD

JSF Program Commitment:

•Availability of complete consistent digital product data

• 24/7 access and navigation of digital product data by the entire team

• Data interoperability between team and suppliers

• Configuration management and publishing in-progress designs

• Accreditation and traceability of digital product data


3

Consistent, Authoritative andReadily-Available Information is Vital

To produce the JSF Coordinate development activities, assess development progress Reduce the confusion and risks that arise if operating on wrong or

logically inconsistent information Establish validity of M&S-based analyses by tracing results back to

authoritative information

To conserve resources Avoid having to repeatedly find, produce and/or translate the same

information Enable more timely analyses, resulting in shorter decision cycle times

and more efficient systems engineering Avoid the costs of mistakes that arise from bad information


4

JSF Simulation Based AcquisitionImplementation Team

SBA Implementation Council

M&S User Action TeamFunctional needs

& processes

Analysis Training Test

M&S Tools Action TeamSoftware applications

Modeling InformationSources Action Team

Authoritative data

V&V

A ctio

n T e

a mQ

uality

co n

trol/r

is k r e

duct

ion

Mission SystemsM&S working group

Associated work groups


5

Data Inputs

Engineering

Campaign

Mission

Engagement

Cont

ext-i

ndep

ende

nt da

ta

Cont

ext-i

ndep

ende

nt da

ta

Exter

nally

or in

terna

lly ge

nera

ted

Exter

nally

or in

terna

lly ge

nera

ted

Prim

itive

Aggr

egate

d

Context-dependent data

Context-dependent data

Internally derived

Internally derived


6

Data Lineage Example(Surface-to Air Missile Vulnerability Analysis - simplified)

ENGAGEMENT

MISSION

CAMPAIGN

ACQUIRECOVART FASTGENHITRAN RCHEM

SPIRITS

ESAMS

ASM

DIADS

BRAWLER SHAZAM

MIL-AASPEM

RADGUNS MOSAIC

JSF VirtualSimulator

JIMM

GIANT

EADSIM

LCOM CFAM

THUNDER

ACAD CATIA NASTRANPGEN 4

Suppressor

WINFIRE

AFMSS(CLOAR)

Live Fire Test

FLIR 92 MODTRAN ENGINEERING


7

Inverse Abstraction Relationship

Engineering levelsimulations

Campaignlevel

simulations

Mission levelsimulations

Engagement levelsimulations

Context-dependentdata item

alternatives

Degr

ee o

f abs

tract

ion

Number of instances Number of instances

Data applicability narrows as context dependencies increaseData applicability narrows as context dependencies increase Standard contexts reduce instances that must be managedStandard contexts reduce instances that must be managed Include context in metadata to guard against inappropriate useInclude context in metadata to guard against inappropriate use


8

JSF Authoritative Modeling Information System (JAMIS) Information Flow Concept

Version 16, 31 Jan 03

MNSORDConcept Exploration & DefinitionDemonstration / ValidationEngineering Manufacturing & DevelopmentProduction & DeploymentUpgrade and Replacent

PHASE

100010010 MGMTJan Apr Jul Oct

Authoring tools(e.g., architecture, CAD)

DSMs / relevant infoin tool-specific form

Study-specificinput datasets

& software code

Analyst actionAnalyst action

Raw M&S results

Threat & friendlysystems[ASDB]

Operationalcontext

Naturalenvironment,infrastructure

JSF[DPD]

Interactions(Pd, Pk,etc.)

Link

ing

& tra

nsla

tion

serv

ices

(map

ping

, sem

antic

s/sy

ntax

)

Study-Study-specificspecificarchivesarchives

MNSORDConcept Exploration & DefinitionDemonstration / ValidationEngineering Manufacturing & DevelopmentProduction & DeploymentUpgrade and Replacent

PHASE

100010010 MGMTJan Apr Jul Oct

Analysis tools(e.g., models, simulations)

Register DT & OTtest results

LM Aero Team,Operational Test

Authorities

Resource AccessResource AccessSystem (RAS)System (RAS)

NIMA,DTRA,

NRO, etc.

ProgramOffices

IntelCenters

Services,JFCOM,

etc.

Threat C&Pinformation

Operationalcontext info

Naturalenvironment &infrastructure

C&P info

Blue systemC&P info

Registerauthoritativedata (JSFPO)

Color key: JSFPO provided LM Aero provided

DesignchangeMoMs

JSFJSFcontrolcontrolboardsboards

Digi

tal S

yste

m M

odel

s Register smoothed /analyzed results

User interfaceadmin data

JSF design changedecisions

Register designdecision

Design changeorders

Registerproposeddesigns

(LM Aero)


9

JAMIS Operation Roles and Responsibilities

Version 12, 30 Jan 03

Review data & metadataRegister dataAuthor metadata

View data & metadataAssess & mark (in metadata)

JSF SORA & LM Aero Tier III IPT directors: Grant privileges

Consumers : JSF Enter- prise, others TBD

Analyses andtesting

Technologydevelopment

JSF AuthoritativeModeling

Information System

Externalreferences(e.g., IntelCenters)

View data & metadataDownload & use data

Implement privilegesProvide training and technical support

Manager : LM Aero, incooperation with JAMISdatabase managers

Examiners : VV&A Action Team, others

Producers : Designers,JSFPO, analysts, testers


Bluesystems

Threatsystems

Naturalenvironment

Civil/militaryinfrastructure

OperationalContext

(scenarios, etc.)

Governmentprovidedcontext

information

LM Aero provided Product Information Model (DPD)(Variants = layered instance values)

Functional allocationtrade studies

Manufacturing/supplier capabilities

Mobility

Specialized views:• OML, Cost, BOM, mass properties, etc.

Coherency & completeness cross-checks• Solid lines = primary• Dashed lines = secondary

Transportation capabilities

• Assemblies & components• Software• Spatial & electrical relationships• Connections• Part records

• System/ subsystem topology• Functional allocation• Technologies• Interfaces

• Air vehicle performance• Mission system capabilities• Autonomic logistics• Signatures• Reliability

JSF Product Development Information Model Scope

• Lethality/ effectiveness• Survivability

Manufacturingprocess

Physicalarchitecture

Support infrastructure Maintainability

Performance

Personnel rqmts

Interactions

Logicalarchitecture

(Requirements are recorded in appropriate

information domain)


11

JAMIS Architecture – System View

Information model as means to locate, navigate, and access distributed data

ResourceResourceAccess SystemAccess System

Metaphase

MASCOT

PVCS

Other databases

Information Model(info taxonomy and

decomposition, interrelationships)& data locations

ASDB

DOORS

RAS application server

Web Browser (RAS GUI)

Database servers


12

Challenge: Information Mapping

Professional disciplines, and M&S developers, represent (model) things in widely differing ways

Conceptual views (abstract ways of thinking about what’s involved)

• What entities, attributes, actions and interactions are represented, at what level of granularity, etc.

Contexts in which information is valid

Semantics and syntax of individual data items


13

Dimensions ofInformation Coherency

Temporal coherence The extent to which the data being used for different tools and sites has

the same time stamp

Organizational coherence The extent to which all enterprise organizations are using the same set of

databases

Layer (tier, level, stratum) coherence The extent to which all the information at a given level of granularity (e.g.,

engagement level) is logically consistent

Perspective coherence The extent to which information from different perspectives (e.g., logical,

physical, behavioral) is logically consistent

Abstraction coherence The extent to which the information at different levels of granularity is

logically consistent (the context dependency issue)

Hard

Hard

Very

Har

dVe

ry H

ard


14

Development Strategy

Due to the absence of MLS and two-way guards, we are establishing separate JAMIS instances at each security classification level Unclassified, SECRET Collateral, TOP SECRET SAR

Spiral development A series of builds, prioritized per program needs, to support:

• An increasing amount of information• An increasing numbers of tools• Interfaced to an increasing number of organizations

Version 3 in development now


15

JPO TOP LEVEL SCHEDULEJPO TOP LEVEL SCHEDULEDISTRIBUTION A – Approved for Public Release; Distribution is Unlimited

Version 8a – current as of 28 Feb 03

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

CY 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Pre-SDDPre-SDD SDDSDD

SDDSDD

PAC

ASRR

IBR

ASCDR 1

ASCDR 2

AS CDR 3

Blk 1FTR

Block 2

Blocks 0 & 1

Block 3

0A 0B 0C

ISR

Development and QualificationDevelopment and Qualification

CTOL STOVL CV Blk 1 RFT Blk 2 RFT Blk 3 RFT

LRIP I (10)

LRIP II (22)

LRIP III (54)

LRIP IV (90)

LRIP V (117)

LRIP VI (166)

Multi Year

Production6 CTOL4 STOVL

14 CTOL 8 STOVL

20 CTOL20 STOVL 9 CV 5 UK 30 CTOL

32 STOVL20 CV 8 UK

DAE IPR 1

DAE IPR 2

DAE IPR 3

DAE IPR 4

DAE IPR 5

DABReview

ORD IOC USMC

ORD IOC USN

ORD IOC USAF

LL FF

LL FF

LL FF

LL FF

LL FF

LL FF

PDR FETT

ASPDR

Blk 2FTR

Blk 3FTR

FY 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

CTOL/CVIFR OCRCTOL/CV STOVL

AutonomicLogisticsDevelopment

Low Rate Initial Production (LRIP)

Air SystemProgram Events

Propulsion

Test and Development

44 CTOL32 STOVL32 CV 9 UK

72 CTOL36 STOVL48 CV10 UK

1 2

2013

110 CTOL36 STOVL48 CV12 UK

CDRFETT STOVL

F135F136

1 2 3 4

2012

Mission & Vehicle Systems

IBR

CTOL Trng Sys PDR ALIS

PDR

TSys/ALIS CDR Blk 1 RFO

Blk 2 Flt Test

Blk 0 & 1 Flt Test

Blk 3 Flt Test

CTOL 1st Flt CV 1st Flt

STOVL 1st Flt

OT&E

OA OA

(4 CTOL, 2 STOVL)

(4 CTOL, 4 STOVL, 4 CV)

EOA OA OA OA

Flt Test Bed

MILCON Test Facilities

(2 CTOL, 2 STOVL, 2 CV)

CDR FETT IFRFETT PDRCDR ISR OCRPDRIBR


16

A Few Challenges

Fostering broad understanding of the need and the solution concept

Integrating into the system engineering process Finding authoritative data sources Overcoming reluctance to release data Replicating less classified data at higher levels Providing lower level representations of highly-

classified data Accomplishing the data engineering


17

Discussion


18

Summary

Breaking new ground in management of modeling information

Information technology advances don’t solve all challenges

Holistic approach and good systems engineering are required

Promises to advance state of the practice, impact SBA concepts

Still a work in progress Incremental deliveries, incremental changes for users Wish had started sooner!

Many challenges common to other acquisition programs DoD/Service-level actions could help


19

Information Types

Parametric data Algorithms Software code Publications Subject Matter Expert knowledge

Initial focus has been on parametric dataInitial focus has been on parametric data


20

Modeling InformationCharacterization

Regardless of its subject, data may be characterized by its: Trustworthiness: “authoritative” or “non-authoritative”

Evidence in the real world: “empirical” or “derived”

Context dependency: “context-independent” or “context-dependent”

Granularity: “primitive” or “aggregated”

Structure: “atomic” or “complex”

Source (relative to the enterprise): “external” or “internal”

Above characterizations are orthogonal A data item could be described by any combination


21

Operational Context DB Scope(Enterprise-wide standards for analysis; in priority order)

1. Scenarios Road to war Blue TPFD Orders of battle (UOB, EOB, etc.) Geographic area Infrastructure targets, location

2. Mission Groups3. Missions4. Vignettes (created by JSFPO)

5. Use Cases (created by JSF IPTs) Initiating inputs, required actions, MOPs

6. Natural environment conditions Time of day, weather

7. Concept of Operations (CONOPS)8. Force doctrine9. Tactics10. Rules of engagement (ROE)11. Firing doctrine12. Infrastructure lay down (other than targets)

Not in Operational Context DB: Study-specific archives

• Input data sets• Raw execution outputs• Discretionary analyst decisions

– e.g., Blue mission routes System C&P data Natural environment instance data Infrastructure C&P


22

JAMIS Architecture - System View

PDMSystem

ResourceAccess System

Product Development Info Model

+Storage Locations

Metaphase

PVCSOther DBsASDB

DOORS

JDL

DPDdata

DPDdata

DPDdata

Internet Explorer (RAS GUI)

Otherdevelop.

data

DPDdata

DPDdata


23

ResourceAccess System

The JSF Product Development Information Model provides a catalog

structure for the DPD represented in the RAS

interface

Metaphase

PVCS

Other DBs

DOORS

JDLDPDdata

DPDdata

DPDdata

DPDdata

DPDdata

System View

The PDM System (PDMS) is a

federated set of data management

systems for development and release control of

product data

Product Development Information Model

The Resource Access System provides 24/7 Web-access to a single, logically unified JSF representation of variants,

and configurations

The DPD is the release

controlled product data

contained in the Product Data Management

System

PDMSProduct Data currently in

JDL will transition to Metaphase

ASDB


24

Impact Analysis and Consistency

DI1

PE1

DI2 DI3

DI6

PE3

DI5

PE4

DI8

DI7

PE2

DI4

Impact Analysis: A change in D1 potentially necessitates the re-execution of PE1 and PE3 to produce DI5 and DI8.

DI1

PE1

DI2 DI3

DI6

PE3

DI5

PE4

DI8

DI7

PE2

DI4

Consistency: A consistent version of DI3 must be used to eventually produce a coherent version of DI8.

DI1

PE1

DI2 DI3

DI6

PE3

DI5

PE4

DI8

DI7

PE2

DI4

DI – Data ItemPE – Process Execution


25

Rationale for aIntegrated Information Model

Datasource 1

Datasource n

Datasource 2

Datasource 3

Modelformat 1

Modelformat 3

Modelformat 2

Modelformat n

......

Datasource 1

Datasource n

Datasource 2

Datasource 3

Modelformat 1

Modelformat 3

Modelformat 2

Modelformat n

......

IntegratedInfo Model

(commonsemantics& syntax)

Many translators (~N2 for N tools)Difficult configuration management

Fewer, less fragile translatorsEasier configuration management


26

Practical Limitations

REDIS process works in domains with high semantic similarities Similar ways of looking at the world

Synthesizing an integrated information model requires an intimate understanding of the producing and consuming tools (models,simulations) Usually a time-consuming task

Conceptual model & assumptions of the tools are often unclear Developer may not have documented this May be a function of the analysis to be performed

Automatic translation of source data into input data sets has practical limitsAutomatic translation of source data into input data sets has practical limits

- Much responsibility must be left with the analyst - Much responsibility must be left with the analyst


27

Information Model is Maintainedas a UML Class Model

DatabaseServer

RAS Server

RASClient

DataViews

Data Base Connectors

Data formatting driven By XSLT rules

UML Class Model

XML Schemas& XSLT rules

Utility program

…which is used to generate XML schemas that are loaded into RAS


28

Recommendations: DoD Data Policy

Information is a DoD corporate resource Establish means to capture data needs

similar to Intel Center ‘Production Requests’ Crisply define production “swim lanes” & responsible

organizations Government data rights for all reusable data (contracting policy) Business/funding rules Report cards

Define user responsibilities and liabilities Establish means to appeal data release denials Provide visibility/awareness means

Internet and SIPRNet registries of core metadata about the data assets, not the data itself• Database description, format, security classification, date, POC, etc.


29

Recommendations: Standards Need tool standards, not standard tools!

Tool standardization is impractical, costly, negates power of marketplace

Define standard data interchange formats (DIFs) Need more, better DIF standards (at the ‘payload’ level)

• Well-defined semantics & syntax• Standard metadata, including context dependencies• More granular, less overlapping

Need DoD to use its ‘bully pulpit’, be engaged Define DoD-sanctioned venues for standards development

• Commercial standards landscape is quite confused, both nationally and internationally

• NIST can help Define full range of standard operational contexts

Reduce context-dependent data alternatives to a manageable level Start with Design Reference Missions for each mission area, flesh

out down to tactics


30

Three JAMIS Architectural Views

Operational ViewJSF team interactions for digital product design and assessment

Systems ViewAn integrated set of information

systems and supporting applications

Technical ViewWeb information services and

standards

JAMISJAMIS


31

Sources of this Requirement

JSF Modeling and Simulation Support Plan (MSSP) a government-produced contract reference document

JSF Acquisition Strategy LM Aero proposal SDD contract Integrated Master Plan events

e.g., JSF Distributed Product Description (DPD) is a DAE IPR & DAB review exit criterion


32

Simulation Based Acquisition (SBA):Shared Information is the Hub of IPPD

OperationalPerformance

F-16CF-22

JSF-AF

F/A-18E/FAV-8B JSF-N

JSF-MC Potential Budget Range

Historical Avg TAC AIR Share of Funding(per CBO Study)

CBO High-End Unit Cost Estimates* -F/A-18E/F, F -22, JSF

TACAIR Procurement Funding Requirements

Data Sources: DoD budget reports, LMTAS estimates using DoD budgetdata derived cost factors

* CBO Testimony on Modernizing Tactical Aircraft,Air land Subcommittee, SASC,10 March 1999

Billionsof

FY99$

2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 2020

0

0

2

4

6

8

10

12

14

16

18

Support

Test and Evaluation

Manufacturing

Engineering Design

Complete,Complete,AuthoritativeAuthoritative

Product Product Development Development

DataDataRequirements Analysis


33

Components of theJSF Modeling & Simulation Toolset

JSF Suite of Models and Simulations (SoM&S)

JSF Strike Warfare Collaborative Environment

(SWCE) Toolset(28 models & sims)

JSF Engineering and Manufacturing Collaborative Environment

SWCE SoM&S EMCE SoM&S

JSF Distributed Product Description (DPD)

MSDBScenarios

DatabasesAuthoritative Systems

Data Base (ASDB)

MSDB

Scenarios

Databases

JSF Resource Access System (RAS)

(EMCE) Toolset(100+ models & sims)

(JSF M&S Support Plan Fig. 5-1, annotated)


34

3 Paths to System Representations

Compile

Load

Develop asoftware component

Develop a simulationapplication

Input data sets

Input data sets

Input data sets

Executableapplications

Model or simulationmodified to represent

a battlespace

Model or simulationdeveloped to represent

a system

Model or simulationinitialized to represent

a battlespace

(Fed

erat

e)ex

ecut

e(F

eder

ate)

exec

ute

(Fed

erat

e)ex

ecut

e

Load

Load

Compile

Data-driven system representation in another’s tool (e.g., Thunder, JIMM)

Software representation within another’s tool (e.g., ESAMS, Brawler)

System representation as a dedicated application (e.g., JSF Virtual Simulator)

Info

rmat

ion

map

ping

, tra

nsla

tion,

pre

-pro

cess

ing

Authoritative infoin its original form

DIFs

Mostauthoritative

systeminformation

Information inapplication-specific

formDISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

35

RAS User Interface Example

Data Item icons appear here

The Data Items associated with a

navigation node are listed by data item

type

Each navigation node has metadata


36

Data Engineering ProcessReverse Engineering for Data Integration and Sharing (REDIS) - simplified

Data InterchangeData InterchangeFormat specsFormat specs Integrated DatabaseIntegrated Database

Integrated Information ModelIntegrated Information Model

InformationInformationmodelmodel





Source 1Source 1data filesdata files

Model 1Model 1data filesdata files

Model 2Model 2data filesdata files

system nsystem ndata filesdata files

Source 2Source 2data filesdata files

Reverse

Reverse

EngineeringEngineering

Integration/Synthesis

Forward

Forward

EngineeringEngineering

Data DictionaryData Dictionary

…

…


37

A Derivation Trace

Preliminary or Detailed Analysis

CATIAfiles

Outer Mold Line Conversion

Process

Facetfile

Xpatch /

3D MoMAzimuth

Frequency

Elevation

Polarization

Material Assignment

Process

…RCS Values

…

Brawler AircraftChar. And Perf.

Parameters

Materials database

Facet filewith Material IDs


38

Sample Aircraft Performance Attributes Lift coefficient vs. angle-of-attack• Temperature vs. mach, altitude, throttle• Post-stall maneuver capability• Min and Max g capacity• Max mach number and calibrated airspeed• Afterburner delay time• Turbine spin-up and spin-down times• Correlation factor for turbine to afterburner thrust• Max mach number with a drag device• Constraints on thrust reversal employment• Time to max thrust reversal• Thrust reversal efficiency• Damping ratio for angle-of-attack response• Roll acceleration factor due to angle-of-attack• Stability axis roll rate limits by AOA and dynamic pressure• Roll acceleration increment for positive AOA acceleration. limit• Radar cross section by azimuth, elevation and wavelength• Infrared signature characteristics• Physical attributes• Weapons quantities, operating modes and locations• Sensor quantities, operating modes and locations


39

Shared Responsibilities During SDDThe Government will: The SDD contractor shall:

Establish and maintain a Strike Warfare Collaborative Environment in which the DPD-based JSF representations will operate, that spans the collaboration focus areas of mission effectiveness analysis, autonomic logistics analysis, engineering and manufacturing analysis and cost analysis

Establish and manage an Engineering and Manufacturing Collaborative Environment

Provide configuration-managed versions of a JSF SWCE Suite of Models and Simulations

Develop, populate, and manage the JSF DPD, which, as a minimum, shall provide, via a DIF, all the JSF information needed to represent the JSF in the SWCE and the EMCE SoM&S.

Develop, maintain, and manage the configuration of a library of SWCE boxed sets. Develop translations and Digital System Models (DSMs) not provided by the WSC.

Develop the software that may be necessary to convert DPD information into the form needed to initialize or configure an individual model or simulation for that portion of the SWCE SoM&S that the WSC uses. This shall include translations and JSF Digital System Models.

Provide an Authoritative Systems Database (ASDB) which is accessible through the JSF Resource Access System

Develop and maintain a web-based JSF Resource Access System

JSF M&S Support Plan Table 1-1DISTRIBUTION STATEMENT A: Approved for Public Release; Distribution is Unlimited.

Documents

JSF Modeling Information Management JSF Modeling Information Management NDIA Systems Engineering Conference San Diego, California October 2003 Ref: JSF03-03-0649