Fundamentals of Enterprise Systems Engineeringcau-ses.net › ... › 07 › Maritime-Awareness-Presentation.pdf© 2010 The MITRE Corporation. All rights reserved. 1 Brian E. White,

© 2010 The MITRE Corporation. All rights reserved.

1

Brian E. White, Ph.D. The MITRE Corporation 8 April 2010 IEEE International Systems Conference Hyatt Regency Mission Bay Spa and Marina San Diego, CA

Complex Systems Engineering Case Study of a Case Study on Maritime Domain Awareness

See Notes Page

Public Release Case Number: 09-2555

© 2009 The MITRE Corporation. All rights reserved

Topics for Discussion

Maritime Domain Awareness (MDA)

– Focus: Information sharing across agencies

Templates for characterizing

– Your systems engineering (SE) environment

– What you’re doing about it (in applying SE techniques)

Complex Adaptive Systems Engineering (CASE) methodology

– “Bubble” chart overview

– Description of activities

Takeaways

2



Introduction

Background

– 2003 challenge: “How would all agencies of government … create cross-domain maritime awareness, and ultimately, safety …?”

– Success depended upon

Creating CONOPS to achieve cross-domain maritime awareness

Eliminating barriers to information sharing.

Purpose

– Significant improvements in MDA

2007: Kennedy School at Harvard published a case study

We analyzed this case study with respect to CASE methodology

Stakeholders

– U.S. Customs, Immigration, Navy, Coast Guard, DoT, Intelligence and civilian communities, …

Came with varying perspectives and differing priorities

Each had systems and processes to conduct MDA activities, and did not have the incentive individually to change

Constrained budgets limited their ability

3

See Notes Page



4

Introduction (Concluded) See Notes Page

Challenges

– Dealing with difficult environment across multiple dimensions of Enterprise Systems Engineering (ESE) framework (see below)

Operating in most difficult “messy frontier” regarding Scope of Effort, Implementation Context, and Stakeholder Involvement

System Context, Mission Environment, and Stakeholder Relationship were close to messy frontier

ESE Profiler [4]



Activities

5


MDA COI goals dictated ESE approach

Each stakeholder managed their systems independently

System development was driven by requirements

At the outset, typical MDA COI member

– Established specific system requirements and adapted systems under their ownership to meet these requirements

– Considered system trade-offs but without formal modeling/ simulation or cost/benefit analyses

– Developed a system architecture and updated the architecture as system evolved to meet new requirements

– Employed existing and new technologies to fulfill requirements

– Pursued alternative solutions within their own system context

– Managed system risks and implemented system improvements

– Separated testing from operational environments to ensure testing was conducted before moving systems into operation

– Addressed operational problems with their system but without attention to interoperability

– Observed operational effectiveness without measures and metrics.

See Notes Page



Typical Systems

Engineering Activity

Left End of

Slider

Left Intermediate Interval Center Intermediate

Interval

Right Intermediate

Interval

Right End of

Slider

Define the System

Problem

Establish

System

Requirements

Adapt to Changing

Requirements; Re-Scope

Revise and Restate

Objectives

Try to Predict Future

Enterprise Needs

Discover Needed

Mission

Capabilities

Analyze Alternatives Conduct

Systems

Tradeoffs

Model/Simulate System

Functionalities

Perform Systematic

Cost-Benefit Analyses

Include Social and

Psychological Factors

Emphasize

Enterprise

Aspects

Utilize a Guiding

Architecture

Apply an

Existing

Framework

Develop Architectural

Perspectives (Views)

Really Define (Not Just

Views of) Architecture

Adapt Architecture to

Accommodate Change

Embrace an

Evolutionary

Architecture

Consider Technical

Approaches

Employ

Available

Techniques

Research, Track, & Plan

for New Technologies

Research and Evaluate

New Technical Ideas

Pro-Actively Plan for

Promising Techniques

Explore New

Techniques and

Innovate

Pursue Solutions Advocate One

System

Approach

Consider Alternative

Solution Approaches

Investigate Departures

from Planned Track

Iterate and Shape

Solution Space

Keep Options

Open While

Evolving Answer

Manage

Contingencies

Emphasize and

Manage System

Risks

Mitigate System Risks

and Watch Opportunities

Sort, Balance and

Manage All Uncertainties

Pursue Enterprise

Opportunities

Prepare for

Unknown

Unknowns

Develop

Implementations

Hatch System

Improvements

Off-Line

Prepare Enhancements

for Fielding

Experiment in

Operational Exercises

Develop in Realistic

Environments

Innovate With

Users Safely

Integrate Operational

Capabilities

Test and

Incorporate

Functionalities

Work Towards Better

Interoperability

Advance Horizontal

Integration As Feasible

Advocate for Needed

Policy Changes

Consolidate

Mission

Successes

Learn by Evaluating

Effectiveness

Analyze and Fix

Operational

Problems

Propose Operational

Effectiveness Measures

Collect Value Metrics and

Learn Lessons

Adjust Enterprise

Approach

Promulgate

Enterprise

Learning

Version 4 – 4 Jan 09

Traditional Systems

Engineering (TSE)

Complex Systems

Engineering (CSE)

Convenient Labels

(Only; interpret them):

“After CASE” “Before CASE”

Aggregate Assessment

of Above Slider Positions

Systems Engineering Activities (SEA) Profiler Showing Maritime Situational Awareness Case Study


See Notes Page



7


Activities (Continued) After the fact, MDA COI became more ESE oriented and

Revised their approach to requirements to better accommodate interoperability objectives

Considered system trade-offs from more enterprise perspective

Focused more on enterprise architecture as systems evolved to enhance participants collective capabilities

Continued to employ existing and new technologies to enable new capabilities

Pursued alternative solutions within larger enterprise context

Managed system risks while pursuing interoperability opportunities

Implemented system upgrades with some attention to collective improvements

Embraced and worked toward integrating interoperable solutions

Addressed operational problems with more attention to interoperability effectiveness.

See Notes Page



8


Activities (Continued) Approach for most difficult environments called Complex

Adaptive Systems Engineering (CASE) has been developed

For details refer to [2, 3]

Eight CASE activities, and various sub-activities are summarized in Table I of the paper but refer to the next chart

In paper we cite CASE activities that resonate with MDA case study and compile correlations sub-activities

See Notes Page



CASE Methodology

9

Legend: Segments that are on

The Main Path

An Alternative Path

Architect a

Strategy

2Create Climate

for Change

1

Reward Results

4Target Outcome

Spaces

3

Formulate

Decision-Making

Heuristics

5

Stimulate Natural

Processes

6

Assess, Learn,

and Re-Plan

8

Develop in

Operational

Environs

7

© 2010 The MITRE Corporation. All rights reserved © 2010 The MITRE Corporation. All rights reserved.


Activities (Continued)

Detailed analysis is summarized

– Stakeholders, Sponsorship, and Governance

Stakeholders, sponsorship, and governance are important topics, related to CASE Activities 1 and 2, Create Climate for Change, and Architect a Strategy, respectively

MDA COI had Mr. Mike Krieger of DoD’s Networks & Information Integration (NII) organization that served as driving force for change. He was proactive in identifying and engaging with prospective customers or constituents, and did not wait for them

Stakeholders were concerned about risks of participating. Incentives were critical to gain participation. Resistance to exposing one’s data had to be addressed up-front. People have to see benefits of change.

Early on, Mr. Krieger gained sponsorship of MDA COI from Admiral Joseph Nimmich of DHS and Rear Admiral Nancy E. Brown of NORTHCOM. They helped ensure stakeholders were on-board with transformational change and helped judge outcomes and what results to reward. This governing body provided needed focus to stakeholders, particularly those involved in developing solutions.

Resulting cross COI collaboration leveraged organic resources. The co-existence of multiple, competing approaches drove natural evolution and led to better outcomes.

10


See Notes Page


Activities (Continued)

Detailed analysis is summarized (continued)

– Motivating Information Sharing and Interoperability

This is related to CASE Activities 3, 4, and 6 but can involve 1 and 2

Information sharing is against culture of many organizations. There are often good reasons for protecting certain information that

- Poses potential security risks

- Violates individual privacy rights

- Discusses degree of internal compliance with externally imposed policies

- Discloses plans for organizational change before internal decisions are made

- Invites unwarranted outside intervention in operational methods

It is important to protect against information misuse by others but this mindset can be detrimental to sharing information

MDA presented opportunity to share collective “stove-piped” technological data for superior situational awareness

Investment was stimulated by showing beneficial end results to users with relatively little effort, lowering barriers to entry.

Effort was also sustained because cooperative participants looked good to their overseeing stakeholders.

11


See Notes Page


Activities (Concluded)

Detailed analysis is summarized (concluded)

– Enhancing Operational Capabilities

This is related to CASE Activities 5, 6, and 7 but 2, 3, and 4, were also involved

Language/terminology barriers were overcome using simplicity and compromise. Non-interoperability challenge was overcome by creating CONOPS that enabled cross-domain maritime awareness.

Collective outcome space was defined. Each provider standardized and shared their data through network interface that was easy to use.

Partnership across stakeholders, early ownership, and pursuit of opportunities with small, incremental improvements led to success. Progress was facilitated by modicum of bureaucracy.

Lessons learned were continually captured as foundation for future efforts

– Summary

In developing cross-domain maritime sharing capability, MDA COI leveraged existing capabilities and infrastructure. Stakeholders contributed data sources and systems to understand and expand situational awareness picture.

12


See Notes Page


Results Most CASE methodology activities were exemplified, but

mainly Activities 1, 2, and 3, followed by Activities 4 and 6.

As detailed separately each citation of MDA case study was allocated to one or more CASE sub-activities. Each such allocation was given a weight or score of 1, 2, or 3, with 3 being the greatest weight.

13


See Notes Page


Results (Concluded)

Greatest resonance between CASE methodology and MDA case study were sub-activities of Activities 1, 2, and 3

– Work with other stakeholders to surface issues, harmonize mutual interests, and propose solutions

– Discuss, define, and analyze: Nature of problem; System boundaries; Desired outcome spaces; Relevant organizations; Potential stakeholders

– Describe them in ways that are: Clear, succinct, and compelling: Oriented toward (mostly qualitative) outcome space capabilities; Devoid of specific (mostly quantitative) solutions

Least correlation was found with CASE Activities 5, 6, 7, and 8

MDA COI “Spiral 1” was started in February 2006 and completed in only eight months

So far information in our case study is based upon status of the MDA program circa June, 2007. At that time there were still some challenges to be faced.

14


See Notes Page


Distribution of CASE Activities/Sub-Activities

15


Epilogue

Since completion of Spiral 1 in late 2006, MDA COI completed

– Spiral 2: three new service lines were developed by early 2008

Self-reporting vessel transmissions were augmented with information from unclassified Navy reference set

Position reports over time enabled vessel tracking

Motion- and/or behavioral-based anomalies were reported

– Spiral 3: two additional service lines by late March 2009

Advanced Notice of Arrival (ANOA)

Single Integrated Look-Out (SILO)

MDA COI is on transition path to provide long-term sustainment and governance for its activities

16


See Notes Page


Lessons Learned

What worked

– Finding senior and credible decision makers to serve as “top-cover” sponsors to stimulate action from constituents

– Collaborating within COI to drive and accomplish change

– Being proactive

– Motivating database owners and providers to standardize data

– Reducing cultural barriers against information sharing

– Discussing, understanding, and embracing outcome spaces

– Pursuing multiple competing but co-existing approaches

– Giving early priorities to achieving better gain-to-cost ratios

– Facilitating incremental improvements without large infusions of funding that can be targeted by others

– Rewarding investments with beneficial results, e.g., enhancing capabilities through network centric services

What did not work

– Refer to last paragraph of Section VI of paper.

17

See Notes Page


18

Putting It All Together…

1. Characterize

Your Environment

3. Apply CASE Methodology

Legend: Segments that are on

The Main Path

An Alternative Path

Architect a

Strategy

2Create Climate

for Change

1

Reward Results

4Target Outcome

Spaces

3

Formulate

Decision-Making

Heuristics

5

Stimulate Natural

Processes

6

Assess, Learn,

and Re-Plan

8

Develop in

Operational

Environs

7

2. Characterize Your

Current Approach

4. Characterize Your

New Approach



Takeaways

Complex Adaptive Systems Engineering (CASE)

– Is offered as a complementary approach (to conventional or traditional SE) that may work better in our most difficult Government acquisition environments

– Try it, you may like it!

– But CASE needs more case studies to help validate this approach and convince skeptics

19


See Notes Page



20

List of References [1] Z. Turnin, “Maritime domain awareness—a case study in cross-boundary information

sharing among the United States Navy, Coast Guard, and Department of

Transportation,” Case 07-2007, Leadership for a Networked World

Program/Executive Education, John F. Kennedy School of Government, Harvard

University, 2007; http://www.defenselink.mil/cio-nii/docs/MDA.pdf

[2] B. E. White, “Complex adaptive systems engineering,” 3rd Annual IEEE International

Systems Conference, Vancouver, British Columbia, Canada, 23-26 March 2009

[3] B. E. White, “Complex adaptive systems engineering,” 8th Understanding Complex Systems

Symposium, University of Illinois at Urbana-Champaign, IL. 12-15 May 2008;

http://www.howhy.com/ucs2008/schedule.html

[4] R. Stevens, “Profiling complex systems,” IEEE Systems Conference, Montreal, Quebec,

Canada, 7-10 April 2008

[5] B. E. White, “Systems engineering activity [sic] (SEA) profiler,” 8th Conference on Systems

Engineering Research (CSER), Hoboken, NJ, 17-19 March 2010

[6] B. E. White, “Enterprise opportunity and risk,” INCOSE Symposium, Orlando, FL, 9-13 July

2006; http://tinyurl.com/6om9r8

[7] B. M. Freeman, “A COI Success Story: Maritime Domain Awareness Data Sharing

Community of Interest (MDA DS COI),” Briefing, U.S. Department of Defense,

Assistant Secretary of Defense, Chief Information Officer/Networks & Information

Integration (CIO/NII), 5-6 June 2007; http://www.defenselink.mil/cio-nii/, Enabling Net-

Centric Operations, [email protected]


http://www.defenselink.mil/cio-nii/docs/MDA.pdf



http://www.howhy.com/ucs2008/schedule.html

http://tinyurl.com/6om9r8

http://www.defenselink.mil/cio-nii/



mailto:[email protected]


Backup



22

Complex Adaptive Systems Engineering

Conventional SE is insufficient and sometimes counterproductive, in addressing the most difficult SE problems.

As an alternative that may work better, we offer a Complex Adaptive Systems Engineering (CASE) methodology.

– Create Climate for Change: Create a climate for engineering the environment of the System. Continually plan for agile, constructive change (accelerating the processes of natural evolution) through proactive dialog with stakeholders, especially customers.

– Architect a Strategy: For the System, within its various system, system of systems (SoS), enterprise, and/or complex system contexts.

– Target Outcome Spaces: Describe the customer’s mission/vision in terms of one or more desired outcome spaces, not solutions.

– Reward Results: Work with the customer and a governing body to create appropriate incentives.


See Notes Page



23

Complex Adaptive Systems Engineering (Concluded)

CASE (concluded)

– Formulate Decision-Making Heuristics: Discover and promulgate management heuristics that will help the customer better know how and when to make decisions.

– Stimulate Natural Processes: Continually “stir the pot” by introducing variation (innovation) and selection (integration) while shaping and enabling future constructive change, and trying to avoid chaos and stasis, respectively.

– Develop in Operational Environs: Create a bias for developing evolutionary improvements of the System in actual operational environments with real users.

– Assess, Learn, and Re-Plan: Continually evaluate overall results and trends focusing on the “big picture,” and revisit all the above activities in an iterative fashion to improve their application.


See Notes Page



Activities 1 and 2: Engineer System’s Environment

Create a Climate for Change (Activity 1)

Architect a Strategy (Activity 2)

24

The Garden Metaphor


See Notes Page



Activities 3 and 4: Engineer System Itself

Target Outcome Spaces (Activity 3)

Reward Results (Activity 4)

25


See Notes Page



Activities 5, 6, and 7: Direct System Interactions

Formulate Decision-Making Heuristics (Activity 5)

Stimulate Natural Processes (Activity 6)

Develop in Operational Environs (Activity 7)

26


See Notes Page



Activity 8: Rinse and Repeat

Assess, Learn, and Re-Plan (Activity 8)

27


See Notes Page



Activity 1: Create Climate for Change

Convince government organizations and leaders (e.g., customers, and other System stakeholders) to adopt a self-organizational approach to creating solutions.

– Understand customers’ environments.

– Pursue a learning process.

– Together suggest potential policy changes.

– Identify and approach those who might adjust policies, formulate new policies, or mandate changes.

– Work with other stakeholders to surface issues, harmonize mutual interests, and propose solutions.

28


See Notes Page



Activity 2: Architect a Strategy

With customers and other System stakeholders, determine how to engineer an environment that enables the System to evolve well.

– Discuss, define, and analyze the

Nature of the problem

System boundaries

Desired outcome spaces

Relevant organizations

Potential stakeholders

– Decide What and Whom to control (if possible) or influence, and How.

– Include and induce the activation of a governing body.

– Keep options open.

29


See Notes Page



Activity 3: Target Outcome Spaces

Describe and share (as widely as possible) the customers’ or users’ mission and vision in terms of desired outcome space(s), including specific goals.

– Describe them in ways that are

Clear, succinct, and compelling

Oriented toward (mostly qualitative) expressions of outcome space capabilities

Devoid of specific (mostly quantitative) solutions

– Continually adapt and reshape the outcome spaces.

30


See Notes Page



Activity 4: Reward Results

Work with System stakeholders and a governing body to

– Establish incentive structures that motivate developers to realize desirable outcomes more rapidly.

– Judge outcomes that ensue, and reward contributors in proportion to how well the mission is satisfied.

– Publicize the rewards with supporting information on what was accomplished and why.

31


See Notes Page



Activity 5: Formulate Decision-Making Heuristics

Discover management heuristics that improve decision-making processes.

– Discuss potential decisions with stakeholders.

– Jointly assess if enough information exists to make such decisions, and take appropriate action otherwise.

– Support the stakeholders as they take action.

– Observe and record System behavior.

– Share useful heuristics with others.

32


See Notes Page



Activity 6: Stimulate Natural Processes

Continually stir the pot seeking to further innovate and integrate.

– Encourage frequent interactions to foster competition and cooperation among System constituents.

– Manage uncertainty considering opportunity and risk.

– Design, propose, conduct, and evaluate new concepts.

33


See Notes Page



Activity 7: Develop in Operational Environs

Develop evolutionary System improvements with users in operational surroundings and circumstances.

– Emphasize safety.

– Participate in field experimentation.

– Use laboratories for prototyping and subsystems.

34


See Notes Page



Activity 8: Assess, Learn, and Re-Plan

Evaluate overall results, revisit CASE activities, and alter the methodology as appropriate.

– Focus on understanding surprises.

– Adjust your strategy.

– Refine CSE principles.

– Record lessons learned and document case studies.

– Celebrate successes

35


See Notes Page


Documents

Fundamentals of Enterprise Systems Engineeringcau-ses.net › ... › 07 › Maritime-Awareness-Presentation.pdf© 2010 The MITRE Corporation. All rights reserved. 1 Brian E. White,