Systems of Systems: Characteristics and Challenges

Barry Boehm, boehm@usc.eduUSC Center for Systems & Software Engineeringhttp://csse.usc.edu October 25, 2006

2

Overview

Definitions, Examples & Motivation

SoS Characteristics and Challenges

Conclusions

3

What is a “System-of-Systems”? Very large systems developed by creating a framework or architecture

to integrate component systems SoS component systems independently developed and managed

– New or existing systems Some outsourced Some externally evolving

– Have their own purpose– Can dynamically come and go from SoS

SoS exhibits emergent behavior not otherwise achievable by component systems

SoS activities often planned and coordinated by a Lead System Integrator (LSI)

Typical domains– Business: Enterprise-wide and cross-enterprise integration to support core

business enterprise operations across functional and geographical areas– Military: Dynamic communications infrastructure to support operations in a

constantly changing, sometimes adversarial, environment

4

What Does an SOS Look Like: Future Combat Systems

6

The Need for Net-Centric Systems of Systems (NCSOS)

Lack of integration among stove-piped systems causes– Unacceptable delays in service– Uncoordinated and conflicting plans– Ineffective or dangerous decisions– Inability to cope with fast-moving events

Increasing NCSOS benefits– See first; understand first; act first– Network-centric operations coordination– Transformation of business/mission potential– Interoperability via Integrated Enterprise Architectures

7

Overview

Definitions, Examples & Motivation

SoS Characteristics and Challenges

Conclusions

8

SoS Characteristics and Challenges - I

Complexity: Size and number of organizations, interfaces, suppliers, coordination groups– Scalability of processes, methods, and tools

Dynamism: Number of changes/month, average time to process change– Rapid change impact analysis, change synthesis,

negotiation, development, validation, implementation Emergence: requirements not pre-specifiable Build-to-spec processes, contracts infeasible

– C2ISR a better metaphor for SoS acquisition than purchasing-agent

– Command, control, intelligence, surveillance, reconnaissance

9

Complexity of SoS Solution Spaces

Size: 10-100 MLOC Number of external interfaces: 30-300 Number of “Coopetitive” suppliers: 20-200

– Even more separate work locations Depth of supplier hierarchy: 6-12 levels Number of coordination groups: 20-200

– Reviews, changes, risks, requirements, architecture, standards, procedures, technologies, -ilities, integration, test, deployment, personnel, infrastructure, COTS,…

– Key personnel spend 60 hours/week in meetings Unprecedentedness Emergence Rapid change

Necessarily software-intensive…

10

Average Change Processing Time: 2 SoSs

Average workdays to process changes

020406080

100120140160

WithinGroups

AcrossGroups

ContractMods

11

Acquisition C2ISR Via Spiral OODA Loops

Life Cycle Architecture Milestone for Cycle

Decide on next-cycle capabilities, architecture upgrades, plans

• Stable specifications, COTS upgrades• Development, integration, V&V, risk management plans• Feasibility rationale

Act on plans, specifications

• Keep development stabilized• Change impact analysis, preparation for next cycle (mini-OODA loop)

Orient with respect to stakeholders priorities, feasibility, risks

• Risk/Opportunity analysis• Business case/mission analysis• Prototypes, models, simulations

Observe new/updated objectives, constraints, alternatives

• Usage monitoring• Competition, technology, marketplace ISR

Operate as current system

Accept new system

Example: ARPANet/Internet Spiral

12

SoS Characteristics and Challenges - II

COTS/Reuse/Legacy diversity– Many sources of incompatibility, changes– COTS: average 8-10mo/release; unsupported after 3 releases

Multiple missions and stakeholders to support– Increment and change content requires negotiation

Independently evolving systems– Often with “coopetitive” suppliers, interoperators

More time needed for systems definition– Before and after source selection

More time needed for teambuilding, partner coordination, supplier management, change management , integration and test

13

How much Architecting is Enough: COCOMO II Analysis

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60

Percent of Time Added for Architecture and Risk Resolution

Perc

ent o

f Tim

e A

dded

to O

vera

ll Sc

hedu

le

Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution

Added Schedule Devoted to Rework(COCOMO II RESL factor)

Total % Added Schedule

10000KSLOC

100 KSLOC

10 KSLOC

Sweet Spot

Sweet Spot Drivers:

Rapid Change: leftward

High Assurance: rightward

14

SoS Integrationand Testing

Size Drivers• # SoS interface protocols• # SoS scenarios• # unique component systems

Cost Drivers• Requirements understanding• Architecture maturity• Level of service requirements• SoS team capability• Maturity of LSI processes• Tool support• Cost/schedule compatibility• SoS risk resolution• Component system maturity and

stability• Component system readiness

COSOSIMO: I&T Sub-Model

LSI I&TEffort

15

Conclusions

Individual SoS attributes are highly challenging– Complexity, dynamism, emergence, uncontrollables,

stakeholder diversity– Their combinations are even more challenging

Acquisition management and negotiation skills are at least as important as systems engineering skills– C2ISR a better metaphor for SoS acquisition than

purchasing-agent

More time needed for systems definition– Before and after source selection

Backup Charts

17

Complexity of Solution Spaces- Breadth, Depth, and Length

Platform N

• • • Platform 1

Infra

C4ISR

Command and ControlSituation AssessmentInfo FusionSensor Data ManagementSensor Data IntegrationSensorsSensor Components:

2008 2010 2012 2014 2016

…1.0 2.0 3.0 4.0 5.0

Width

Length

Depth

DOTMLPF

Legend: DOTMLPF Doctrine, Organization,

Training, Materiel, Leadership, Personnel, Facilities

C4ISR Command, Control, Communications, Computers,

Intelligence, Surveillance, and Reconnaissance

18

Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May 2004

1. Acquisition management and staffing2. Requirements/architecture feasibility3. Achievable software schedules4. Supplier integration5. Adaptation to rapid change6. Quality factor achievability and tradeoffs7. Product integration and electronic upgrade8. Software COTS and reuse feasibility9. External interoperability10.Technology readiness

19

$100M

$50M

Arch. A:Custommany cache processors

Arch. B:ModifiedClient-Server

1 2 3 4 5

Response Time (sec)

Original Spec After Prototyping

Available budget

Effect of Unvalidated Requirements-15 Month Architecture Rework Delay

20

Effect of Unvalidated Software Schedules

Original goal: 18,000 KSLOC in 7 years– Initial COCOMO II, SEER runs showed infeasibility – Estimated development schedule in months for

closely coupled SW with size measured in equivalent KSLOC (thousands of source lines of code):

– Months =~ 5 * 3√KSLOC- KSLOC 300 1000 3000 10,000

- Months 33 50 72 108

•Solution approach: architect for decoupled parallel development;Schedule As Independent Variable (SAIV) process

21

The SAIV* Process Model

1. Shared vision and expectations management2. Feature prioritization3. Schedule range estimation and core-capability determination

Top-priority features achievable within fixed schedule with 90% confidence

4. Architecting for ease of adding or dropping borderline-priority features And for accommodating past-IOC directions of growth

5. Incremental development Core capability as increment 1

6. Change and progress monitoring and control Add or drop borderline-priority features to meet schedule

*Schedule As Independent Variable; Feature set as dependent variable. Also works for cost, schedule/cost/quality as independent variable.

22

Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May 2004

1. Acquisition management and staffing2. Requirements/architecture feasibility3. Achievable software schedules4. Supplier integration5. Adaptation to rapid change6. Quality factor achievability and tradeoffs7. Product integration and electronic upgrade8. Software COTS and reuse feasibility9. External interoperability10.Technology readiness

23

COTS UpgradeSynchronization and Obsolescence Many subcontractors means an increasing number of evolving COTS

interfaces Aggressively-bid subcontracts can deliver obsolete COTS

– New COTS released every 8-9 months (GSAW)– COTS unsupported after 3 releases (GSAW)– An actual delivery: 120 COTS; 46% unsupported

Emphasize COTS interoperability in source selection process Develop contract/subcontract provisions/incentives to ensure

– Consistency and interoperability across contractor and subcontractor-delivered COTS software products

– Such products are recent-release versions Develop a management tracking scheme for all COTS software products in

all NCSOS software elements Develop a strategy for synchronizing COTS upgrades

24

Emerging Scalable Spiral Process- For 21st century systems engineering and acquisition

The C4ISR Metaphor for NCSOS Acquisition– Role of OODA loops– Example: Internet Spiral– Example: FCS Win Win Spiral Model

Risk-Driven Scalable Spiral Model– Increment view– Life-cycle view– Role of anchor point milestones

Acquisition management implications People management implications

25

Risk-Driven Scalable Spiral Model:Increment View

Increment N Baseline

Rapid Change

High Assurance

Short, Stabilized Development of Increment N

Increment N Transition/O&M

Short Development Increments

Stable Development Increments

Foreseeable Change (Plan)

Increment N Baseline

Rapid Change

High Assurance

Short, Stabilized Development of Increment N

Increment N Transition/O&M

Short Development Increments

Stable Development Increments

Foreseeable Change (Plan)

26

Risk-Driven Scalable Spiral Model:Increment View

Increment N Baseline

Future Increment Baselines Rapid Change

High Assurance

Agile Rebaselining for Future Increments

Short, Stabilized Development of Increment N

V&V of Increment N

Increment N Transition/O&M

Current V&V

Short Development Increments

Future V&V Stable Development Increments

Continuous V&V

Concerns Artifacts

Deferrals Foreseeable Change (Plan)

Resources Resources

Increment N Baseline

Future Increment Baselines Rapid Change

High Assurance

Agile Rebaselining for

Short, Stabilized Development of Increment N

V&V of Increment N

Increment N Transition/O&M

Current V&V

Short Development Increments

Future V&V Stable Development Increments

Continuous V&V

Concerns Artifacts

Deferrals Foreseeable Change (Plan)

Resources Resources

Unforseeable Change (Adapt)

27

Risk-Driven Scalable Spiral Model:Life Cycle View

SystemInception

SystemElaboration

Agile DI2 (OO&D) Rebaselining

Plan-Driven DI1 Construction (A)

DI1 V&V

Plan-Driven DI2 Construction (A)

DI2 V&V

System LCA System, DI1 LCA DI2 B/L LCA

DI2 LCA

Changes

LCA: Life Cycle ArchitectureIOC: Initial Operational CapabilityOO&D: Observe, Orient and DecideV&V: Verification and ValidationDI: Development IncrementB/L: Baselined

28

Risk-Driven Scalable Spiral Model:Life Cycle View

SystemInception

SystemElaboration

Agile DI2 (OO&D) Rebaselining

Plan-Driven DI1 Construction (A)

DI1 V&V

Agile DI3 (OO&D) Rebaselining

Plan-Driven DI2 Construction (A)

DI2 V&V

Agile DI4 (OO&D) Rebaselining

Plan-Driven DI3 Construction (A)

DI3 V&V

DI1 Trans’n

DI1 Usage

DI2 Trans’n

DI2 Usage

DI3 Trans’n

DI3 Usage

System LCA

DI3 LCA

System, DI1 LCA DI2 B/L LCA DI3 B/L LCA DI4 B/L LCA

Update

Update

Update

DI2 LCA

Changes

Changes

Changes

DI4 LCA

. . .

. . .

DI1 IOC

DI3 IOC

DI2 IOC

LCA: Life Cycle ArchitectureIOC: Initial Operational CapabilityOO&D: Observe, Orient and DecideV&V: Verification and ValidationDI: Development IncrementB/L: Baselined

29

LCO (MS A) and LCA (MS B) Anchor Points Pass/Fail Criteria A system built to the given architecture will

– Support the operational concept– Satisfy the requirements– Be faithful to the prototype(s)– Be buildable within the budgets and schedules in the

plan– Show a viable business case– Establish key stakeholders’ commitment to proceed

LCO: True for at least one architectureLCA: True for the specific life cycle architecture; All major risks resolved or covered by a risk management plan

30

Spiral Feasibility Rationale Deliverable

LCO, LCA reviews not just UML/PowerPoint charts Need to show evidence of product and process feasibility Evidence provided by prototypes, production code, benchmarks,

models, simulations, analysis– Sizing and cost/schedule model results for process feasibility

Evidence provided in advance to LCO/LCA review team– Key stakeholders, specialty experts

Lack of evidence risks destabilizing the process– Needs coverage by viable risk mitigation plan

Key new progress metric– Feasibility evidence progress vs. plans

31

Acronym Definitions

B/L BaselinedC4ISR Command, Control, Communications,

Computers, Intelligence, Surveillance, and Reconnaissance

CCD Core Capability Drive-ThroughCOTS Commercial Off the ShelfCRACK Collaborative, Representative, Authorized,

Committed, KnowledgeableDI Development IncrementDODAF Department of Defense Architectural

FrameworkDOTMLPF Doctrine, Organization, Training,

Materiel, Leadership, Personnel, FacilitiesERP Enterprise Resource PlanningFEAF Federal Enterprise Architectural

FrameworkGSAW Ground System Architectures WorkshopIESG Internet Engineering Steering GroupIETF Internet Engineering Task ForceIKIWISI I’ll Know It When I See ItIOC Initial Operational Capability IPT Integrated Product TeamIRR Inception Readiness Review

LCA Life Cycle ArchitectureLCO Life Cycle ObjectivesLSI Lead System IntegratorMLOC Million Lines of CodeMS MilestoneNCSOS Net-Centric System of SystemsOO&D Observe, Orient, and DecideOODA Observe, Orient, Decide, ActPM Person-Month/Program ManagerPRR Product Release ReviewSAIV Schedule As Independent VariableSE System EngineeringSoS System of SystemsSOW Statement of WorkSW SoftwareSys Engr System EngineeringV&V Verification and ValidationWMI War-fighter machine interface