Architecting a System of Systems Responding to Maritime

Architecting a System of SystemsResponding to Maritime DomainTerrorism by Orthogonal ArrayExperiment& Thomas Huynh, Brian Connett, Jared ‘‘Chewey’’ Chiu-Rourman, Jennifer Davis, Andrew Kessler,

Joseph Oravec, Michael Schewfelt, and Shaunnah Wark

AbstractIn this work we solve the problem of architecting a conceptual, cost-effective, near-term system

of systems (SoS) to respond to terrorist threats to the United States emanating from the maritime

domain. The threats include a weapon of mass destruction smuggled on a container ship, a com-

mandeered ship used as a weapon, and small boats used by terrorists to attack maritime commerce

traffic and critical shore infrastructures. We formulate the problem as an assignment problem, which

is then solved using the orthogonal array experiment. The optimality of the resulting SoS architectureis validated against a heuristically developed architecture and an optimal effective, but not necessar-

ily cost-effective, architecture obtained also with the orthogonal array experiment approach. The

principal results of the orthogonal array experiment method reported herein underline this successful

exploratory work in architecting an SoS. This method can be extended to architecting of other sys-

tems of systems.

IntroductionBased on previous terrorist attacks (Eberhart

2003; Fritelli 2003) and directly derived from

threats and threat scenarios described in the

Homeland Security Council’s Planning

Scenarios document (Howe 2004), three

commonly postulated terrorist attacks in the

maritime domain are (1) using a standard

cargo container to smuggle a weapon of mass

destruction (WMD) such as a nuclear device

into the United States for a terror attack,

(2) commandeering a merchant ship and

using it as a weapon against targets in the

US coastal harbors and ports, and (3) ramming

small boats packed with explosives into

US ships or coastal and shore critical

infrastructure.

Whereas there have been no reported precedents

to support a WMD attack, using a standard

cargo container to smuggle a nuclear device into

the Unites States to be detonated is known as a

viable means employed by terrorists (Eberhart

2003; Fritelli 2005).

Historical precedents of ships used as weapons

(SAW) and their destructive power lend support

to commonly postulated SAW attacks (Eberhart

2003). Exploded in the harbor of Halifax,

Nova Scotia, on December 6, 1917, the French

ammunition ship Mont Blanc killed approxi-

mately 1,900 people, injured 9,000 others, and

damaged or destroyed 1,600 buildings (Kitz

1989). On April 16, 1947, due to an onboard

fire, the French ammonium nitrate carrier

Nomenclature:C2: Command and Control

C4ISR: Command, Control,Communications, Computers,Intelligence, Surveillance,and Reconnaissance

CG: Guided Missile Cruisers

CIP: Common IntelligencePicture

COP: Common OperatingPicture

COTS: Commercial Off theShelf

DoD: Department of Defense

DoE: Department of Energy

DDG: Guided Missile De-stroyers

F/F: Find and Fix

FFG: Guided Missile Frigates

FIN: Finish

FRC: Fast Response Cutter

HPGe: High-Purity Germa-nium

LCS: Littoral Combat Ship

LRM: Linear Radiation Moni-tor

JTO: Joint Technical Opera-tions

HVU: High-Value Unit

MMG: Medium Machine Gun

MTR: Maritime Threat Re-sponse

NaI: Sodium Iodide

NPS: Naval PostgraduateSchool

O&S: Operating and Support

PACAREA: Pacific Area

PBS: Prepare the Battlespace

PoR: Program of Record

Ps: Probability of Success

RDT&E: Research, Develop-ment, Test, and Evaluation

RID: Radioisotope Identifier

T E C H N I C A L P A P E R

& 2009, American Society of Naval Engineers

DOI: 10.1111/j.1559-3584.2009.01142.x

2009 #1&79

S.S. Grandcamp exploded at the pier in Texas

City, TX, killing 581 and injuring 5,000; the

resulting fire destroyed two additional merchant

ships near the Grandcamp and burned the city

for a week, not to mention that the blast threw

the ship’s 3,000-pound anchor over two miles

(Stephens 1996). On March 27–28, 1942, in

an extensive special operations mission, the

World War I–era destroyer HMS Campbelltown

(ex-USS Buchanan) with four tons of explosives,

disguised as a German gunboat, entered the

St. Nazaire harbor under intense fire, as its

deception was uncovered by the enemy, from a

number of large-caliber guns, taking multiple

hits and suffering numerous personnel casual-

ties, and rammed into the St. Nazaire dry dock at

18 knots, exactly as planned (Smith 2003;

McRaven 1993). On May 9, 1980, in the midst

of dense fog and thunderstorms, the bulk carrier

Summit Venture hit one of the supports of the

Sunshine Skyway Bridge, a 15-mile cantilever-

truss bridge connecting St. Petersburg and

Bradenton, FL, and caused a 1,300-foot section

of the bridge to fall into Tampa Bay, killing

35 people (Mair 1982). On May 26, 2002, a tug

and barge hit a bridge portion of Interstate 40

over the Arkansas River, collapsing a 600-foot

section of the bridge, killing

14 and injuring 5 (NTSB 2004).

A small boat attack (SBA) by terrorists is the

most likely future attack because bombing of

public transportation, suicide or otherwise, is

the most common form of terror attack. The

number of terrorist attacks on transportation is

too numerous to be included here; only the most

recent attacks are cited. On October 12, 2000,

the detonation of a terrorist suicide boat packed

with high explosives severely damaged the USS

Cole, killing 17 and wounding 39 sailors (CRS

2001); the cost of repairing the ship was approx

imately $250 million (Perl and O’Rourke 2001).

On October 6, 2002, the French oil tanker M/V

Limburg suffered a similar attack 3 nautical

miles from the coast of Yemen, spilling an esti-

mated 90,000 barrels of oil and breaching both

hulls of the ship’s double-hull design (Howland

2004). Finally, two separate ferry bombings in

the Philippines in 2004 and 2005 killed over 100

people (Villanueva 2004; Pareno 2005).

To aid the United States in combating these ter-

rorist threats against its coastal harbors and

ports, enormous amounts of resources have been

allocated to obtaining intelligence on impending

attacks by these threats (Wilson 2004). Even

with intelligence availability, fighting such ter-

rorist attacks, however, remains a challenge, in

the sense of how to effectively organize, equip,

and train the US forces to thwart a planned at-

tack by these terrorists. There is thus a need to

develop a system of systems (SoS) to respond to

those terrorist attacks. An SoS is a composite

system that is composed of component systems,

each of which serves organizational and human

purposes and may be locally managed and opti-

mized independently, or nearly so, of the

objectives to be met by the composite system

(Sage and Cuppan 2001). We shall call the SoS

developed with the objectives to respond to

those terrorist attacks the maritime threat re-

sponse (MTR) SoS.

In this work, a conceptual, near-term, cost-

effective SoS architecture is developed to

respond to three attacks aimed at the San

Francisco Bay and to do so with minimal impact

on commerce and economic cost. By a concep-

tual SoS we mean a concept, not an actual SoS; it

is merely a concept proposed in an academic

project.1 As defined in the Data Analysis section,

the cost-effective measure is a dimensionless

quantity normalized through the range of costs

and performance. The near-term MTR SoS will

consist of systems that are currently in service, in

development, and commercial off-the-shelf

technologies or systems that would be available

and/or could be developed within the next 5

years. The systems that constitute the MTR SoS

include hardware, software, and human re-

sources, which, as will be seen later in the paper,

account for the total SoS cost. The MTR SoS ar-

SBA: Small Boat Attack

SOP: Standard OperatingProcedures

US: United States

USCG: US Coast Guard

USV: Unmanned Surface Ve-hicles

VAMOSC: Visibility and Man-agement of Operating andSupport Costs

WHEC: High Endurance Cut-ter

WMD: Weapon of Mass De-struction

WMSL: Maritime SecurityCutter

1The SoS architectures proposed in this paper do not rep-

resent the views of the Naval Postgraduate School, the USNavy, or the Department of Defense.

NAVAL ENGINEERS JOURNAL80 & 2009 #1

Architecting an SoS Responding to Maritime Domain Terrorism

chitecture is fixed—in the sense that the systems

that form the SoS remain unchanged. It is

intended to be a single, simplistic SoS architec-

ture that will be used to counter the three attacks

aimed at the San Francisco Bay. In countering

these attacks, the MTR SoS and its concepts

of operations must satisfy two requirements:

(1) meet and defeat the threats as early as

practicable and (2) operate with minimum im-

pact on commerce.

Huynh et al. (2007) solve the problem of archi-

tecting a conceptual, cost-effective, near-term

SoS to respond only to a SBA. In this work, we

solve the problem of architecting a conceptual,

cost-effective, near-term SoS to respond to all

three attacks—a container ship carrying a WMD

, a large commercial ship used as a weapon

(SAW), and an SBA. As in Huynh et al., due to

the scope of this paper, the concepts of opera-

tions of the MTR SoS will not be discussed here.

The target of the three attacks is the San Fran-

cisco/Oakland major metropolitan area, known

as the Bay area, which has numerous features that

make it an attractive target for terrorist attacks.

Millions of people live in the Bay area and millions

of visitors and tourists visit it annually. The Bay

area has the second-largest container port in Cal-

ifornia, which is the fourth largest in the nation.

The combined ports of San Francisco, Oakland,

and Richmond receive an average of 10 interna-

tional merchant vessels daily, primarily crude oil

tankers and container ships (Young 2005).

Numerous points of critical infrastructure

also exist in or near the San Francisco Bay. Con-

necting San Francisco to the Marin peninsula,

the Golden Gate Bridge is one of the nation’s

premier landmarks and one of the most famous

bridges in the world. The San Francisco–

Oakland Bay Bridge is a vital economic connec-

tion between San Francisco and Oakland. There

are other large public transportation systems

and hubs, such as two large airports, numerous

ferries, rail lines, and three other bridges of

significant size. Any response to a maritime

threat that would curtail or stop transportation

in the Bay area would have significant economic

impact estimable in billions of dollars (CQ 2002;

Arnold et al. 2006). In addition, a large explo-

sion, fire, or chemical cloud at the highly visited

Fisherman’s Wharf waterfront tourist area

(approximately 10 million visitors in 2004) has

the potential for mass casualties and the ‘‘cine-

matic’’ effect terrorists pursue. Furthermore, the

Bay area is also relatively isolated from large

military concentration areas, such as the West

Coast naval assets in San Diego and Seattle. The

main assets for immediate maritime defense are

therefore US Coast Guard (USCG) units already

in the Bay area. Even USCG PACAREA and

District 11 assets are spread from the Oregon to

Mexican borders. Assistance from the other as-

sets may thus be several days in arriving in the

San Francisco Bay.

In the WMD scenario, some information con-

cerning the origin and the time of departure of a

smuggled nuclear WMD is available, but the in-

formation is insufficient and not specific enough

to allow identification of a single vessel that car-

ries the WMD. As available intelligence

information received does allow the search to be

focused on 20 merchant ships inbound daily to

the Bay area, a group of 20 potential attacking

vessels must therefore be of suspect and subject

to identification (Kessler et al. 2006). Two types

of nuclear devices are postulated:

(1) a nuclear device that uses a significant

amount of either enriched uranium (4 25 kg)

or Plutonium-239 (greater than 8 kg); or (2) a

radiological dispersion device or ‘‘Dirty Bomb,’’

which is composed of a small amount of Cesium-

137, Americium-141, Strontium-90, or Cobalt-

60 wrapped in approximately 100 pounds of

conventional explosive. Any of the two devices,

shielded by either a 0.64–5.08-cm-thick square

lead container or a 128-cm layer of high-density

nitrogen, is placed, undetected by port authori-

ties or the originating company, in a cargo

container on one of the 20 container vessels that

have departed a common Far East port within a

24- to 48-hour period for the United States. No

terrorists are assumed to be onboard the ship to

help ‘‘shepherd’’ the device to its destination.

NAVAL ENGINEERS JOURNAL 2009 #1&81

Oblivious to its presence, the ship’s crews and

owners are expected to cooperate with friendly

forces when approached.

In the SAW scenario, a team of terrorists on-

board the ship in a legitimate capacity—

operating and navigating the vessel and/or per-

forming other tasks—would need to seize

control of the vessel at the last possible moment.

Presumably, there are 10 terrorists, 2 for each of

five major control stations: bridge, engineering

control, after steering, and two engine rooms.

The terrorists are armed and will defend against

boarding by MTR forces. If the ship is boarded

(or an attempt to board is made) by friendly se-

curity forces, the terrorists will offer armed

resistance and will seize control of the ship if

they have not already commandeered it.

The SBA scenario involves protection of five oil

tankers inbound to the Bay area and 13 ferries

operating on five different routes. The arrival of

the five oil tankers is uniformly distributed over

a 24-hour period; the ferries operate 12 out of

every 24 hours. The scenario also requires the

constant protection of points of critical infra-

structure representing strategic targets such as

oil offload terminals, pipelines, power facilities,

and so forth. The attacker uses a single 30-foot

civilian speedboat with a top speed of 40 knots.

The speedboat is loaded with 1,000 pounds of

conventional explosives.

An MTR mission includes searching and detect-

ing the threat, neutralizing the detected threat,

and supporting and maintaining the MTR units.

To execute the WMD, SAW, and SBA missions,

an MTR SoS performs five top-level functions:

(1) Command, Control, Computers, Communi-

cations, Intelligence, Surveillance, and

Reconnaissance (C4ISR); (2) Prepare the Battle-

space; (3) Find/Fix Threat; (4) Finish Threat;

and (5) Sustain. The C4ISR function ensures

that the SoS has the appropriate means to carry

out a mission in terms of command and control

and to have appropriate communication chan-

nels to keep the MTR forces informed of the

status of operations.

The Prepare the Battlespace function ensures

that the SoS has the appropriate personnel,

equipment, and platforms to carry out the mis-

sions; it also renders the area of operations ready

for countering a potential attack. The Find/Fix

and Finish functions (the word ‘‘Threat’’ being

left out for convenience) are executed as the

MTR forces actually carry out the missions. The

Sustain function ensures that all units and

equipment are properly supported and main-

tained for the duration of operations. As the

system concepts for Sustain are unique (Kessler

et al. 2006), system concepts corresponding only

to the first four top-level functions are identified

for use in an MTR SoS, and associated concepts

of operations are assessed for cost, applicability,

and utility. As will be seen later, some functions

may be supported by as many as four different

system concepts, whereas the others by as few as

two concepts. We do not suggest that the system

concepts considered here form a complete set of

system solutions. Rather, they are potential ‘‘best

fit’’ solutions associated with the top-level SoS

functions. The ‘‘best fit’’ solutions are meant to

be the solutions that result from a combination

of the operational experience of the SEA-9 class

and the information collected from its research.

Simulations of the selected system concepts

along with the various concepts of operations

are conducted to determine the effectiveness of

an SoS in each individual scenario and in all the

scenarios combined. The different missions

performed by an MTR SoS are not necessarily

congruent, but they all affect the architecture of

the MTR SoS.

As in Huynh et al. (2007), the focus of this

paper2 is the methodology employed to develop

architectures of a conceptual, cost-effective,

near-term MTR SoS to respond to the three

maritime-domain threats and to do so with

2This focused area is only a part of a major campus-wide,

integrated systems engineering and analysis project car-ried out at the Naval Postgraduate School by the academic

year 2006 SEA class, known as SEA-9, whose members

are all but the first author of this paper, who is their ad-visor.

NAVAL ENGINEERS JOURNAL82 &2009 #1


minimal impact on commerce and economic

cost. By cost effectiveness we mean maximum

performance at a minimum cost; as defined in

the Data Analysis section, the cost-effective

measure is a dimensionless quantity normalized

using the range of costs and performance.

The MTR SoS will consist of systems that are

currently in service, in development, and com-

mercial off-the-shelf technologies or systems

that could be developed within the next 5 years

(Kessler et al. 2006).

Intelligence is always needed before a response is

carried out, and the MTR SoS needs and makes

use of intelligence. Due to the limited scope of

this work, we consider any intelligence system

such as national, military, and human assets to

be external to the MTR SoS; the boundary of the

MTR SoS does thus not enclose these intelli-

gence assets, but intelligence produced by them

can be received and used by the MTR SoS. The

MTR SoS itself, however, has its own internal

intelligence provided to the components within

the SoS boundary; the nature of the internal in-

telligence will be discussed later. For practical

reasons, we also assume an MTR SoS solution to

be free from any political and jurisdictional

issues that can potentially exist.

The use of specially constructed tables known as

orthogonal arrays for designing robust products

and processes was originally espoused by

G. Taguchi in Japan in the 1950s and 1960s.

Taguchi (1978, 1993), Taguchi and Wu (1980),

and Roy (1990) contain a detailed description of

the Taguchi method. It has been employed in-

creasingly in many American industries, such as

AT&T, ITT, Xerox, and Ford. As in Huynh

(1997), Huynh and Gillen (2001), and Huynh

et al. (2007), our work here is a novel applica-

tion of the orthogonal array experiment to an

SoS architecting problem. The orthogonal array

experiment is extremely efficient and, for this

class of problems, provides optimal SoS archi-

tecture design. Optimality is ensured by the

Taguchi method (Taguchi 1978, 1993;

Taguchi and Wu 1980; Roy 1990). Parentheti-

cally, the orthogonal array experiment and the

Taguchi method are used interchangeably in

this paper.

The scope of this paper emphasizes the mechan-

ics of applying the Taguchi method to solve the

MTR SoS architecting problem. We will there-

fore confine our discussion of the Taguchi

method to the architecting problem at hand,

without delving into all the statistical back-

ground and details of the Taguchi method as it

applies to the quality control problem (Taguchi

1978; Taguchi and Wu 1980; Roy 1990; Bendell,

Disney, and Pridmore 1989).

Our goals in this paper are (1) explain our

exploratory work in applying the orthogonal

array experiment to solve system architecting

problems, which we treat as assignment prob-

lems; (2) delineate the mechanics of solving the

system architecting problems or assignment

problems of this kind and the procedures to

process experimental results; and (3) illustrate

our approach with the problem of architecting

an MTR SoS to respond to the terrorist attacks

involving a nuclear device smuggled on a con-

tainer ship, SAW, and small boats.

In the remaining of the paper, we define the

problem of architecting an MTR SoS for the

MTR missions and formulate it as an assignment

problem. We follow with a discussion of the ad-

aptation of the Taguchi method, which involves

selecting an appropriate orthogonal array, iden-

tifying the factors and their levels for the

problem at hand, running the orthogonal array

experiments, and then analyzing the experimen-

tal results to obtain an optimal cost-effective SoS

architecture. To verify that the resulting cost-

effective SoS architecture is indeed a ‘‘best’’

architecture, we compare its performance and

cost with those of a heuristically developed

architecture and an optimal effective, but not

necessarily cost-effective, architecture obtained

also with the orthogonal array experiment ap-

proach. Parenthetically, a ‘‘best’’ architecture is

in the sense that it satisfies the required perfor-

mance while keeping the cost as low as possible.

We then end with some concluding remarks.


MTRSoSArchitectingas anAssignmentProblemWe first define the problem of architecting an SoS

to respond to the three maritime terrorist threats

and then formulate it as an assignment problem.

MTR SoS ARCHITECTING PROBLEM

As its system concepts are unique and common

to all architectures (Kessler et al. 2006), Sustain

will not be included in the formulation of the

MTR SoS architecting problem as an assignment

problem. Again, we do not suggest that the

system concepts considered here represent all

possible system solutions; rather, they are

potential ‘‘best fit’’ solutions associated with the

top-level SoS.

C4ISR—The C4ISR has four elements:

Command and Control (C2), Communicate,

Compute, and Provide Intelligence, Surveillance,

and Reconnaissance. The C4ISR system attri-

butes are span of control, command structure,

and the suite of communications and computing

tools employed. ‘‘Span of control’’ relates to the

size of the geographic region and the number of

operating units in the region under a single com-

mander’ control. The span of control can be

Area or Local. An Area commander controls the

forces to search and/or protect approximately

20 commercial ships across the Pacific Ocean as

well as within San Francisco Bay. A Local com-

mander controls the forces to protect a single

high-value unit (HVU).

A command structure can be control-free,

selective control, mission oriented, problem

bounding, problem solving, objective oriented,

interventionist, or cyclic. In the highly distrib-

uted control-free structure, the commander

assigns missions to his subordinates, who then

employ all the assets available to them to

accomplish their missions. In a selective-control

structure, the higher command issues mission

orders, expects subordinates to take broad and

deep initiatives, follows the battle in detail, and

remains prepared to intervene. In a mission-

oriented structure, each command level assigns

missions to its subordinates and permits them to

define further details of the military situation,

beginning with selecting the objectives necessary

to accomplish the missions. In a problem-

bounding command structure, the higher

command composes its directives in terms of the

objectives to be accomplished but couches them

in very general terms. A problem-solving

approach involves issuing directives that articu-

late both missions and objectives for two levels

of subordinates and substantial guidance as to

how the objectives are to be achieved. An objec-

tive-oriented structure allows some level of trust,

creativity, and initiative in subordinate com-

mands, but it stresses synchronization of assets

and actions. An interventionist structure relies

heavily on central authority to issue directives,

but it also maintains very detailed information

about the battle and attempts centralized control

through detailed directives. The greatest degree

of centralization occurs in a cyclic structure in

which the senior command issues orders to all

subordinates on the basis of a preset cycle time.

In the maritime domain, the objective-

oriented structure is the most appropriate C2

structure option (Alberts and Hayes 1995).

As it incorporates the advantages of the prob-

lem-bounding and problem-solving structures, it

allows increased coordination and continuous

contact between superior and subordinate com-

mands as well as among subordinate commands.

The problem-solving approach will also be a C2

structure option, which, as a back-up structure,

would be used in the event of either net-centric

technology failure or lack of trust in either tech-

nology or subordinates. MTR communications

infrastructure must be near real time, transoce-

anic, and interoperable across local law

enforcement, National Fleet, and coalition

forces. The communications system must ensure

that messages, data, voice, and images ex-

changed between parties in the MTR SoS are

transmitted and received efficiently with mini-

mal delays. Internal communications take place

within a small group, task force, or agency;

external communications refer to communica-

tion links among all MTR actors. Whereas the

internal communications rely on local area



networks, the external communications employ

wireless networks and paging systems.

The two main computing system components

are information assurance and data fusion.

Information assurance refers to the ‘‘technical

and managerial measures designed to ensure the

confidentiality, possession or control, integrity,

authenticity, availability and utility of informa-

tion and information systems’’ (Answers.com

2006). The MTR information assurance system

will thus concentrate on protecting and securing

the systems and information within the MTR

domain. As information is being transmitted,

received, processed, and stored within the MTR

domain, the MTR system will employ encryp-

tion and authentication to protect information

against unauthorized access, hash the informa-

tion to protect it from modification without

notice, and implement system redundancy to

protect the missions from the loss of informa-

tion. To prevent the loss of information or

services to the commanders, redundant systems

are included for disaster recovery. The Defense

in Depth Security Model (Harrison 2004) is the

guiding framework for the MTR information

assurance system concept.

Data fusion must enable a high level of situa-

tional awareness while minimizing the chance of

information overload. Data/information certi-

fied as authentic from the trusted external

sources is then processed and correlated based

on the set of rules and requirements provided by

the commanders. A hybrid data fusion concept

employs both rule-based and self-learned algo-

rithms.

The intelligence component of C4ISR sends the

entire fused common operating picture (COP) to

all operating units, the entire fused COP blended

with the common intelligence picture (CIP) to all

teams, and specific fused COPs blended with the

CIP to the appropriate teams.

Based on speed and cost, four selected C4ISR

system concepts have common Communicate,

Compute, and Provide Intelligence components

but different C2 components. The Communicate

concept is a combined Local Area Network

(LAN), Wireless Metropolitan Area Network

(WMAN), and Wide Area Paging (WAP). The

Compute concept is Defense in Depth and

Hybrid Data Fusion. The Provide Intelligence

concept is Customized COP and CIP. The C2

system concepts are Area Problem Solving, Area

Objective Oriented, Local Problem Solving, and

Local Objective Oriented.

Prepare Battlespace for WMD and SAW Mis-sions—Preparing the Critical Infrastructure and

activating Preplanned Operations Orders are,

for the WMD mission, primarily notifying the

Department of Energy (DoE) Joint Technical

Operations (JTO) Teams as well as other

existing chemical and biological specialists

throughout the Department of Defense (DoD)

and other various government and civilian

agencies, and, for the SAW mission, identifying

the critical structures and population centers

that could be affected and take the necessary

efforts to protect those areas to the greatest ex-

tent possible.

Assemble Forces will gather the forces such as

the special operations forces, marines, and

boarding teams specially trained to use search

equipment and to execute visit, board, search,

and seizure tactics. Deploy Forces identifies the

locations of the assets and platforms to be de-

ployed and their response capability. For both

WMD and SAW missions, the platforms would

need to be deployable for 49 days in order to

intercept multiple ships. This need quickly nar-

rows to the use of vessels from the DoD and the

Department of Homeland Security (USCG).

One-half of Navy ships in any location are

assumed to be available for surge operations at

any time. Grouped by availability, the system

concepts are then: (1) Current Ship Systems such

as US Navy Guided Missile Frigates (FFG),

Guided Missile Destroyers (DDG), Guided

Missile Cruisers (CG) and USCG High Endur-

ance Cutters (378-foot High Endurance Cutter

[WHEC]); (2) the Program of Record (PoR)

Littoral Combat Ship (LCS) and Maritime


Security Cutter (WMSL); and (3) Commercial

Off-The-Shelf (COTS) Modification (commer-

cial existing systems capable of modification to

meet the missions) such as a converted National

Steel and Shipbuilding Company Tote Orca

carrier that houses up to six smaller 118-foot

high-speed ‘‘Wally’’ boats. The current Ship

Systems (CG, DDG, FFG, and WHEC) and the

Program of Record Ships (LCS and WMSL)

need AOE (i.e., underway replenishment ship)

logistical support, whereas the COTS modifica-

tion option will be able to operate independently

for over 20 days.

PBS for SBA Mission—Four system concepts are

selected for the SBA mission: (1) two small

escorts, (2) two medium escorts, (3) two small

escorts and two medium escorts, and (4) high-

value vessel-based escort teams. A small escort is

a highly maneuverable, small boat (25–35 feet

long) with a top speed of 40 knots and a crew of

four or five. A medium escort is a larger craft

(80–150 feet long) with inboard engines, a top

speed of 35 knots, and a crew of 20. For exam-

ple, the US Navy 34-foot Dauntless Boat Units

and the 110-foot Coast Guard cutter are, re-

spectively, small and medium escorts. The

medium escorts has a longer endurance, which

affects the number of vessels required in an

overall force structure. The small escort mounts

one medium machine gun (MMG) in each posi-

tion. The medium escort has a medium caliber

gun only in the bow position and two MMGs on

the port and starboard positions. The team on-

board the HVU consists of six 2-man teams,

each armed with a light machine gun.

Find/Fix for WMD Mission—Find/fix of the

WMD mission involves conducting a search of

each container ship to determine if any of the

containers holds a nuclear device. Find/Fix

considers four system concepts: (1) the Linear

Radiation Monitor (LRM) and Fission Meter,

(2) the LRM and the High-Purity Germanium

(HPGe) Radioisotope Identifier (RID), (3) the

sodium iodide (NaI) detector and Mission

Meter (FM), and (4) the NaI detector and

HPGe RID.

A 24.4-meter long, self-contained g-ray

detector system for use in the interdiction and

location of nuclear materials, the LRM has

18 g-ray detectors and 9 neutron detectors on a

rope, with a control module at the operator end

for display and alarms. Lowered down

between the guide rails between individual

containers, the LRM detector system can simul-

taneously scan up to eight containers and allows

actual detector elements to be significantly

closer (on the order of 1.5–6 m). Placed in a

given cargo hold, the FM can collect neutron

data over extraordinarily long periods of time,

often necessary with lower energy emitting

nuclear devices that may be shielded. Being not

focused on any one particular container, the FM,

which collects neutron emissions throughout

the hold, can also search multiple containers.

An HPGe RID has superior performance in

identifying radionuclides in static conditions

(i.e., identifying the isotope once the source has

been located) (Keyser et al. 2005). Coupled with

a device capable of locating the source, it can be

highly successful in finding illicit materials. A

complement to the g-ray identifier (Rowland

2006), the FM detects neutron sources beyond

what would be expected from background.

Finally, a sodium iodide (NaI) detector is effec-

tive in detecting a point source, but its size and

weight limit its capabilities onboard a cargo

carrying container ship.

Find/Fix for SAW Mission—Find/Fix for the SAW

mission involves the only system concept—

searching for and identifying suspected

terrorists among the crew. Identification

mechanisms are fingerprinting, database

searches, and biometric data collection and

comparison. A biometric search system will

be feasible within the next 5 years and thus

satisfies the near-term requirement for emerging

technology (Hunton 2005).

Find/Fix for SBA Mission—Searching for

surface contacts during escort operations,

Find/Fix for the SBA mission involves visual

means (e.g., binoculars) or both radar and

visual means.



Finish for WMD Mission—If a nuclear device is

found on a vessel, specialized personnel from

the DoE will handle the response functions,

which remain out of the scope of the MTR SoS.

In unforeseen circumstances, or by order of DoE

or higher authority, a suspect vessel may need to

be sunk. An assumption of a cooperative mer-

chant and no terrorists onboard the vessel then

implies only one system concept for Finish:

Sink/Disable.

Finish for SAW Mission—Two SAW Finish

concepts are (1) escort potential attackers and

recapture seized vessels and (2) escort potential

attackers and disable seized vessels. A variety of

weapons and platforms could disable or sink a

large merchant vessel, such as command-

activated, deployable mines potentially to be

fielded in the 5-year timeframe.

Finish for SBA Mission—Added to the weapons

organic to the escorts are armed helicopters and

unarmed unmanned surface vehicles (USV). An

armed helicopter offers two potential benefits:

challenge suspicious small boat traffic and ‘‘clear

a path’’ for the HVU and offer additional

engagement capability. An unarmed USV offers

these capabilities: allow friendly forces to

physically impose themselves between suspi-

cious vessels and the HVU without risk to

personnel to deliver challenges and warnings

at greater distances from the HVU, shoulder

suspect vessels or ram identified targets, and

complicate enemy plans by forcing the enemy to

take action earlier than desired. Four SBA Finish

system concepts are (1) organic weapons only,

(2) organic weapons and armed helicopters,

(3) organic weapons and USVs, and (4) organic

weapons, USVs, and armed helicopters.

Table 1 summarizes the seven functions and

their associated system concepts. Note that the

numbers 1, 2, and 3 in the parentheses stand for

the WMD, SAW, and SBA missions, respectively,

and PBS, F/F, and FIN for Prepare Battlespace,

Find/Fix, and Finish, respectively. Both F/F(2)

and FIN(2) are supported by two system con-

cepts; PBS(1,2) is supported by two concepts;

and C4ISR, PBS(3), and FIN(3) all are carried

out by four systems concepts. An MTR SoS ar-

chitecture is a combination of these system

concepts. But, which pertinent system concept is

selected to perform a top-level function, so that,

together, the selected system concepts constitute

an optimal MTR SoS architecture? The optimal

architecture will maximize the probability of

mission success while minimizing the cost of de-

lay to commerce caused by the response to the

threats and the cost associated with the SoS.

Answering this question amounts to solving an

assignment problem to determine which perti-

nent system concept is assigned to a top-level

function, so that, put together, the assigned sys-

tem concepts result in an optimal MTR SoS

TABLE 1: System Concepts per Function Considered for SoS Architecture

Top-level

function

System concept

1 2 3 4

C4ISR Area—problem solving Local—problem solving Area—objective-

oriented

Local—objective-

orientedPBS(1,2) (WMD,

SAW)

AO—CG/DDG/FFG/

WHEC

AO—LCS/WMSL Modified COTS (Mer-

chant)PBS(3) (SBA) Small escorts Medium escorts Small & medium escorts HVU-based teamsF/F(1) (WMD) LRM & fission meter LRM & HPGe RID NaI RID & fission meter NaI & HPGe RIDF/F(3) (SBA) Visual means Visual means and radar — —FIN(2) (SAW) Escort potential attack-

ers & recapture seized

vessels

Escort potential attackers &

disable seized vessels

— —

FIN(3) (SBA) Organic weapons only Organic weapons & armed

helicopters

Organic weapons and

USVs

Organic weapons,

USVs & armed he-

licopters


architecture, in the sense that it maximizes some

objective function of performance and cost.

We now formulate the MTR SoS architecting

problem as an assignment problem.

MTR SoS ARCHITECTING AS AN ASSIGNMENT

PROBLEM

Let F denote the set of the seven SoS top-level

functions, Fj, j 5 1, . . ., 7, and Sj the set of system

concepts that can perform function Fj. Let X de-

note an allocation function defined according to

Xjk ¼1; if system concept k of Sj is assigned to Fj

0; otherwise

�

where j 5 1, . . ., 7, and k 5 1, . . ., |Sj|; |Sj| denotes

the number of elements (i.e., system concepts) in

Sj. As shown in Table 1, |S1| 5 |S3| 5 |S7| 5 4,

|S5| 5 |S6| 5 2, and |S2| 5 3.

Probability of Mission Success—An MTR

mission is declared successful if each of the three

terrorist attacks is successfully stopped, i.e., the

functions satisfy the mission success criteria (i.e.,

operational effectiveness requirements) spelled

out in Table 2 while minimizing delay and

impact on commerce. The mission success or

failure is related to the allocations of the system

concepts to the top-level functions; that is, it is a

function of Xjk. The probability of mission

success, Ps, is the fraction of the number of

Monte Carlo simulation runs in which the

missions are a success. It is therefore also a

function of Xjk. The probability of mission

success is obtained by Monte Carlo simulation.

The Monte Carlo method is a convenient and

useful method for obtaining solutions to the

problem at hand.

Cost—The total cost of an SoS architecture

depends on the allocations Xjk. It is contributed

by the cost of procurement of both additional

existing and new SoS components (platforms),

the cost of operating and supporting (O&S) the

SoS, and the cost associated with time delay

suffered by commerce while the SoS is respond-

ing to a potential attack, and the cost associated

with damage to the physical entities resulting

from failures of the SoS to neutralize the terrorist

threat. The delay cost is determined in terms

of time delay to commerce via a mission-level

EXTEND models described in (Kessler et al.

2006). (EXTEND is a modeling and simulation

tool developed by Imagine That Inc., San Jose,

CA.) The damage cost is determined from these

same models in terms of the percentage of sys-

tem failures to neutralize the terrorist threats. All

costs are in monetary units (FY2006$M).

For a given SoS architecture, whereas the cost

of procuring the SoS is fixed, the remaining costs

change with mission execution and are gener-

ated by Monte Carlo simulation (Kessler et al.

2006). The cost of procuring the SoS depends on

the number of platforms in the SoS architecture

and the cost of each platform. The platform

costs are obtained from official DoD budget

TABLE 2: Mission Success Criteria

Function

Mission

WMD SAW SBA

C4ISR Process time � 24 h Process time � 30 min (depend-

ing on intelligence latency)

Process time � 1 h

PBS Time to assemble teams

& deploy vessels o24 h

Time to assemble teams

& deploy vessels o24 h;

Alert team with Pilot

Immediately start clearing non-essen-

tial boats; Time to assemble crews &

deploy escort vehicles o1 hFind/Fix Time to search 9400 TEU ship

o160 h; Pd � 0.96, PFA �10� 6; Dwell time per container

� 3 min

Determine Potential Attack

Vehicle status upon boarding;

Search Potential Attack Vehicle

with escort teams

Detect incoming small boats at suffi-

cient range to allow warning, ID, and

two shots prior to vehicle approach;

Ps � 0.94Finish Transfer to DoE JTO Time to disable Potential Attack

Vehicle � 2 min; Ps � 0.91

Time to defeat attack � 15 s; � 0.94



documents (OUSD [Comptroller] 2004), Jane’s or

the original equipment manufacturer, and official

USCG materials for USCG Deepwater assets

(USCG Fact File 2004 and 2005); the costs of new

systems are obtained by modifying the costs of the

existing systems that are analogous to the new

systems. Parenthetically, PoR National Fleet as-

sets, such as the LCS and the National Security

Cutter (WMSL), are assumed to be sunk costs and

are therefore not included in the total cost compu-

tation. The O&S cost reflects the number of days

per year during which the platforms in the SoS

would be involved in MTR-related activities and

the daily O&S rate, which accounts for both the

system and personnel O&S rates. The latter are

obtained by adjusting the data drawn from the

Naval Center for Cost Analysis (NCCA) Visibility

and Management of Operating and Support Costs

(VAMOSC) database (NCCA), for the expected

amount of time during which the SoS would be

involved in annual MTR-related training, exer-

cises, and operations. If the needed VAMOSC data

do not exist, the existing analogous VAMOSC

data are used for cost scaling. The VAMOSC data

include all annual costs for personnel, mainte-

nance, fuel, and expendables. The average O&S

costs for selected classes of ships and aircraft form

the basis for MTR SoS platform O&S cost esti-

mates. Since the emphasis of this paper is the use of

the Taguchi method to optimize MTR SoS archi-

tecture, we will not delve into the details of the

estimation of the O&S cost, which can be found in

Kessler et al. (2006).

Objective Function—In this work, we introduce

a dimensionless objective function, z, which

results from mapping the performance measure

(Ps) and the total system cost (C) by means of a

rule r,

z ¼ rðPs; CÞ

The objective function z is thus a function of the

allocations Xjk. A specific rule r resulting in a

dimensionless objective function is elaborated in

the Data Analysis section.

The problem of optimizing the MTR SoS

architecture amounts to determining the alloca-

tions Xjk that maximize the probability of

mission success while minimizing the cost of

delay on commerce and the system cost. By

virtue of r, the problem becomes one of deter-

mining an assignment of the system concepts to

the seven SoS top-level functions that maximizes

the objective function z. We solve this problem

using the orthogonal array experiment approach

employed in Huynh (1997) and Huynh and

Gillen (2001).

TheOrthogonalArrayExperimentOrthogonal arrays have a mathematical founda-

tion in linear algebra, specifically, the Galois

field theory; they began with Euler as Latin

squares in the 18th century (Euler 1849). R.A.

Fisher was the first to apply them extensively.

Factorial design of experiments was first intro-

duced by Fisher in the 1920s. For a full factorial

design the number of possible conditions or ex-

periments is Lm, where m is the number of

factors and L is the number of levels for each

factor. Taguchi’s partial factorial design requires

only a smaller number of unique factor/level

combinations captured in an orthogonal array.

All combinations of levels occur an equal num-

ber of times in every pair of columns of an

orthogonal array. This combinatorial property

ensures the orthogonality property (Pao,

Phadke, and Sherrerd 1989); all columns in the

array are thus orthogonal to each other.

DEFINITION OF FACTORS AND LEVELS

A key step in designing orthogonal array exper-

iments is defining the factors and their levels. In

the Taguchi parlance (Roy 1990), factors are the

causes that produce an effect, levels are the way

in which the factors are changed, and the re-

sponse is the result produced by the factors. In

our approach, we identify the SoS top-level

functions as factors; thus, C4ISR, PBS(1,2),

PBS(3), F/F(1), F/F(3), FIN(2), and FIN(3) are

the factors. From here on, functions and factors

are used interchangeably. Care should be exer-

cised in choosing factors and levels

appropriately so as to take advantage of the es-

tablished orthogonal arrays. In general, there is

no systematic way to define factors and their


levels for assignment problems (Huynh 1997).

For the problem at hand, the levels of a function

(factor) are defined as the different system con-

cepts that can perform the function. The system

concepts supporting a function are thus the lev-

els of the function (factor).

THE ORTHOGONAL ARRAY EXPERIMENT

DESIGN

At the outset, for the problem at hand, an

experiment is a computer simulation. As

mentioned above, for a full factorial design the

number of possible combinations of these system

concepts, 3,072, need be evaluated for their

effectiveness, using Monte Carlo simulation.

Each simulation run takes more than 3 minutes

on an Intel Pentium (R) CPU 3.40 GHz Dell

computer employed in this work. It would

therefore take 704 days of continuous running of

the simulation to evaluate those potential com-

binations (architectures), with each combination

requiring 100 simulation runs. This would be

impractical.

The use of orthogonal arrays reduces the number

of experiments drastically. Taguchi and Wu

(1980) have tabulated a number of orthogonal

arrays that can be conveniently used to construct

orthogonal designs for any experimental situa-

tion. Given the number of factors (7) and levels

(2–4), the appropriate orthogonal array for the

problem at hand is a portion of the mixed L32

(21, 49) orthogonal array (Taguchi and Konishi

1987), shown in Table 3. The columns in the or-

TABLE 3: The Orthogonal Array Reduced from L32ð21; 49Þ

TRIAL C4ISR PBS(1,2) PBS(3) F/F(1) F/F(3) FINISH(2) FINISH(3)

1 1 1 1 1 1 1 12 1 2 2 2 2 2 23 1 3 3 3 1 1 34 1 1 4 4 2 2 45 2 1 1 2 2 1 36 2 2 2 1 1 2 47 2 3 3 4 2 1 18 2 2 4 3 1 2 29 3 1 2 3 2 1 210 3 2 1 4 1 2 111 3 3 4 1 2 1 412 3 3 3 2 1 2 313 4 1 2 4 1 1 414 4 2 1 3 2 2 315 4 3 4 2 1 1 216 4 1 3 1 2 2 117 1 1 4 1 2 2 318 1 2 3 2 1 1 419 1 3 2 3 2 2 120 1 2 1 4 1 1 221 2 1 4 2 1 2 122 2 2 3 1 2 1 223 2 3 2 4 1 2 324 2 3 1 3 2 1 425 3 1 3 3 1 2 426 3 2 4 4 2 1 327 3 3 1 1 1 2 228 3 1 2 2 2 1 129 4 1 3 4 2 2 230 4 2 4 3 1 1 131 4 3 1 2 2 2 432 4 2 2 1 1 1 3



thogonal array correspond to the factors (func-

tions). Each of the 32 rows or experiments (or

conditions) corresponds to an architecture trial.

For example, the seventh experiment (row) in-

volves the C4ISR locally controlled, problem

solving concept (system concept [level] 2), the

PBS(1,2) modified COTS (system concept [level]

3), the PBS(3) small and medium escorts option

(system concept [level] 3), the F/F(1) NaI and

HPGe RID (system concept [level] 4), the F/F(3)

visual means and radar (system concept [level]

2), the FIN(2) escort potential attackers and re-

capture seized vessels (system concept [level] 1),

and the FIN(3) organic weapons only (system

concept [level] 1). We now thus deal with only

32 experiments.

EXPERIMENT

Experimental Procedure—Carrying out an exper-

iment for each row of the orthogonal array

means performing a Monte Carlo simulation of

the MTR SoS response, z, to the three types of

attack. The Monte Carlo simulation involves

between 500 and 2,000 simulation runs of the

EXTEND MTR mission model, each of which

produces success or failure of the WMD, SAW,

and SBA missions. Again, the missions are de-

clared success if the mission success criteria in

Table 2 are satisfied. A statistical analysis of the

Monte Carlo simulation results then yields the

probability of success for the tree missions.

MTR Modeling and Simulation—MTR modeling

and simulation aid in the evaluation of the effec-

tiveness of the proposed SoS architectures. To

strictly evaluate the capability of the SoS, cost is

not included in the simulation. Cost will be ap-

plied later to determine affordability and the

possible need for trade-off considerations. In

addition to the EXTEND MTR mission model,

the modeling tools include also a Microsoft Ex-

cel model and the Fatigue Avoidance Scheduling

Tool (FAST) v1.0.26. Since this paper is focused

on the application of the Taguchi method, we

will not elucidate these models in detail. We will,

however, provide a cursory discussion of only

the MTR mission model. Regarding the Excel

and FAST model, we mention only that the level

of training and proficiency in FAST is assumed

to be at peak performance for all units and

personnel involved and that the Excel model

determines the probabilities associated with a

subfunction and a subsystem, which are then

used in the determination of the overall SoS

capability.

MTR Mission Model—The MTR mission model

captures the capabilities of the various subsys-

tems and equipment to be employed in the three

missions as well as the operating limitations of

the personnel and equipment. Also incorporated

in the model are the location and availability of

assets to be used in each mission and the relative

distances to be traversed in order to respond to

the maritime threat, the actual threat location,

vessel sizes, latency of information, actions of

the terrorists, random variation of search and

detection times, and the probabilities of detec-

tion, false alarm, and success. The MTR mission

model represents the C4ISR, PBS, Find/Fix, and

Finish functions. Constrained by the scope of the

paper, we will not describe the modules of the

MTR mission model. We also hasten to add that

the accuracy and fidelity of the models used in

the simulation affect the response of each exper-

iment, and, hence, the Taguchi analysis results.

Care must thus be taken to ensure the different

models in the simulation are accurate and have a

similar level of fidelity. Table 4 depicts the mod-

ules that represent the MTR SoS functions

carried out in each of the WMD, SAW, and SBA

three missions. We will not elaborate on the

modules, whose detailed descriptions can be

found in (Kessler et al. 2006). We briefly de-

scribe, however, as an example, the application

of the modules to the SBA mission. Four mod-

ules represent the sub-functions of the C4ISR

function—Receiving Communications, Com-

mand and Control, Compute, and Transmit

Communications. Two modules—Mission

Database and Initial Orders—represent the

PBS function. Finally, four modules represent

the sub-functions of the Finish function—

Small Boat Attacker Generator, Helicopter

Engagement, MTR Escorts or Teams Onboard

Engagement, and Delay to Commerce. Input to


the EXTEND MTR mission simulation also in-

cludes the L32 (21, 49) orthogonal array, the SoS

architecture alternatives with their components

and pertinent characteristics. Output from the

MTR mission model is mission success or failure

for each Monte Carlo run. Post-processing yields

the probability of mission success for each of the

32 trials (experiments). Table 5 displays the ex-

perimental results—the probability of mission

success (in the second column) and the total cost

(in the third column) for each of the 32 trials—

and the dimensionless response (in the fourth

column), which is defined in the section that

follows.

DATA ANALYSIS

As in Huynh (1997) and Huynh and Gillen

(2001), the data analysis performed here

consists of the standard analysis outlined by

Taguchi (Roy 1990). The purpose of the data

analysis is to study the main effects of each of the

factors (the SoS top-level functions) in order to

identify the optimal condition (Roy 1990).

The main effects indicate the general trend of

the influence of the factors, that is, the effect of

a factor (function) on the objective function

when it goes from one level (system concept)

to another. The analysis of the main effects

involves the calculation of the averages for the

levels of all factors. We now describe the data

analysis.

Let Ne be the number of experiments (rows), aij

denote the level of the column (function or fac-

tor) in the ithh

row (trial, experiment, or

condition), and z denote the response (i.e., the

objective function) corresponding to the ith row.

The architectural trial (row in the orthogonal

array) that yields the most expensive architec-

ture is assigned a score of 0 for cost, whereas the

trial that yields the least expensive architecture a

score of 100 for cost. These two extreme data

are then used in a linear function to obtain the

dimensionless cost scores of the remaining trials,

xi associated with the ith experiment (row),

according to

xi ¼100

xmax � xmin

ðxmax � CiÞ;

where xmax ¼ maxi2Ne

Ci, xmin ¼ mini2Ne

Ci, and Ne

TABLE 4: Application of MTR SoS Functional Modules to MTR Missions

Function Module

Mission

WMD SAW SBA

C4ISR Receiving communications X X X

Command and Control X X X

Compute X X X

Transmit communications X X XPBS Mission Database X X X

PAV generator X X

Ship intercept X X

Sea state generator X X

Initial orders XFind/Fix Container search X

Ship search and engagement XFinish Small boat attacker generator X

Helicopter engagement X

MTR escorts or teams onboard engagement X

Delay to commerce XSustain Ship fuel consumption X X

Watch team sleep analysis X X X

Small boat availability and reliability X

Helicopter availability and reliability X



denotes the number of experiments (rows). The

dimensionless cost score thus varies between 0

and 100.

Likewise, the trial that yields the highest proba-

bility of success is assigned a score of 100 for

effectiveness, whereas the trial that yields the

lowest probability of success a score of 0 for

effectiveness. These two extreme data are then

used in a linear function to obtain the dimen-

sionless effectiveness scores of the remaining

trials, pi associated with the ithh

experiment

(row), according to

pi ¼100

pmax � pminðPsi� pminÞ

where pmax ¼ maxi2Ne

Psiand pmin ¼ min

i2NePsi

. The

dimensionless effectiveness score varies between

0 and 100. In this work, the rule r that amal-

gamates the total cost, Ci, and the probability of

success, Psi, into a the dimensionless response zi

associated with the ith row is simply the sum of

the two resulting dimensionless scores,

zi ¼ xi þ pi

The dimensionless response zi thus varies

between 0 and 200.

Then the average performance (i.e., the objective

function) of the jth factor (function) at the athj

level, denoted by fjaj

D E, is calculated according to

fjaj

D E¼ 1

Njaj

XNe

i¼1

dðaij � ajÞzi;j ¼ 1; � � � ; 7

aj ¼ 1; � � � ; Sj

�� !

;

TABLE 5: Normalized Cost-Effectiveness Scores by Trial Number

Trial Probability of Success, Ps Cost ($M) Effectiveness score Cost score Cost-effectiveness score

1 0.84 290.3 55.5 55.5 151.32 0.90 1628.2 95.8 62.5 146.93 0.89 4100.5 62.5 0.9 82.64 0.80 174.1 34.5 98.7 133.25 0.86 309.9 65.6 95.3 160.96 0.92 1650.9 93.4 61.9 155.37 0.89 4077.8 81.4 1.4 82.88 0.84 123.0 55.1 100.0 155.19 0.92 1721.7 94.4 60.1 154.610 0.82 286.2 42.8 95.9 138.711 0.90 1365.0 85.4 69.0 154.412 0.86 4134.7 62.8 0.0 62.813 0.93 1699.0 100.0 60.7 160.714 0.82 266.6 46.4 96.4 142.915 0.87 1384.6 70.4 68.6 138.916 0.87 1850.2 71.1 56.9 128.117 0.75 246.8 10.4 96.9 107.418 0.92 1664.4 93.5 61.6 155.119 0.77 4065.7 21.7 1.7 23.420 0.87 184.7 67.3 98.5 165.821 0.73 239.1 0.0 97.1 97.122 0.93 1714.4 98.6 60.3 158.923 0.84 4015.7 54.6 3.0 57.524 0.90 1377.5 85.3 68.7 154.125 0.86 1785.2 66.5 58.6 125.126 0.79 215.3 30.7 97.7 128.427 0.85 1434.4 61.4 67.3 128.828 0.87 1765.4 69.3 59.1 128.429 0.89 1735.2 79.4 59.8 139.230 0.74 223.1 6.6 97.5 104.131 0.88 1426.7 74.4 67.5 141.932 0.87 1752.4 71.6 59.4 131.0


in which Njaj, the number of experiments (rows)

the athj level (system function) assigned to the jth

factor (function), is

Njaj¼XNe

i¼1

dðaij � ajÞ;

and

d aij � aj

� �¼

1; if aij ¼ aj

0; otherwise

�

As an example, consider the Fix/Find function

for the WMD mission (F/F1). The F/F1 function

can be carried out by four system concepts—

LRM and Fission Meter, LRM and HPGe RID,

NaI RID and Fission Meter, and NaI and HPGe

RID (Table 1). In this case, j 5 4, |S4 5 4|, a4 5 1,

2, 3, and 4, and

N4a4¼ 8;

in all cases, Ne 5 32. From the expression of

fjaj

D Eabove, the average performance of the F/

F1 at Level 1 (LRM and Fission Meter) is 140.

It is roughly 130 at Level 2 (LRM and HPGe

RID), 115 at Level 3 (NaI RID and Fission

Meter), and 125 at Level 4 (NaI and HPGe RID).

These results are captured in the graph corre-

sponding to F/F1 in Figure 1. Since the larger the

better is the quality characteristic, Level 1 (LRM

and Fission Meter) is selected to provide the F/F1

function.

Reflecting the data analysis results in Table 5,

Figure 2 displays the graphs of the average

performance fjaj

D Eagainst the levels (system

concepts) for reach factor (function). It depicts

the main effects of the factors (functions) on the

overall objective function. The selected level of a

factor corresponds to the largest the average

performance.

OPTIMAL, COST-EFFECTIVE MTR SoS

ARCHITECTURE

Figure 1 shows that f14h i , f22h i , f31h i , f41h i ,

f52h i , f61h i , and f74h i are the largest values

among, respectively, the values f1a1

� �, f2a2

� �,

f3a3

� �, f4a4

� �, f5a5

� �, f6a1

� �, and f7a7

� �. In other

words, the maximum average performances are

obtained with factor 1 at level 4, factor 2 at level

2, and factor 3 at level 1, factor 4 at level 1, fac-

tor 5 at level 2, factor 6 at level 1, and factor

7 at level 4. This means that, as displayed in

Table 6, the optimal cost-effective MTR SoS

architecture consists of locally controlled,

objective-oriented approach for C4ISR, LCSs

and Maritime Security Cutters supported by

oil tankers for PBS(1,2), small escort boats

only for PBS(3), LRM and Fission Meter for

Mea

n of

CO

ST

-EF

F

432

140

120

100321 4321

432

140

120

10021 21

432

140

120

100

C4 PBS1,2 PBS3

F/F1 F/F3 FIN2

1

1

1

FIN3

Figure 1: Main Effects of System Conceptson Overall SoS Cost Effectiveness



FF(1), visual look-out backed up by radar

search for FF(3), escort potential attackers and

recapture seized vessels for FIN(2), and organic

weapons, armed helicopters, and USV support

for FIN(3).

Figure 2 graphically depicts the physical, func-

tional, and operational views of the optimal,

cost-effective MTR SoS architecture employed

to counter the three terrorist attacks—WMD,

SAW, and SWBA. The SoS components (in up-

percase) reflect those in Table 6. The top-level

SoS functions (in lowercase) are also included.

The ‘‘lightning strokes’’ depict the communica-

tions among the SoS components.

Comparisonof ResultsTo verify that the resulting cost-effective SoS ar-

chitecture is indeed a ‘‘best’’ architecture, we

compare its performance and cost with those of

two additional architectures, namely, a maxi-

mum-performance SoS architecture and a

heuristic cost-effective SoS architecture. The

Taguchi method is also employed to develop the

TABLE 6: Cost-Effective MTR SoS Architecture

SoS Function

System Concept

1 2 3 4

C4ISR Locally controlled, objec-

tive oriented approachPBS(1,2) (WMD,

SAW)

Littoral combat ships & maritime security

cutter supported by oil tankersPBS(3) (WMD,

SAW)

Small escorts only

F/F(1) (WMD,

SAW)

Linear radiation monitor &

fission meterF/F(3) (WMD,

SAW)

Visual Look- out backed by Radar Search

FIN(2) (WMD,

SAW)

Escort potential attackers &

recapture seized vesselsFIN(3) (WMD,

SAW)

Organic weapons, USVs &

armed helicopters

Figure 2: Optimal, Cost-Effective MTR SoSArchitecture


maximum-performance architecture, but with

the objective function being the probability of

mission success; the cost is not considered. The

heuristic cost-effective SoS architecture is based

on the collective experience of the SEA class

(Kessler et al. 2006) and consists of the lowest

cost system concepts that would meet system

effectiveness requirements. Table 7 shows the

components of these two architectures along

with those of the optimal cost-effective SoS ar-

chitecture. Figure 3 shows the cost-effectiveness

curve, which depicts the cost of each architecture

against the probability of success across all three

missions. The ‘‘knee’’ in the cost-effectiveness

curve corresponds to the optimal cost-effective

architecture. The reason for the ‘‘knee,’’ and

hence the selection of this ‘‘best’’ architecture, is

now elaborated. First, the large difference be-

tween the cost of the maximum performance

architecture and the other two architectures is

caused by the cost of the procurement of the

requisite number of medium-sized escorts for the

TABLE 7: Allocation of System Concepts to Top-level Functions in the HeuristicCost-effective, Optimal Cost-effective and Optimal Effective SoS Architectures

Function Mission Option System concepts

Architecture

Heuristic

Optimal cost

effective

Optimal

effective

C4ISR 1 Area control/problem solving

WMD, SAW, & SBA 2 Area control/objective-oriented X

3 Local control/problem solving

4 Local control/objective-oriented X XPBS 1 AO – CG/DDG/FFG/WHEC

WMD & SAW 2 AO – LCS/WMSL X X X

3 Modified COTS (Merchant)

1 Small escorts X X X

SBA 2 Medium escorts

3 Small & medium escortsFind/Fix 1 LRM & fission meter X X X

WMD 2 LRM & HPGe RID

3 NaI RID & fission meter

4 NaI & HPGe RID

1 Visual means X X

SBA 2 Visual means & radar XFinish 1 Escort potential attackers & recapture seized vessels X X

SAW 2 Escort potential attackers & disable seized vessels X

1 Organic weapons only

2 Organic weapons & armed helicopters X

SBA 3 Organic weapons and USVs

4 Organic weapons, USVs & armed helicopters X X

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 500 1000 1500 2000Total SoS Cost (FY2006 $M)

Pro

babi

lity

of S

ucce

ss

Heuristic cost-effective

architecture Optimalcost-

effectivearchitecture

Maximum-performancearchitecture

Figure 3: SoS CostEffectiveness for Com-bined Missions



SBA mission. The cost for procurement alone of

the small escort only SBA force in the optimal

cost-effective architecture is $83.6 million. The

result is a 1,385% increase in the overall SBA

mission cost for only a 12% increase in SBA

mission effectiveness. Second, although the

optimal cost-effective architecture is slightly

more expensive than the heuristic cost-effective

architecture, it delivers 5% improvement in per-

formance (i.e., the mission probability of

success). The optimal cost-effective architecture

can be procured and operated at a cost of $315.1

million in FY2006 dollars. Table 8 shows the

cost estimated for the results of the three differ-

ent missions without the MTR system in place

as well as the expected value of damage cost

associated with the optimal cost-effective archi-

tecture in place. With the MTR system in place,

the expected value of damage suffered drops

from $1,900 million to $127 million. This drop

in the expected value of damage of $1,773 mil-

lion is obtained through the expenditure of only

$315.1 million in FY2006 dollars. In other

words, the procurement of the optimal cost-

effective architecture should save $5.63 for

every dollar spent.

ConclusionAs in Huynh (1997) and Huynh and Gillen

(2001), our exploratory work here establishes

the applicability of the orthogonal array experi-

ment (or the Taguchi method) to optimizing an

SoS architecture. In this work we develop the

architectures of a conceptual, cost-effective,

near-term SoS to respond to the terrorist threats

to the United States that emanate from the mar-

itime domain. To this end, we formulate this SoS

architecting problem as an assignment problem,

which is then solved using the orthogonal array

experiment. The orthogonal array experiment

approach allows us to solve this assignment

problem by carrying out the smallest possible

number of experiments and determining the

solution from the responses of the experiments.

It is efficient and, for this class of problems,

provides an optimal cost-effective architecture

for the MTR SoS. The results discussed in this

paper underline this successful exploratory

work in architecting an SoS. This method

can therefore be extended to other SoS

architecting.

AcknowledgmentsWe thank Ling Siew Ng, Seng Chuan Lim,

Cheng Lock Chua, Eng Choon Yeo, Kok Long

Lee, Heng Hui Chew, Kwang Yong Lim, Ee Shen

Tean, Sze Tek Ho, and Choon Chung for their

contribution to this MTR SEA-9 project at the

Naval Postgraduate School.

ReferencesAlberts, D.S. and R.E. Hayes, Command arrangements

for peace operations, Command and Control

Research Program (CCRP) Publications, National

Defense University, 1995.

Answers.com, ‘‘Information assurance,’’ http://

www.answers.com/, 2006, accessed April 2006.

Arnold, B., C. Cammarata, D. Farmer, K. Kowalewski, F.

Lapido, M. Lasky, D. Moore, ‘‘The economic costs of

disruption in container shipments,’’ Congressional

Budget Office, March 26, 2006.

Bendell, A., J. Disney, and W.A. Pridmore (Editors),

Taguchi methods: Applications in world industry, IFS

Publications/Springer-Verlag, New York, 1989.

TABLE 8: Probability of Mission Success & Damage Cost of Optimal Cost-EffectiveArchitecture

MTR mission

Optimal cost-

effective SoS,

Ps (%)

Raw damage

cost from

attacks ($M)

Relative prob-

ability of

occurrence (%)

Expected damage

without optimal

cost-effective SoS ($)

Expected damage

with optimal

cost-effective SoS ($)

WMD 99 500,000 0.001 1,000 10SAW 99 2,500 1 500 5SBA 72 1,000 2 400 112


Chu, W.W., ‘‘Optimal file allocation in a multiple

computer system,’’ IEEE Transactions on Computers,

Vol. 18, pp. 885–889, 1969.

CQ, ‘‘Port shutdown for terrorist incidents could cost

billions, drill shows,’’ CQ Homeland Security, Decem-

ber 5, 2002.

CRS, ‘‘Terrorist attack on USS Cole: Background

and issues for Congress,’’ Report RS20721,

March 2001.

Eberhart, D., ‘‘Container ships: the next terrorist

weapon?’’ http://www.newsmax.com, accessed April

2006; ‘‘Peril on the Sea,’’ The Economist, October 2,

2003.

Euler, L., ‘‘Recherches Sur une Espece de Carres

Magiques,’’ Commentationes, Arithmeticae Collectae,

Vol. II, pp. 302–361, 1849.

Fritelli, J.F., ‘‘Port and maritime security: Background

and issues for Congress,’’ CRS Report RL31733, US

Library of Congress, Washington, D.C., 2005.

Harrison, R., The antivirus defense-in-depth guide,

version 2.0, Microsoft TechNet, Seattle, WA, 2004.

Howe, D., ‘‘Planning scenarios,’’ Homeland

Security Council, July 2004.

Howland, J., ‘‘Hazardous seas/maritime sector vulner-

able to devastating terrorist attacks,’’ JINSA Online,

April 1, 2004.

Hunton, R., ‘‘A proposed model for the collection and

use of biometric identifiers obtained at sea as an effort

to prevent seaborne terrorist activity and enhance

security at the port of Charleston, South Carolina,’’

Master’s Thesis, Naval Postgraduate School, Monterey,

CA, March 2005.

Huynh, T.V., ‘‘Optimal file allocation in a distributed

computer network by orthogonal array experiments,’’

IEEE Aerospace Applications Conference Proceedings,

Vol. 4, pp. 105–114, 1997.

Huynh, T.V. and D.C. Gillen, ‘‘Dynamic bandwidth allo-

cation in a satellite communication network,’’ IEEE

Aerospace Applications Conference Proceedings, Vol. 3,

pp. 1221–1232, 2001.

Huynh, T.V., A. Kessler, J. Oravec, S. Wark, and

J. Davis, ‘‘Orthogonal array experiment for architecting

a system of systems responding to small boat

attacks,’’ Systems Engineering, Vol. 10, No. 3, pp.

241–259, 2001.

Kessler, A., M. Schewfelt, B. Connett, C. Chiurourman, J.

Oravec, S. Wark, J. Davis, L.S. Ng, S.C. Lim, C.L. Chua, E.C.

Yeo, K.L. Lee, H.H. Chew, K.Y. Lim, E.S. Tean, S.T. Ho, and

K.C. Chung, ‘‘Maritime threat response,’’ Report NPS-

97-06-004, June 2006.

Keyser, R.M., T.R. Twomey, and D.L. Upp, ‘‘An improved

handheld radioisotope identifier (RID) for both locat-

ing and identifying radioactive materials,’’ ORTEC, HPS

Midyear Meeting, January 2005.

Kitz, J.F., Shattered city: The Halifax explosion and the

road to recovery, Nimbus, Halifax, Nova Scotia, 1989.

Mair, G., Bridge down, Stein and Day, New York, 1982.

McRaven, W., ‘‘The theory of special operations.’’

Master’s Thesis, Naval Postgraduate School, Monterey,

CA, June 1993.

NCCA (Naval Center for Cost Analysis), ‘‘Visibility

and management of operating and support costs,’’

Database, http://www.navyvamosc.com

NTSB, US, ‘‘Towboat Robert Y. Love Allision with inter-

state 40 highway bridge near Webbers falls, Oklahoma,

May 26, 2002,’’ Highway/marine accident report NTSB/

HAR-04/05 by National Transportation Safety Board,

Washington, DC, 2004.

OUSD (Comptroller), Office of the Under Secretary of

Defense (Comptroller), DoD summary budget materi-

als/budget links, http://www.dod.mil/comptroller/

budgetindex.html

Pao, T.W., M.S. Phadke, and C.S. Sherrerd, ‘‘Computer

response time optimization using orthogonal array

experiments, ‘‘Taguchi methods, applications in world

industry, IFS Publications/Springer-Verlag, London,

1989.

Pareno, R.D., ‘‘Thirty injured in Philippines ferry bomb

attack,’’ Philippine Star, August 29, 2005.

Perl, R. and R. O’Rourke, ‘‘Terrorist attack on USS Cole:

Background and issues for Congress.’’ CRE Report

RS2072, 2001.

Rowland, M., CG-SMG-2 3.2.2, 3.2.3, 3.2.4, Lawrence

Livermore National Laboratory, accessed 2006.

Roy, R., A primer on the Taguchi method, Van Nostrand

Reinhold, New York, 1990.

Sage, A.P. and C.D. Cuppan, ‘‘On the systems engineer-

ing and management of systems-of-systems and

federations of systems,’’ Information, Knowledge, Sys-

tems Management, Vol. 2, No. 4, pp. 325–345, 2001.



Smith, R.B., ‘‘British raid on St. Nazaire: The greatest

raid of all,’’ World War Two, 2003.

Stephens, H.W., The Texas city disaster, 1947, Universityof Texas Press, Austin, 1996.

Taguchi, G., ‘‘Off-line and on-line quality control

systems,’’ International Conference on Quality Control,Tokyo, Japan, 1978.

Taguchi, G., Taguchi on robust technology development:

bringing quality engineering upstream, ASME Press,

New York, 1993.

Taguchi, G. and S. Konishi, Orthogonal arrays

and linear graphs–tools for quality engineering,

American Supplier Institute Inc., Dearborn,MI, 1987.

Taguchi, G. and Y.I. Wu, Introduction to off-line quality

control, Central Japan Control Association, Meieki Nak-

amura Ku Nagaya, Japan, 1980.

USCG Fact File, Fiscal year 2004 Coast Guard report:

FY2003 performance report and FY2005 budget in

brief, http://www.uscg.mil/hq/g-cp/comrel/factfile/index.htm

Villanueva, M., ‘‘Superferry sinking last February a

terrorist act,’’ Philippine Headline News Online,

October 12, 2004.

Wilson, E. ‘‘Maritime domain awareness (MDA),’’

Stakeholder, July 12, 2004.

Young, R., ‘‘Baseline study of U.S. port merchant ship

traffic during 2004,’’ Office of Naval Intelligence, August

31, 2005.

AuthorBiographiesThomas Huynh is an associate professor of

systems engineering at the Naval Postgraduate

School in Monterey, CA. His research interests

include uncertainty management in systems en-

gineering, complex systems and complexity

theory, system scaling, simulation-based acqui-

sition, and system-of-systems engineering

methodology. Prior to joining the Naval Post-

graduate School, Dr. Huynh was a Fellow at the

Lockheed Martin Advanced Technology Center,

where he engaged in research in computer net-

work performance, computer timing control,

bandwidth allocation, heuristic algorithms,

nonlinear estimation, perturbation theory, dif-

ferential equations, and optimization. He was

also a lecturer in the Mathematics department at

San Jose State University. Dr. Huynh obtained

simultaneously a B.S. in Chemical Engineering

and a B.A. in Applied Mathematics from Uni-

versity of California, Berkeley and an M.S. and a

Ph.D. in Physics from University of California,

Los Angeles.

Brian Connett, an Information Warfare Officer,

is a graduate of the Officer Candidate School,

Surface Warfare Officer Division Officer

Course, Information Warfare School and a

recent graduate of the Naval Postgraduate

School’s Graduate School for Engineering and

Applied Sciences. While at Naval Postgraduate

School, he led an effort to analyze the nuclear

material detection capabilities of various tools

and the overall probability of actually locating

such materials on cargo ships in-transit across

the Pacific Ocean for a group project tasked

with developing architectures of a system of

systems to respond to maritime terrorist threats.

He served as the Communications Officer on the

USS Lake Erie CG-70 and as an Operations

Officer at Special Boat Team 20. He is currently

assigned to the Naval Information Operations

Center, Hawaii serving as the Operations

Officer and Senior Watch Officer for a joint

team with over 500 service members from the

every branch of the military. Before joining

the Navy, he worked several projects directly

related to the Y2K issues as a co-operative

education student. He holds a B.S. in Informa-

tion Systems from Drexel University, and an

M.S. in Systems Engineering & Analysis from

the Naval Postgraduate School in Monterey,

California.

Jared ‘‘Chewey’’ Chiu-Rourman, a Surface War-

fare Officer, is a graduate of the Officer

Candidate School and the Surface Warfare

Officer Division Officer Course and a recent

graduate of the Naval Postgraduate School’s

Graduate School for Engineering and Applied

Sciences. While at NPS, he led an effort to

analyze the logistic and operational availability


requirements for a group project tasked with

developing architectures of a system of systems

to respond to maritime terrorist threats. LT

Chiu-Rourman served as First Lieutenant on

USS Cormorant MHC-57 and as Assistant Plans

Officer in Commander Destroyer Squadron 15,

forward deployed in Yokosuka, Japan. In his

next assignment, he will be the Combat Systems

Officer on USS Hawes FFG-53, home-ported in

Norfolk, VA. Before joining the Navy, he was

enlisted in the Hawaii Army National Guard as

a Combat Engineer, attached to the 227th

Engineer Co. He holds a B.A. in Physics (minor

in Mathematics) from the University of Hawaii

at Hilo and an M.S. in Systems Engineering &

Analysis from the Naval Postgraduate School in

Monterey, California.

Jennifer Davis is a systems engineer at Northrop

Grumman Ship Systems in Pascagoula, Missis-

sippi, and a recent graduate of the Naval

Postgraduate School’s Graduate School for En-

gineering and Applied Sciences. While at NPS,

she led a C4ISR effort for a group project

tasked with developing architectures of a system

of systems to respond to maritime terrorist

threats. A former U.S. Navy helicopter pilot,

Ms. Davis has worked at Northrop Grumman

Ship Systems since 2001, specializing in the

ship/aircraft interface. She holds a B.S. in

Ocean Engineering from the U.S. Naval Acad-

emy and an M.S. in Systems Engineering

from the Naval Postgraduate School, Monterey,

California.

Andrew Kessler is an FA-18 Hornet pilot, a

Strike Fighter Tactics Instructor, and a recent

graduate of the Naval Postgraduate School’s

Graduate School for Engineering and Applied

Sciences. While at NPS, he was the student team

leader for a group project tasked with develop-

ing architectures of a system of systems to

respond to maritime terrorist threats. During his

career in the Navy, he has served in a number of

operational strike fighter squadrons and made

numerous deployments to the Western Pacific,

Mediterranean, and Indian Oceans as well as the

Arabian Gulf. LCDR Kessler was an instructor

at the Navy TOPGUN School, where he was the

subject matter expert on FA-18 radar systems

and the F-35 Joint Strike Fighter. He holds a B.A.

in Political Science from Yale University, an M.S.

in International Relations from Troy State Uni-

versity, and an M.S. in Systems Engineering from


California.

Joseph Oravec, a Surface Warfare Officer, is a

graduate of the Officer Candidate School and

the Surface Warfare Officer Division Officer

Course (with distinction) and a recent graduate

of the Naval Postgraduate School’s Graduate

School for Engineering and Applied Sciences.

While at NPS, he led an effort to analyze the

small boat attack for a group project tasked with


to respond to maritime terrorist threats. LT

Oravec served as a Fire Control Officer on USS

Stump DD-978 and as a Weapons Officer on

USS Whirlwind PC-11. In his next assignment,

he will be a Chief Engineer on USS Harper’s

Ferry LSD-49, forward deployed to Sasebo,

Japan. Before joining the Navy, he taught Amer-

ican Government as an adjunct instructor at St.

Petersburg College. He holds a B.A. with a dual

major in Political Science and International

Affairs (minor in Economics) from Florida State

University, an M.S. in Political Science Educa-

tion from University of South Florida, and an

M.S. in Systems Engineering & Analysis from


California.

Michael Shewfelt is a recent graduate of the

Naval Postgraduate School’s Graduate School

for Engineering and Applied Sciences. Marine

Corps MAJ Shewfelt holds a B.S. from the US

Naval Academy and an M.S. in Systems En-

gineering from the Naval Postgraduate School in

Monterey, California. [ROM]

Shaunnah Wark is a recent graduate of the Naval

Postgraduate School’s Graduate School for En-

gineering and Applied Sciences. While at NPS,

she earned the award for Most Valuable Team

Member of the 2006 SEA class tasked with




to respond to maritime terrorist threats. ENS

Wark graduated with distinction from the U.S.

Naval Academy in 2005, where she won the

Patriot League Championships in women’s

rowing and garnered awards of National Scho-

lar-Athlete, Patriot-League Scholar-Athlete of

the Year for Women’s Crew. She was also named

to the 2005 ESPN Magazine’s Academic All-

America Women’s At-Large Third Team. She

will serve as a surface warfare officer on the USS

Shoup (DDG 86). She holds a B.S. in Systems

Engineering from the U.S. Naval Academy and

an M.S. in Systems Engineering from the

Naval Postgraduate School in Monterey,

California.


Documents

Architecting a System of Systems Responding to Maritime