-
Global Journal ofFlexible Systems Management2009, Vol. 10, No.
4, pp 45-54
Impact of Identified Causal Factors to "System of
Systems"Integration Complexity from a Defense Industry
Perspective
Jessica JovelGraduate Student
Stevens Institute of Technology, Email: [email protected]
Rashmi JainAssociate Professor
Stevens Institute of Technology. Email:
[email protected]
Abstract
Multi-mission projects such as Mis.sile Defense Agency's (MDA)
Ballistic Missile Defense (BMD) program involve .systems withunique
capabilities developed by different companies to communicate
together. While .çy.ç/cm integration for an independent.•system is
a challenge in itself, integration for system of systems is a huge
complex effort in part due to interoperability challenges.The focus
of this paper is research, analyze, and prioritize some of the
causal factors of system integration complexity, such
asinteroperability, for complex .systems such as BMD.
Complex system integration is used interchangeably with system
of systems integration in this paper because currently in
thedefense industry, system of systems integration is the new
complex system integration. From the defense industry perspective,
anattempt is made to contribute in developing a cause and effect
relationship model between system requirements, system
architecture(specifically open system architecture), and system of
.systems integration process complexity.
Keywords: complexity, systems engineering, system of systems,
systems architecture, systems integration.
Introduction
In the defense industry, there are numerous systems that
arecomposed of subsystems that are developed by
differentcotnpanies. For example, in the Ballistic Missile
Defense(BMD) program, a ground-based radar developed byRaytheon
needs to communicate with a ship-based radardeveloped by Lockheed
Martin about the status of targetobjects such as missiles or enemy
aircrafts. Although this isa large-scale type system of systems
integration, smaller-scale integration efforts involve companies
subcontractingdifferent software or hardware elements and become
the|irime integrators of these elements.
Carney and Obemdorf acknowledge in their
presentation"Integration and Interoperability Models for Systems
ofSystems" that the current trend in the technological industryis
new systems are mostly expected to cooperate with othersystems.
This cooperation, which they term interoperability,is one of the
challenges systems integration face. Thecontinuous demand for
systems to work with other systemsmakes "system of systems"
"uncertainly unbounded", aconcept defined by Dr. Kaplan.
Uncertainly unbounded iswhen the customer or system contractor does
not know whenthe system will stop growing making it difficult to
havecontrol during pre-system integration. Not knowing when
thenetwork of systems will stop growing or changing makes
itdifficult to design an interoperable network of systems. Also,not
knowing when the system will be fixed from expanding
makes it difficult to design the system architecture to
reducesystem integration complexity. This is an issue that
defen.sesystems have faced for the past 20 years.
Defense system acquisitions made by the government arenow based
on the ability of a system to be affordable,dynamic, and support
multi-mission modes. Currently indefense products there are many
legacy systems that are inneed of upgrades but use unique
infrastructures that makeit a challenge to upgrade (Hanratty,
Lightsey & Larson1999). A change that many defense contractors
haveembraced is the approach to system design. Instead ofcreating
unique and closed system architecture, an openarchitecture approach
and using commercial-off-the-shelf(COTS) parts have become part of
system architecture design(Hanratty, Lightsey & Larson, 1999).
Using the open systemsapproach when developing a new system or an
upgrade oflegacy products reduces system integration complexity
inthe future for the system.
In 1999 an article about open system approach toweapon systems
was written by Hanratty et. al. Accordingto the authors, using an
open systems approach to systemarchitecture/design lessens the risk
of the product becomingobsolete by making room for upgrade
feasibility. Thedevelopment of defense related complex systems may
takeseveral years because of the politics, disagreements,changing
requirements, etc, (Hanratty, Lightsey & Larson1999). With
technology ever evolving, the risk of a technical
© 2009, Global Institute ofFlexible Systems Management
-
46 Jessica jovel and Rashmi jain
product becoming obsolete after the concept has beendesigned is
high in defense because by the time a system isin its production
phase new technology has been out foryears. When a system has been
designed to be an opensystem, the possibility of easily adding
functionality orperformance upgrades is high and at a more
affordable costwithout having to start from scratch; in addition
the systemlife-cycle is extended. i
One of the challenges that face open architecture isgrasping
where in the system it makes most sense to useCOTS, use
subcontractors for parts, and where some uniqueparts should be used
in order to decrease system integrationcomplexity (Hanratty.
Lightsey & Larson 1999). Systemattributes already defined
should then be used to gainknowledge and understanding of how to
design the systemarchitecture to become an open architecture design
and resultin decreased system integration complexity. Figure 1
showsthe relationship between system requirements, andarchitecture
design for legacy systemupgrades. It was developed to try tomake
system integration easier.
Identify rapidly changingtechnologies applicable tothe
system
Identify subsystems that arelikely to grow and evolveover the
course of thesystem's life
Identify high life cycle costdrivers
Establish a listof criticalinterfaces
iIdentify openstandards forall criticalinterfaces
JSystem Requirements Design synthesis and architecture
development
Almost 10 years later, systemintegration for legacy systems
still face challenges. Furtherresearch has been made into the cause
and effect relationshipof system integration complexity. The system
integrationprocess complexity is the "interaction between degree
offeasibility and level of effort required to understand,describe,
implement, manage and document the systemintegration process for a
given system development andoperation environment" (Jain, et al..
2008). Systemrequirements and system architecture have been
investigatedto determine which have most impact on system
integrationcomplexity. Concepts from the earlier research by the
secondauthor (Jain et al. 2(X)8) are used for this research. This
papersummarises our findings based on the application andextension
of the earlier research. A difference however issystem architecture
is considered for both hardware andsoftware systes and is studied
from mostly system engineer'sperspective instead of a principally
software standpoint.
The application of the framework (Jain et al. 2(X)8) inthis
research is toward defense systems, which as previouslymentioned
involve system of systems. The survey used inthe earlier research
was used as part of this paper's researchmethodology to capture the
perspective of defense industryengineers. The survey was given to
lead/chief engineers inthe defense industry who work in
areas
Current trend in the technologicalindustry is new systems are
mostlyexpected to cooperate with other systems.
Figure 1 : Relationship between System Requirements andSystem
Architecture for Legacy System Upgrades (Hanratty,I Ligbtsey &
Larson 1999)
Causal Factors Investigated
Exisitng literature indicates that one of the causal factorsthat
add to system integration complexity is the continuous
change of software or hardwareelements within the system such
asupgrades and additionallunctionality. As mentionedpreviously,
currently many projects
are becoming "system of systems" that are
"uncertainlyunbounded". There is no determination if the system
willbe expected to support other missions in the future
orcompletely change its purpose altogether. As a result,systems
need to be backward compatible to existing productsand forward
compatible to emerging technology.
Camey and Obemdorf propose in their presentation
thatsynchronicity of the system may help
interoperability.Synchronicity allows all parties (customers,
developers) tosee the rates at which subsystems undergo change such
asupdates, repair, and modifications in order to keep the
wholesystem synchronized. With synchronicity of all
elements,interoperability between individual systems is more
feasible.However synchronicity requires extensive documentation
toshow the growth of the modified system. The level ofdocumentation
of synchronicity may add to systemintegration complexity becau.se
of the effort needed.
During software/hardware subsystem contract awards,vendors are
selected based on price, product turnaround,availability, and
product technology features. Many vendorsclaim open architecture
capability however not all productsare completely based on open
architecture which many
times is manifested during systemdifferent engineering areas
such as ^^ open systems approach tO system nnegration and
verification &software, hardware, and systems io
architecture/design lessens the risk of validation. One problem is
thatacquire their perspectives on factors the product becoming
obsolete by vendors are not informed of otherthey think most impact
system making room for upgrade feasibility. vendor's products or
applicarionintegration complexity. The responseshelped produce a
cause and effect relationship betweensystem requirements, system
architecture and systemintegration complexity. The research
methodology alsoinvolved reviewing exisiting literature on system
of systemsintegration practices & from personal experience of
the authors
information. When the primecontractor uses producís from two
different vendors, theproblem lies in either interoperability
issues between thedifferent vendors or with the prime contractor's
subsystems.This interoperability between three systems in this case
addsanother layer to system integration complexity.
in system integration & test. One possibility to overcome
this issue is information
-
impact of identified Causal Factors to "System of Systems"
Integration Complexity from a Defense Industry... 47
sharing between vendors. During the design phase thesevendors
could communicate the types of standards usedwithin each element in
their product (since these vendorsin turn may sub-contract some
parts or materials), designintent for their applications,
schematics or high levelinformation on coding scheme. A team of
system integratorswithin the prime contractor may be needed to keep
eachvendor aware of some issues that may arise in their productwhen
integrating it to other vendor products. Knowledgeof this
information during the design phase infiuences thesystem
integration to run more smoothly. The desire totnaintain a
competitive advantage decreases vendors' interestin information
sharing,making it difficult for .1system to achieve
interfaceopenness. This causal factormay be difficult to
overcome.
During the requiremetiisphase in defining stakeholderneeds,
focus is given to the system's current capabilities thatwill define
the performance and constraints (Buede 2000).The National Society
of Professional Engineers modelconsiders the requirements phase to
be a part of the conceptdevelopment phase (Kossiakoff & Sweet
2003). The conceptdevelopment phase contains needs analysis,
conceptexploration, and concept definition. None of these
phasescontain future system capabilities as a task. As a result
notmuch discussion is centered on the future capabilities of
thesystem. One of the reasons may be because it may causemore
constraints or performance requirements that the primesystem
integrator may not desire.
When a system is uncertainly unbounded it isunderstandable that
not much focus is placed on futurecapability of the system. The
capability is unknown.However, the minimal communication on future
capabilitiesof a system may cause the interoperability challenges
in thefuture during system integration. Currently requirements
arebeing written in such a way that openness is achievedwhether by
commonality or
and littoral operations.
The system was designed to be open-architecture and useCOTS
modules including receiver and digital signalprocessor. If it's a
mix (customer, business developmentleaders, and engineers) then
perhaps all parties can think ofpotential uses for a system; such
as the case in recent newsabout a falling satellite being destroyed
by a missile defenseinterceptor designed for missile defense. It
was converted tohave anti-satellite capability. Instead of starting
from scratchthe missile interceptor was reprogramtned to complete a
taskit was not tested for.
When a system has been designed to be an opensystem, the
possibility of easily adding functionalityor performance upgrades
is high and at a moreaffordable cost without having to start from
scratch;in addition the system life-cycle is extended. Ä
It is difficult to fully knowthe environment in whichuncertain
unbounded systemswill function in order to drive the system
architecture. Dothe customers give this information or the lead
engineer ofthe system with business developtnent leaders or a mix
ofboth? In the case of the customer, he/she may have a limitedview
on the system since they are focused on satisfying aneed whether
it's large-scale or a simple task. On the otherhand, business
development leaders along with the lead/chief engineers tnay have a
bigger picture of future uses forsystetn therefore pitching to
customers a flexible system thatcould be used for different
missions. For example the SolidState S-band radar (S4R) system
developed at LockheedMartin is a system designed to be scalable to
supportmultiple missions, including air surveillance, cruise
missiledefense, ballistic rnissile defense, counter target
acquisition
factors that add to systemintegration complexity is the
continuous change ofsoftware or hardware elements within the
systemsuch as upgrades and additional functionality.
There are some instanceswhen systems are beingenhanced to
support morecomplex missions without riskanalysis. Of course
anytimenew software is added thereare potential conflicts
withexisting code. One way to
work around this is install software patches to the systemfor
issues that are found. These patches are somewhat abridge between
both codes. However, we run into theproblem where other parts of
the system may be using someof the patch code only to find that it
does not cooperatewith a subsystem code. These types of issues are
seen overand over again in current systems that are becoming
systemof systems. Either developing an abstract system
architectureor following standards when designing the
systemarchitecture helps this issue. For example, Prasad and
Rossakdefine four types of integration architectures (Rossak
&Prasad 1991). These types of architectures can be given
asguidelines for type of system architecture and flowed downto
vendors as which type to use. One drawback is theinfiexibility of
having to use a defined type; however witha system of systems this
would provide huge benefits fromreducing system integration
cotrtplexity. Integrating theAbstract Design Paradigm introduced by
Dogru et. al. to the
systems architecture/designphase is one way of
reducinginteroperability issues. In thisparadigm, the design
isviewed from an integrationperspective rather than inphases
(Dogru, Delcambr,
Bayrak, Christiansen & Tanik 1992),
Documentation for legacy systems is not always
detailed,coherent, or even complete. Proper documentation
isnecessary to flow down the technical infonnation to newengineers,
technicians, etc...trying to understand the existingtechnology to
cooperate with the new one. Many engineerswho worked on the legacy
system go onto new projects,different companies, without leaving
their knowledge indocumentation form. The cycle can continue to be
on-goinguntil proper documentation guidelines and rules
arefollowed. Other losses resulting from engineers who left anddid
not document their design understand the intent of theirwork. Why
did they do it that way and not this way? What
© 2009, Global Institute ofFlexible Systems Management
-
48 Jessica Jovel and Rashmi Jain
was the intent behind that? Knowing these details helpbetter
design the updated system architecture to integratewith the
existing. Interoperability, understanding of subsystemsand
documentation are all factors during integration.
A summary of the causal
focuses on causal factors that may have an impact tointegration
and test. A brief description of such factorsis given below.
1. Modularity according to Kossiakoff is a measure of thedegree
of mutual independence of
factors described so far as well as Jfie level of documentation
of synchronicity the individual system components,possible
solutions are listed in ffmy ddd ¡Q system integration complexity
According to Kossiakoff a systemTable 1. A survey was given to five
/.„^^„„p nf thp effort npeded should achieve a high degree
ofengineers to assess the system •^ "'•' ' modularity such as
makingintegration level of effort they perceive each solution to
each interfaces and interactions as simple as possible for
ea.secausal factor described in this paper will create. The
averagesof engineers' responses on the survey were calculated
foreach causal factor solution to get the resulting level of
effortduring system integration. A high level of effort for
systemintegration means an increase in system
integrationcomplexity.
Table 1 lists the phases of the system life-cycle
eacharchitecture issue and solution impacts as well as
engineers'responses to the level of effort needed for each
solution.
The purpose of the survey was to determine whetherpossible
solutions that have been described to mitigate theeffect of causal
factors to "system of system" integrationcomplexity add yet another
layer of integration complexity.Our results show that vendor
compatibility andinteroperability and defining a specific type of
systemarchitecture (whether DoDAF, object-oriented, etc..) for
allvendors to use does not involve a high level of effort. Bothof
these elements are part of system architecture design,which gives a
preliminary
of system integration and test. Physical modularity intheory
should make it easier (less complex duringsystem integration) to
add system upgrades by beingable to replace a unit or add another
unit onto theexisting system. However when different vendors
areresponsible for different modules system integration hasthe
potential of getting quite difficult in the vendor isunreliable or
unknown (stemming from ftrst time beingused, unfamiliar
technology)
2. Commonality according to this earlier research is theease of
ability to use a component or subsystem in morethan one place
within the system. This categoryincludes physical familiarity such
as percent of vendors,subcontractors known and hardware and
softwaretechnology known, and operational commonality suchas number
of unique skill codes required or estimatedoperational training
time.
3. Standards based system architecture is designed intclation to
open standards. In
assessment that system Proper documentation is necessary to flow
down ^^^ standards based systemarchitecture does result in the
technical information to new engineers, ¡tems such as
interface,increasing or decreasing system technicians, etc...trying
to understand the existing hardware, and softwareintegration
complexity, technology to cooperate with the new one. standards are
considered. Thewhichever case may apply. number of multiple
vendors
that manufacture the same product are considered aswell as
multiple business domains the system will be
System requirements, in the earlier research of the used for
whether medical, commercial or defense. Alsosecond author, are
analyzed as inputs to system consistency orientation determines the
architecture suchintegration complexity. This research (Jain et al.
2008) as using common guidelines to implement fault
Table 1 : System Ititegration Level of Effort
System Integration Complexity Survey
Isstie
Not enough focus given to sy.stem futurecapabilities during
concept development
Interoperability of existing system to newtechnology caused by
continuous change
Vendor compatibility / interoperability
Continuous system changes
System Enhancements cause unforseenissues in other
subsystems
New engineers having unfamiliarity withlegacy systems
Pos.sibleSolution
Need business development leaders, cu.stomer,and lead engineers
to focus on potentialfuture uses for system
System design needs to be forward, present,and backward
compatible
Vendor information sharing
Documentation for synchronization
Defmed system architecture type to makeenhancements easier to
add
Thorough documentation of legacysystems(intent of design,
lessonsleamed)
Phase
Requirements
ArchitectureDesign
ArchitectureDesign
Integration
ArchitectureDesign
Integration
Resulting Level ofEffort during System
I ntegration (1-10)
S
10
1
7.5
3
10
-
Impact of Identified Causal Eactors to "System of Systems"
Integration Complexity from a Defense Industry... 49
Table 2: Survey Categoriesdetection and isolation.
4. RMT embodies reliability, maintainability, andtestability.
Lessons from Steven Institute systemsengineering courses (SYS605:
System Integration) stressRMT helps extend the life-cycle of the
system. RMThelps verification and validation by way of
havingreliable equipment testing, help write test plans
tothoroughly test reliability and maintainability
againstrequirements.
This earlier research (Jain et al. 2008) included a
surveyquestionnaire that addressed the factors identifted as havina
big impact on system integration complexity and qualityof
veriflcation and validation (V&V) to form a cause andeffect
relationship. Verification and validation are animportant part of
the system integration process whichBuede terms "Integration and
Qualification". Qualificationis the process of verifying and
validating the system designand then obtaining the stakeholders'
acceptance of thedesign (Buede 2000).
The survey was modified for this research paper in twoways.
First, a new grouping of the questions was created fromthe original
survey into different categories such ascommonality, modularity,
and requirements. The purpose ofthe grouped questions was to assess
thoughts from engineersabout elements within each category which
they thoughtwere the biggest contributors to system
integrationcomplexity and V&V.
There were 3 percentage choices for the survey in orderto assess
engineers' perspective on each element's impact tosystem
integration complexity. The 3 percentage choiceswere 75%.
Thequestions that had more than >75% as ratings under
eachcategory were then selected to be part of the second
survey.Categories were broken down as shown in Table 2.
Surveyresults showed that factors considered being most
influentialcommonality, modularity, standards (hardware, software)
andsystem requirement as shown in Table 2. Results showed that the
second survey. They ranged from lead engineers toreliability,
maintainability, and test had much some principal engineers in
software, hardware and systeminfluence on integration and
engineering. Results from thetest but not enough to increase
Results show that vendor compatibility and survey are shown inthe
complexity by too much. . . L-I-. J • r. • •/- ^ j- .
^ ^ ^ interoperability and defining a specific type of systemFor
the first part of the architecture for all vendors to use does not
involve
ea system ^ ^ • ^ ^̂ ^̂ ^ /• gff^yf^ ß^^j^ ^/- these elements
gives aengineers ,r . , . . *
arniwprpH Fnr thp srnnH mrt preliminary assessment that system
architecture doesanswered, hor the second part i- / _ J' _ _ number
of components is aof the survey nine engineers result in increasing
or decreasing system integration significant factor in systemtook
the survey. Two third of complexity, whichever case may apply.the
engineers are principal or
Category
Commonality
Commonality in hardwareand software subsystems
Operational commonality
Physical familiarity
Percentage of known vendorsand subcontractors
% of familiar technology
Modularity
Physical modularity
Functional modularity
Orthogonality
Abstraction of systemarchitecture
Standards Based
Interface Openness
Open system orientation
Consistency Orientation
RMT
Required level of reliability
Required level ofmaintainability
Testability
Requirements
Clear identification of thesystem requirements
Prioritization of systemrequirements
System requirementsfeasibility analysis
System requirementsrisk analysis
Categorization ofsystem requirements
Importance to SystemIntegration
75% importance to systemintegration complexity. Nine engineers
were selected to take
integration complexity.System requirements can be a
factor in decreasing the number of components such as LRUsto
control system integration level of effort. Even in
openarchitecture systems, there are many unique elements for
thesimple fact that vendors are subcontracted to develop
somehardware or software components. Although a company mayuse
vendors due to lowered costs or fast delivery turn-around instead
of using in house resources, considerationshould be made as to the
number of allowed unique
© 2009, Global Institute ofFlexible Systems Management
-
50 t 1
Commonality
• Physical Commonality(Within the system)• Hardware
Commonality
• Number of uniqueLRUs• Unique cables• Unique standards• Unique
connectors
• Software Commonality• Number of
Languages• Number of
Compilers• Number of unique
Code implementation• Physical familiarity
(from other systems)• % of vendors
subcontractors known• % hardware and
software technologyfamiliarity
• Operational Commonality• Number of unique skill
required• Estimated training time
for maintenance• Estimated training time
for operation
••• :i 1
Jessica Jove
Modularity
• Physical modularity• Ease of system element
upgrade• System rework• Modifying lines of code
• Ease of operating systemupgrade
• Functional modularize• Ease of adding new
functionality• Orthogonality
• Amount of functionalrequirement fragmentedacross multiple
processingelements and interface
• Amount of throughputrequirements acrossinterfaces
• Abstraction• Does the system
architecture provide andoption for informationhiding
• Interfaces• Number of unique
interfaces per systemelement
• Number of differentnetworking protocols
• Number of explicitimplicit interfaces
/ and Rashmi Jain
Standards Based
• Open Systems Orientation• Interface Standards
•# of interface standards/*of interfaces
• Multiple VendorsGreater than 5) exist forproducts based
onstandards
• Multiple businessdomains apply/usestandards
(aerospace.medical.(telecommunications)
• Hardware Standards•# of Form Factors•# of LRU• Multiple
Vendors• Multiple Business
Domain Apply/Use• Standard Maturity
• Software Standards•# of proprietary & unique
operating systems•# of non standard databases
ware• Consistency Orientation
• Common Guidelines forimplementing diagnosticsand
performance
RMT
• Reliability• Fault Tolerance
• % of missionfunctions with singlepoints of failure
• % of safety criticalfunctions with singlepoints of failure
• Critical Points ofDelicateness (SystemLoading• % Processor
Loading• % Memory Loading• '7o Network Loading
• Maintainability• Expected MTTR• Maximum Fault Group
Size• Whether system
operational undermaintenance
• Accessibility• Testability
•# of LRUs covered by BIT• Reproducibility of Errors• Online
Testing• Automated
Input/Simulation Insertion
Figure 2: Architecture Assessment and Evaluation Parameters and
Priorities (Verma 2000)
components. This goes into play with the
vendorinformation-sharing problem. Willimore vendors sharing
their
number of components used. Since there are more spares that„ . ,
^ , , .^ . . . . . . . can be shared the turnaround timeRMT helps
verification and validation by way ^^^ ^^^^^^ integration
increases
architecture, the system integrators of having reliable
equipment testing, help and requires less troubleshootinghave more
control and feedback u> write test plans to thoroughly test
reliability during integration. This isall vendors to try and limit
tiic ^nd maintainability against requirements.number of unique
elements.
Most survey takers made similar comments aboutcommonality in
hardware and software making integrationeasier by 50-80% because
when engineers are working withfamiliar items, efficiency is
increased. The biggest concernwas compatibility issues that stem
from integrating a highnumber of unique components within the
system. Theprobability of discovering unfamiliar issues increases
with
ISystem Integration Complexity Impact
assuming that the product is notdefective and integration
issues
have already been overcome. Some said that integrationeffort is
reduced in relationship by the number of uniquepieces of equipment.
However, there needs to be a definedreference point before
determining how much better havingcommonality is to system
integration. One additional detailto include in commonality of
hardware is the productionequipment involved in assembly and test
of subsystems preand post system integration. Common-off-the-shelf
(COTS)
System Verification & Validation Impact
Figure 3: System Integration Complexity Survey Results Figure 4
: System Verification and Validation Results
-
Impact of Identified Causal Eactors to "System of Systems"
Integration Complexity from a Defense Industry...
change due to changing environmental regulation such
asRestriction of the Use of Certain Hazardous Substances(RoHS):
therefore testing tools and products vary. However,the methodology,
analysis and processes to implement themodified COTS or assembly
will not. One survey takercommented that the commonality between
these two(equipment and testine
The biggest concern was compatibility issuesthat stem from
integrating a high numberof unique components within the
system.
51
methodologies) is what makessystem integration less complex.
The response to whetherlower percentage of known vendorsand
subcontractors lessen system integration complexity was80% no
meaning it is best to use a higher percentage ofknown vendors. One
of the reasons was integration becomesdifficult when percentage of
known vendors decrease.Unknown issues can arise from electronic
parts or softwarepackages. The less percentage of known vendors the
greater
Table 3: System Integration ComplexitySurvey Scaled
Responses
Sy.stcni IntegrationComplexity
ConmioiKilitN 111 I IW
and S W subsy.slcm.s
Lower 9f> of knownvendors/subcontractors
Increase in % oflamlliar technology
lncrea.se in physicalmodularity
Increase in functionalmodularity
Increase in opensystem orientation
t)ecrease in interfaceopenness
Prioritizatlon ofsystem requirements
.System requirementrisk analysis
Scale
C o m m o n a l i t y in I IWand SW subsystems
Lower % of knownvendors/subcontractors
Increase in % offamiliar technology
Increase in physicalmodularity
Increase in functionalmodularity
Increase in opensystem orientation
t)ecrease in interfaceopenness
Prioritization ofsystem requirements
System requirementrisk analysis
No
0
7
1
1
1
0
3
">
1
1
0
7
1
1
1
0
3
2
1
80%
-)
0
4
3
1
1
0
5
10
0
10
20
\5
2^
5
5
0
Total
9
9
9
9
9
9
9
9
9
Total
4
I
4
4
4
4
3
3
3
the likelihood of unknown trends, characteristics,
reliabilitymethods and products used. Issues raised from
usingunknown vendors again stems from the minimalcommunication and
information given between primecontractor and vendor during product
development. Vendorsgive the technical information after the
product has been
produced as even thesedocumentations can be insufficientor
unintelligible. Choice of theright vendor is critical. Fewervendors
lead to less competition
causing less product innovation and higher prices forproducts.
Both of these problems offset the benefits of fewervendors. In the
current marketplace, benchmarking ofproducts (periodic verification
of current markets for criticalproducts to verify the
vendors/costs) by programs hasbecome the standard and funded by
contracts for the bestprice/product. Product vision (with vendors)
in the future iscritical for system integration to be
effective.
Physical and functional modularity botli in general
wasconsidered by more than 75% of responses to have 50% ormore
impact in lessening system integration complexity.Answers ranged
from modularity making redesign simplerto making substitution of a
failing module easier to do. Inhardware, when proper initial
integration analysis is done,modularity allows this to be leveraged
for future upgrades,future packages or cabinet improvement within
heating,cooling, power, weight, and center of gravity
limitations.With modular design controls comes simplified
I/O,mounting, and installation. In functional modularity,
thecapability to integrate one function at a time allows foreasier
troubleshooting and debugging.
Engineers that claimed a lower percentage of knownvendors help
integration by 20% up to 80% believed thatwith a smaller number of
known vendors, the leaming curvedecreases of what to expect. It
also helps make integrationeasier to manage. The burden on the
prime contractor forinter-subcontractor communication is reduced.
The claimsare with the caveat that the vendors that are being used
havereliable products. However, if the single
vendor/subcontractoris not reliable, then having a fewer number of
vendors andsub-contractors is a risk.
The responses for increased percentage of familiartechnology
used generally (more than 85% agreed) were thatit led to less
system integration complexity. Many of theunknowns for the
technological product decrease when usingfamiliar technology making
it easier to integrate with othersubsystems. Engineers that said it
made sj-stem integrationeasier by more than 80% commented that
efficiency isincreased whenever working with familiar technology.
Whenengineers become familiar with capabilities and limitationsof a
technology, less time is spent on integrating.
One remark about familiar technology not leading tobetter
integration was due to the fact that good engineeringefforts and
analysis is still needed to ensure the applicationof familiar
technology such as COTS are integrated correctlyand within the
performance capabilities of the system.
© 2009, Global Institute ofFlexible Systems Management
-
52 Jessica Jovel and Rashmi Jain
Without diligent system integration team the
interfaces,interconnections, mounting, and technical performance
ofthe system will not work or communicate even with
familiartechnology used.
About 65% of engineers who took the survey agreed thata decrease
in interface openness helped ease systemintegration by at least
20%. It is difficult to comply withdifferent components especially
from different vendors; itrequires additional effort which
increases time and cost.Ideally, the use of common standards.and
interfaces across components and Prioritization of system
requirementssubsystems developed by separate giygg ^ ^/^^^
understanding of the
should reduce the incidence of
lessons learned from solving integration hurdles with a setof
interfaces can be applied when integrating the same typeof
interface somewhere else in the system. The one engineerthat did
not agree that a decrease in interface openness helpsSI mentioned
that interfaces need to be well understoodbefore really assessing
that this improves the systemintegration.
In system requirements related questions about 40%agreed that
system integration complexity decreases by more
than 50%. Within systemrequirements, that prioritization
system integration
problems, when teams are working totechnical performance
requirements and
the same set of rules. However, this program requirements that
are necessaryis not always the case. With a for successful system
integration.decrease in interface openness, the
sitntlar to Dr. Jatn s results m whtchagreed that system
requirement
prioritization made systemintegration easier by 50-80%.
Table 4: System Verification andValidation Survey Scaled
Responses
System Verificationand Validation
Commonality in HWand SW subsystems
Lower % of knownvendors/subcontractors
Increase in % offamiliar technology
Increase in physicalmodularity
Increase in functionalmodularity
Increase in opensystem orientation
Decrease in interfaceopenness
Prioritization ofsystem requirements
Systetn requirementrisk analysis
Scale
Commonality in HWand SW subsystems
Lower % of knownvendors/subcontractors
Increase in % offamiliar technology
Increase in physicalmodularity
Increase in functionalmodularity
Increase in opensystem orientation
Decrease in interfaceopenness
Prioritization ofsystem requirements
System requirementrisk analysis
No
2
4
3
1
1
0
2
1
2
1
2
4
3
1
1
0
2
1
2
2
0
4
0
2
0
0
2
->
4
20-50%
0
1
4
1
1
1
2
3
•}
3
0
3
12
3
3
3
6
9
6
50-80%
7
2
0
2
4
3
3
3
2
4
28
8
0
8
16
12
12
12
8
>80%
0
0
2
4
3
5
1
1
1
5
0
0
III
20
15
25
5
5
5
Total
9
9
9
y
9
9
9
9
9
Total
3
2
3
4
4
4
3
3
3
Prioritization of system requirements gives a clearunderstanding
of the technical performance requirements andprogram requirements
that are necessary for successfulsystem integration. All inputs
from the technical experts,stakeholders and functional managers are
key to ensure norequirements are overlooked or left out (this is
tnostimportant during verification and validation).
Prioritizationof system requirements ensures proper focus is
applied tothe requirements (frotn highest priority to lesser
priority) andwill result is effective verification and validation
when theproper stakeholders are consulted. Engineers who did
notagree that prioritization of requirements helped
systemintegration said that stakeholder requirements do
notnecessarily map well into what needs to take place in orderto
get the system functional.
Risk analysis of system requirement feasibility had highimpact
to system integration as well; 90% of engineersagreed that it
improved integration efforts by at least 20%especially if risk is
based on previous products with similarcapability. Performing
requirement risk analysis helps futuremodification of the
product.
One of the disagreements to the common census is dueto the fact
that requirement risk analysis may have noimpact to system
integration. It was thought that effectiveProgram Risk Management
is more important than just riskanalysis of requirements always
reduced complexity andmitigated problems before system integration
began. It wasthought that identifying key risk areas helps plan
theintegration effort to schedule appropriate amount of effortto
higher risk functionality. This sense may improve thesystem
integration but it will not facilitate the actualprocess.
Identifying program weakness and deficiencies willalways result in
an effective system integration effort. This,however, does not come
with a cost as the resolution ofproblems will require additional
efforts, schedule andproduct performance specifics identified at a
level notcommonly addressed at the program level.
The same questions were given for verification andvalidation
since this is another part of system integration.
-
Impact of Identified Causal Factors to "System of Systems"
Integration Complexity from a Defense Industry... 53
Similarities between verification & validation and
systemintegration complexity survey results were physicalmodularity
and open system orientation had the highestpercent of making more
than 80% impact. The use ofcommon hardware and software sub-system
allow test plans
validate the requirement.
Cause and Effect Relationship
Survey responses for system integration complexity andsystem
verification and validation were weighted to find the
and equipment from previous modules . . _ „ „i. w • causal
factors with most impact. Theto be leveraged for verification and H
is important to note that leveraging ^^^^^.^^^ ^^^ then averaged by
the
ff f b b l d l b i l dvalidation. Commonality helps make off of
Subassembly modular build number of responses and rounded
toverification and validation less successes makes the probability
for nearest integer to help form a causecomplex for next
generations of the success greater for VáíV. and effect
relationship for systemsystem (or subsystems). For example,after
the initial installation, testing of subsystems and thefully
integrated system have been complete for the ftrst timeon a
production build, lessons learned can be leveraged forihe next
generations.
Some argued that if system integration was complete andissues
were resolved, V&V would not rely on commonalityof hardware and
software subsystems. It was also arguedthat the number of unique
parts does not lead to easiersystem validation, but it does lead to
easier verificationbecause there are fewer unique items to verify.
With morecommon hardware/software in a system, common methodsand
techniques can be applied therefore the number oftmique methods
that need to be proven or demonstrated areless.
Although modularity in general was thought to be apositive
effect on V&V (many of the comments as to why itwas helpful
were the same as for system integration), therewas argument on this
idea. The argument was that samefailure rates are shown in modular
assembly as non-modularas.scmblies. It is important to note that
leveraging off of sub-assembly modular build successes makes the
probability forsuccess greater for V&V.
Unlike in system integration a lower percentage ofknown vendors
and subcontractors actually simplifiedverification and validation
in this case. Control of productdevelopment with the correct vendor
(having technicalexpertise and the knowledge base with key
personnel) willtninimize schedule overruns, test/qualification
risks and theintegration of the qualified equipment into the
designedsystem. The correct vendor is crucial however to
theassessment of a decrease in vendor having a positive impactto
V&V. Only when the vendor offers reliable products arefewer
errors are encountered during V&V. Fewer knownvendors may lead
to better conditions if control of processesis in place.
Increasing the percentage of "familiar technology" forV&V
helps set up the process for success. Resources are notwasted on
determining whether any bad results are a resultof the system being
tested or due to unreliable vendortesting equipment.
Prioritization of system requirements in V&V helps aidthe
process by understanding the most importantrcquiretiients that need
to be validated and verifted. Riskanalysis does not benefit
verification and validation muchttnless the part of the risk is in
the capability to verify/
requirements, system architecture, andsystem integration
complexity. The scale for answercategories in the survey are:
No
80%
1
2
3
4
5
The scaled response of system integration show that thefollowing
had most impact (50-80%)
• Commonality in HW and SW subsystems
• Increase in % of familiar technology
• Increase in physical modularity
• Increase in functional modularity
• Increase in open system orientation
The scaled response of system integration show that thefollowing
had most impact (50-80%)
• Increase in physical modularity
• Increase in functional modularity
• Increase in open system orientation
Elements shown in the cause and effect relationshipdiagram.
Figure 5, are all practiced in system architecture.System
requirements in this case did not affect systemintegration
complexity much. If system requirementscontained constraints or a
required level of commonality,standards, or modularity in system
architecture, systemrequirements would indirectly have an affect on
SI. For
Commonality inHW and SWsubsystems
\
tncTMse tn opensystem
orientation
\
tncTÉAse Inhincttonalmodularity
\
Incrtate in % offamiliar
technology
\
/
tncrus« tnphysical
modularity
/
s,\ 8y«l«fnIntegritlon
Comptutty
...
Figure 5: Cause and Effect Relationship forSystem Integration
Complexity
© 2009, Global Institute ofFlexible Systems Management
-
54 Jessica jovel and Rashmi Jain
thatuse the mentioned elements to
technology; also modularitysuch as physical and
functionalmodularity, as well as open systems
example, low levels of hardware andsoftware commonality in
systemsarchitecture have negative impact tointegration complexity.
System their system architecture approach orientation. Addressing
these elementsrequirements could include constraints will help ease
system integration lor the architecture greatly reduceson the
number of unique hardware and ^ „ j verification and validation.
-system of system integration efforts,software components within
the system Awarding contracts to vendors that useas a way to
decrease system complexity. the mentioned elements to their system
architecture
. , . .r. • , ,• 1 • approach will help ease system integration
and verificationSimilarly in system venfication and validation,
system ^^ ,•
•',,•' . , . „. , and validation,architecture has the most
impact as shown in Figure 6.
References
Buede D. The Engineering Design of Systems: Models and Methods.
NewYork: Wiley Series in Systems Engineering, 2000.
Camey D, and P Oberndorf. Integration and Interoperability
Models forSystems of Systems. Systems and Software Technology
Conference. 2004.
Dogru A, S Delcambr. C Bayrak. M Christiansen and M Tanik.
Thedevelopment of an integrated system design environment. ICSl,
1992:691-698.
Elias G and Jain R. Exploring Attributes for Systems
ArchitectureEvaluation. CSER, 2007.
Hanratty M, R H Lightsey and A. G. Larson. Open Systems and
TheSystem Engineering Process. Acquisition Review Quarterly, 1999:
47-60.
Jain R., A. Chandrasekaran. G Elias and R Cloutier. Exploring
the Impactof Systems Architecture and Systems Requirements on
Systems IntegrationComplexity, IEEE Systems Journal, 2(2) June
2008.
Kaplan J. Challenges and Approaches to System of Systems
Engineering.ICAF. 2005.
Kossiakoff A and W N Sweet. System Engineering: Principles
andPractices. New York: John Wiley & Sons, Inc, 2003.
Rossak W and S. Prasad. Integration Architectures - A Framework
forSystems Integration Decisions. Proceedings of the IEEE
InternationalConference on Systems, Man, Cybernetics. 1991.
545-550.
Verma D. Application of Innovative Methods to Early
SystemsEngineering Phases: Need Analysis and Requirements
Defmition. ConceptEvaluation, and System Architecture Development.
NSEBS. 2000.
Increase infoundation almodutarlty
ncreasosyst<
orient«
n openïmtfon
\ s./
Increase inphysicai
modularity
SystemIntegrationComptexity
Figure 6; Cause and Effect Relationship for SystemVerification
and Validation
Summary
The cause and effect relationship between systemarchitecture and
system integration complexity indicatesthat the former is most
significant in impacting systemintegration complexity and system
verification andvalidation. The most important aspects of system
architectureimpacting system integration complexity are:
commonalitysuch as hardware & software commonality, a high
percent
Ms. Jessica Jovel recently completed her Masters of Systems
Engineering at Stevens Institute of Technology. She has 4 years
ofexperience in the defense industry working as a systems engineer.
Her line of work has mostly been concentrated in systemintegration,
verification & validation.
Dr. Rashmi Jain is an Associate Professor of Systems Engineering
at Stevens Institute of Technology. Dr. Jain has over 15 yearsof
experience of working on Information Technology (IT) systems. Prior
to joining Stevens she was with Accenture (formerlyknown as
Andersen Consulting). Over the course of her career she has been
involved in leading the implementation of large andcomplex systems
engineering and integration projects. She has done invited lectures
internationally at Keio University, and ShibauraInstitute of
Technology. Japan, Overseas Chinese Institute of Technology (OCIT),
Taiwan. Indian Institute of Technology - Delhietc. She is a
visiting professor for System Architecture and Integration at Keio
University. Her teaching and research interestsinclude systems
integration, systems architecture and design, business process
reengineering, and rapid systems engineering. Dr.Jain has authored
several papers on these topics. She holds Ph.D. and M.S. degrees in
Technology Management from StevensIn.stitute of Technology.
-
Copyright of Global Journal of Flexible Systems Management is
the property of Global Institute of Flexible
Systems Management and its content may not be copied or emailed
to multiple sites or posted to a listserv
without the copyright holder's express written permission.
However, users may print, download, or email
articles for individual use.
