Defence Standard 21-59, Systems Engineering Guide for Naval Combat Systems

Systems Engineering Guide for Naval Combat Systems

Ministry of Defence Defence Standard 21-59 Issue 2 Publication Date 27 Oct 2006

DEF STAN 21-59 Issue 2

Unclassified ii

Contents

Foreword ...........................................................................................................................................................v 1 Overview ...............................................................................................................................................1 1.1 Introduction.....................................................................................................................................1 1.2 Warning............................................................................................................................................3 1.3 Normative References....................................................................................................................3 1.4 Abbreviations..................................................................................................................................3 1.5 Combat System Design..................................................................................................................4 1.6 Def Stan 21-59 Combat Systems Engineering Process............................................................10 2 Concept Phase ...................................................................................................................................25 2.1 Introduction...................................................................................................................................25 2.2 Purpose of the Concept Phase....................................................................................................25 2.3 Requirements Engineering ..........................................................................................................26 2.4 Concept Phase Inputs ..................................................................................................................27 2.5 Concept Phase Outputs ...............................................................................................................31 2.6 Key Concept Phase Processes ...................................................................................................34 3 Assessment Phase ............................................................................................................................41 3.1 Introduction...................................................................................................................................41 3.2 Purpose of the Assessment Phase.............................................................................................41 3.3 Inputs .............................................................................................................................................43 3.4 Outputs ..........................................................................................................................................43 3.5 Technology Demonstration Programmes / R & D - Technology Survey.................................47 3.6 Key Assessment Phase Processes ............................................................................................47 4 Demonstration and Manufacture Phase ..........................................................................................85 4.1 Introduction...................................................................................................................................85 4.2 Purpose of the Demonstration and Manufacture Phase ..........................................................85 4.3 Inputs .............................................................................................................................................88 4.4 Outputs ..........................................................................................................................................89 4.5 Organisation.............................................................................................................................................90 4.6 Demonstration and Manufacture Phase Processes..................................................................93 5. In-Service Phase ..............................................................................................................................131 5.1 Introduction.................................................................................................................................131 5.2 Purpose of the In-Service Phase...............................................................................................131 5.3 Inputs ...........................................................................................................................................134 5.4 Outputs ........................................................................................................................................134 5.5 Organisation................................................................................................................................135 5.6 In-Service Phase Process ..........................................................................................................135 6 Termination Or Disposal Phase......................................................................................................153


Unclassified iii

6.1 Introduction.................................................................................................................................153 6.2 Purpose of the Termination or Disposal Phase.......................................................................153 6.3 Inputs ...........................................................................................................................................155 6.4 Outputs ........................................................................................................................................155 6.5 Organisation................................................................................................................................156 7 Integration, Test, Evaluation and Acceptance Process...............................................................165 7.1 Introduction.................................................................................................................................165 7.2 Purpose of the Integration, Test, Evaluation and Acceptance Process ...............................165 7.3 Inputs ...........................................................................................................................................167 7.4 Outputs ........................................................................................................................................168 7.5 Organisation................................................................................................................................169 7.6 Integration, Test, Evaluation and Acceptance Process..........................................................170 7.7 Practical Integration, Test, Evaluation and Acceptance ...................................................................192 7.8 Integration, Test and Trials Stage Documentation .................................................................195 7.9 Integration, Test and Trials Stage Specialist Discipline Support ..........................................200 Annex A Normative References..................................................................................................................205 A.1 Reference Information................................................................................................................205 Annex B Abbreviations ................................................................................................................................209

Figures

Figure 1.1 The Combat System.......................................................................................................................4 Figure 1.2 The Iterative Spiral of Design Activities ....................................................................................11 Figure 1.3 Generic Combat System Life Cycle ...........................................................................................15 Figure 1.4 Acceptance Process ....................................................................................................................17 Figure 1.5 ILS Activities.................................................................................................................................18 Figure 1.6 Configuration Management through the Life Cycle .................................................................19 Figure 1.7 Systems Engineering Process and Procurement Approach...................................................20 Figure 2.1 Concept Phase Activities ............................................................................................................34 Figure 3.1 Assessment Phase Activities .....................................................................................................42 Figure 3.2 Assessment Phase Process .......................................................................................................49 Figure 3.3 Composition of Systems Analysis Activity ...............................................................................57 Figure 3.4 Composition of System Design Activity....................................................................................59 Figure 3.5 Design Assessment and Selection ............................................................................................69 Figure 4.1 Shift in Emphasis of Demonstration and Manufacture ............................................................86 Figure 4.2 Demonstration and Manufacture Phase, Overall Process.......................................................87 Figure 4.3 Demonstration and Manufacture Organisational Relationships.............................................91 Figure 4.4 Demonstration and Manufacture Phase Activities ...................................................................94 Figure 4.5 Hierarchy of Project Management Activities.............................................................................96 Figure 4.6 Design Change Management Activities...................................................................................103 Figure 4.7 Subsystem/Equipment Demonstration and Manufacture......................................................109 Figure 4.8 Combat System Acceptance Process......................................................................................113


Unclassified iv

Figure 4.9 Combat System Studies in Context .........................................................................................115 Figure 4.10 ILS Activities Conducted during the Demonstration and Manufacture Phase ..................120 Figure 4.11 Configuration Management Baseline .....................................................................................128 Figure 5.1 Combat System In-Service Phase ............................................................................................131 Figure 5.2 In-Service Support Phase, Overall Process ............................................................................132 Figure 5.3 Staged Handover of Support Responsibilities........................................................................133 Figure 5.4 In-Service Support (ISS) Phase Activities ...............................................................................136 Figure 5.5 Combat System Design Modification.......................................................................................146 Figure 6.1 Removal and Disposal, Overall Process ..................................................................................153 Figure 6.2 Removal and Disposal Organisation Relationships................................................................156 Figure 7.1 Combat System Integration, Test, Evaluation and Acceptance............................................165 Figure 7.2 Integration, Test, Evaluation and Acceptance, overall Process ...........................................167 Figure 7.3 Integration Test and Trials Organisational Relationships .....................................................170 Figure 7.4 Stages of Combat System Integration .....................................................................................171 Figure 7.5 Practical Sequence of Integration, Test and Trials.................................................................172 Figure 7.6 Integration, Test, Evaluation and Acceptance Activities .......................................................173 Figure 7.7 Combat System Acceptance.....................................................................................................186

Tables

Table 1.1 Smart Procurement Principal Themes and Def Stan 21-59 ......................................................21 Table 1.2 Smart Acquisition Commercial Practices and Def Stan 21-59 .................................................23 Table 4.1 Project Management Activities ....................................................................................................97 Table 4.2 Systems Engineering Management Activities ...........................................................................98 Table 4.3 Summary of Life Cycle Process Factors .................................................................................102 Table 4.4 Summary of Documentation ......................................................................................................122 Table 4.5 Specialist Discipline Support.....................................................................................................125 Table 5.1 Project Through Life Management Plan (TLMP) ......................................................................137 Table 5.2 Upkeep Policy..............................................................................................................................140 Table 5.3 Summary of Documentation ......................................................................................................150 Table 7.1 Equipment Test and Trials Activities ........................................................................................178 Table 7.2 Subsystem/Equipment Integration............................................................................................179 Table 7.3 Combat System Integration Activities ......................................................................................180 Table 7.4 Ranging Trials .............................................................................................................................185 Table 7.5 Summary of Documentation ......................................................................................................196 Table 7.6 Specialist Discipline Support.....................................................................................................200


Unclassified v

Foreword AMENDMENT RECORD

Amd No Date Text Affected Signature and Date

REVISION NOTE

This standard is raised to Issue 2 to update its content.

HISTORICAL RECORD

This standard supersedes the following:

DEF STAN 21-59, Volume 1, Issue 1, Dated 14 January 2005

DEF STAN 21-59, Volume 2, Issue 1, Dated 14 January 2005

Sea Systems Publication 59 Volumes 1 and 2, Issue 3, Dated June 2002

Sponsorship

1. This Defence Standard (Def Stan) is sponsored by the Defence Logistics Organisation (DLO) Agency, Ministry of Defence (MOD).

2. The complete Def Stan Issue comprises:

Def Stan 21-59 Issue 2 – System Engineering Guide for Naval Combat Systems.

3. If it is found to be unsuitable for any particular requirement the MOD is to be informed in writing of the circumstances.

4. Any user of this Defence Standard either within MOD or in industry may propose an amendment to it. Proposals for amendments that are not directly applicable to a particular contract are to be made to the publishing authority identified on Page 1, and those directly applicable to a particular contract are to be dealt with using contract procedures.

5. No alteration is to be made to this Defence Standard except by the issue of an authorised amendment.

6. Unless otherwise stated, reference in this Defence Standard to approval, approved, authorised or similar terms, means the Ministry of Defence in writing.

7. Any significant amendments that may be made to this Defence Standard at a later date will be indicated by a vertical sideline. Deletions will be indicated by 000 appearing at the end of the line interval.

8. Extracts from British Standards within this Defence Standard have been included with the permission of the British Standards Institution.


Unclassified vi

Conditions of Release

General

9. This Defence Standard has been devised solely for the use of the MOD, and its contractors in the execution of contracts for the MOD. To the extent permitted by law, the Crown hereby excludes all liability whatsoever and howsoever arising (including but without limitation, liability resulting from negligence) for any loss or damage however caused when the Defence Standard is used for any other purpose.

10. This document is Crown Copyright and the information herein may be subject to Crown or third party rights. It is not to be released, reproduced or published without written permission of the MOD.

11. The Crown reserves the right to amend or modify the contents of this Defence Standard without consulting or informing any holder.

MoD Tender or Contract Process

12. This Defence Standard is the property of the Crown. Unless otherwise authorised in writing by the MOD must be returned on completion of the contract or submission of the tender in connection with which it is issued.

13. When this Defence Standard is used in connection with a MOD tender or contract, the user is to ensure that he is in possession of the appropriate version of each document, including related documents, relevant to each particular tender or contract. Enquiries in this connection may be made of the Authority named in the tender or contract.

14. When Defence Standards are incorporated into contracts, users are responsible for their correct application and for complying with contractual and other statutory requirements. Compliance with a Defence Standard does not of itself confer immunity from legal obligations.

Categories of Naval Defence Standard

15. The Category of this Naval Defence Standard has been determined using the following criteria:

a) Category 1. If not applied may have a Critical affect on the following: Safety of the vessel, its complement or third parties. Operational performance of the vessel, its systems or equipment.

b) Category 2. If not applied may have a Significant affect on the following: Safety of the vessel, its complement or third parties. Operational performance of the vessel, its systems or equipment. Through life costs and support.

c) Category 3. If not applied may have a Minor affect on the following: MOD best practice and fleet commonality. Corporate experience and knowledge. Current support practice.

Related Documents

16. In the tender and procurement processes the related documents in each Section and Annex A can be obtained as follows:

a) British Standards British Standards Institution, 389 Chiswick High Road, London, W4 4AL


Unclassified vii

b) Defence Standards Defence Procurement Agency An Executive Agency of the MoD UK Defence Standardization, Kentigern House 65 Brown Street, Glasgow, G2 8EX

c) Other documents Tender or Contract Sponsor to advise.

17. All applications to Ministry Establishments for related documents are to quote the relevant MOD Invitation to Tender or Contract Number and date, together with the sponsoring Directorate and the Tender or Contract Sponsor.

18. Prime Contractors are responsible for supplying their subcontractors with relevant documentation, including specifications, standards and drawings.

Health and Safety

Warning

19. This Defence Standard may call for the use of processes, substances and procedures that may be injurious to health if adequate precautions are not taken. It refers only too technical suitability and in no way absolves either the supplier or any user from statutory obligations relating to health and safety at any stage of manufacture or use. Where attention is drawn to hazards, those quoted may not necessarily be exhaustive.

20. This Defence Standard has been written and is to be used taking into account the policy stipulated in JSP430: MOD Ship Safety Management System Handbook.

Additional Information

(There is no relevant information)


Unclassified viii

This page intentionally left blank


Unclassified 1

Systems Engineering Guide for Naval Combat Systems - Overview

1 Overview

1.1 Introduction

1.1.1 Purpose

1.1.1.1 Naval Combat Systems are highly complex systems. This publication documents the design engineering strategy that should be used for systems development and procurement. It reflects good management and engineering practice and is predicated on the adoption of Systems Engineering principles.

1.1.2 Objectives

1.1.2.1 The procurement of a Combat System is required to meet a number of objectives, primarily the provision of a defined level of Military Capability (MC) within a specified time frame and at an acceptable cost. A Combat System cannot be engineered in isolation: its interactions with the rest of the warship, other military systems, the procuring and operating organisations, and industry must also be considered. A Military Capability is a combination of elements across the Defence Lines of Development, only one of which is the ‘Equipment’ strand.

1.1.2.2 A Combat System design strategy must provide a management framework for Combat System design that emphasises a whole systems approach and provides advice and guidance to other areas of its procurement. Its primary objectives are to:

a) Identify the design solution that provides the best Value for Money (VFM);

b) Provide both comprehensive and systematic coverage of all aspects of Combat System design across all phases of the Combat System life cycle;

c) Facilitate a range of Systems Engineering (SE) and project management activities necessary to achieve project objectives;

d) State how requirements and constraints are gathered, analysed and transformed into a Combat System design which not only satisfies the need, but which can also be successfully procured, developed, integrated, accepted and supported in-service.

1.1.3 Scope

1.1.3.1 The Combat System design strategy should use a systematic approach involving the:

a) Gathering of requirements within the context of the required Military Capability (MC);

b) Generation, examination and comparison of system design options;

c) Preparation of formal specifications and MoD documentation as a basis for acquisition;

d) Acquisition of Combat System equipments including new, modified, Non Development Items (NDIs) and Commercial Off The Shelf (COTS) technology;

e) Integration of acquired equipment items into a coherent system (which itself is integrated with the platform);


Unclassified 2

f) Provision of interoperability with other capabilities and systems e.g. to achieve Network Enabled Capability (NEC). This involves the Combat System’s integration into the wider MoD Enterprise and infrastructures;

g) Conduct of testing and trials as a basis for acceptance against requirements;

h) Support of the system whilst in-service;

i) Disposal of the system and its components at the end of its operational life.

1.1.4 Applicability

1.1.4.1 This Defence Standard is directly applicable to Naval Combat System projects. This standard amplifies AMS Guidance specifically for acquisition activities in the Maritime domain.

1.1.4.2 The strategy can be applied at the inception of a Combat System project or can be introduced at any subsequent stage in the life cycle. It can be applied to the whole range of Combat System projects, including so-called greenfield design (i.e. systems for which no precedence has been set in terms of design or implementation directives), systems largely comprising NDIs, and to Mid Life Update (MLU)/enhancement studies. During MLUs, the capability of installed Combat Systems can be significantly enhanced, however the design problems of integrating new technologies, especially the emerging COTS technologies and standards, and older bespoke technology should not be underestimated.

1.1.4.3 In applying the strategy, individual projects must apply judgement in terms of which design issues are important, which activities must consequently be carried out and which output products and other project documents/data must be produced.

1.1.5 Document Structure

1.1.5.1 Throughout this document the following convention applies:

a) Bold type is used to identify labels and text explaining an accompanying figure;

b) Italic type is used for text providing specific guidelines, advice or for emphasis;

c) Normal type is used for explanatory text.

1.1.6 Relationship with other Documents

1.1.6.1 The Acquisition Management System (AMS)

1.1.6.2 The prime source of guidance for the implementation of Smart Acquisition processes within the DEC, DPA, DLO and DCSA for EP and STP projects is contained in the AMS at www.ams.dii.r.mil.uk or www.ams mod.uk. The content of this Def Stan uses the basic principles contained within the AMS and whilst these are unlikely to change, the implementation details may. The AMS is the subject of frequent update and should therefore be consulted for clarification of that detail.

1.1.6.3 This document is fully compatible with the AMS guidelines and MoD procedures for “Smart Approvals” as at November 2005.

1.1.7 Intended Readership

1.1.7.1 This document is intended to be used by all Combat System Stakeholders, that is, all parties with a legitimate interest in the successful design, development, assurance, testing, acceptance, operation and support and ultimately the disposal of the Combat System.

1.1.8 Use of this Document

1.1.8.1 Def Stan 21-59 represents a coherent corporate strategy for ‘best practice’ in the delivery of Combat System design. It is both forward-looking in presenting how such studies should be conducted in


Unclassified 3

the future, and realistic since it reflects experience gained on a significant number of projects drawing on both the MoD and industry perspectives.

1.1.8.2 However, whilst this document is intended to identify the range of specific activities which are likely to have to be conducted in any Combat System design project, the information presented neither mandates activities nor specific techniques and tools. Every project must decide how the strategy should be implemented, what activities are key and therefore must be conducted, and what techniques and tools are acceptable for their conduct.

1.2 Warning

The Ministry of Defence (MOD), like its contractors, is subject to both United Kingdom and European laws regarding Health and Safety at Work. All Defence Standards either directly or indirectly invoke the use of processes and procedures that could be injurious to health if adequate precautions are not taken. Defence Standards or their use in no way absolves users from complying with statutory and legal requirements relating to Health and Safety at Work.

1.3 Normative References

The Publications shown in Annex A are referred to in the text of this standard. Publications are grouped and listed in alpha-numeric order.

1.4 Abbreviations

The abbreviations in shown Annex B are referred to in the text of this standard.


Unclassified 4

1.5 Combat System Design

1.5.1 Background

1.5.1.1 A Combat System may be defined as:

The set of men and machine resources which comprise the fighting capabilities of a platform. Essential sub-systems will include weapons, sensors, intelligence and information sources, and the combat management system.

1.5.1.2 Figure 1.1 illustrates the typical scope of surface ship and submarine Combat Systems and their domains of interest. It is not exhaustive and recent shifts in defence policy have increased maritime involvement in Littoral and Land operations and are also increasing the emphasis on interoperability and network-based capabilities. The Combat System integrates into a cohesive whole, all equipments and operators that contribute to the platform’s ability to fight as part of a force package.

Environmental Sensors

Above Water Surveillance External Communications

Navigation

Weapons & Countermeasures

Underwater Surveillance

Command and ControlSystem

Figure 1.1 The Combat System

1.5.2 Historical Approach

1.5.2.1 The traditional method of warship procurement for the Royal Navy has involved the design of a platform (i.e. the combination of hull, propulsion and control machinery) in parallel with the development of a Combat System comprising various sensor, weapon and other equipments. The advent of the Smart Acquisition process and attendant AMS Guidelines has not necessarily removed the potential for incoherence between these parallel activities but it does provide a common procurement framework for all the stakeholders to work together to provide the overall Military Capability required.

1.5.2.2 Much of the emphasis in the early stages of the life cycle has been on the selection of suitable equipments already in existence or under development, including COTS, and on the initiation of development of further new equipments. These new equipments offer improvements in performance over current equipments to counter threat developments and exploit technological advances. They have been developed as distinct projects, which in many cases are separate from the project(s) responsible for developing the platform Combat System(s).


Unclassified 5

1.5.2.3 Integration has seldom been an issue addressed from the outset, more an activity conducted during the latter stages of Combat System design, possibly using a Shore Development/Integration Facility (SDF/SIF) prior to installation of the Combat System on board the First Of Class (FOC) platform. This latter consideration of integration has often resulted in the main emphasis being placed on physical factors rather than issues such as information flow and Human Factors (HF). It concentrates on tailoring existing in-service equipments and interfacing new development equipments to achieve a reasonable degree of cohesion in the whole Combat System rather than designing the separate equipments to work together from the beginning and thus ensuring compatibility of interfaces and operation.

1.5.2.4 Generally, throughout the Combat System development process, its overall capability has been somewhat unknown due to performance and other interdependencies. Once integration has been completed, the approach provides the platform with a combat capability, but possibly with certain shortfalls as compared with that originally required.

1.5.3 Whole Systems Approach

1.5.3.1 A ‘whole systems’ approach to Combat System design has evolved in order to deliver a coherent combat capability. This involves consideration of the Combat System as part of a larger system, namely the warship (i.e. the combination of the platform and its propulsion and Combat System).

1.5.3.2 Warship design encompasses hull design, marine engineering and Combat System engineering. It includes overall responsibility for Combat System design and not just the physical installation and integration of the Combat System equipments. Therefore, in order to ensure consistency between the platform and the Combat System, it is essential to address the Combat System impact on elements of the warship (and therefore the platform) design and vice versa.

1.5.3.3 Functionally, the purpose of a warship comprises the naval architects’ three primary design objectives, namely float, move and fight.

1.5.3.4 Following the ‘whole systems’ concept, the Combat System is designed as an integrated whole from the outset of the project to meet the ‘fight’ aspects of the whole warship requirement. More specifically, the ‘whole systems’ approach involves:

a) Identification of the need for a new ship/submarine;

b) Scoping of the Combat System requirements including the operational scenarios and operational Concepts;

c) Definition of the Combat System acceptance criteria;

d) Definition of the required Information Exchange Requirements and interfaces;

e) Identification of Combat System (explicit and implicit) design drivers;

f) Production of clear, consistent and comprehensive specifications which can be used to procure the Combat System and its constituent subsystems and equipments;

g) Systematic and comprehensive treatment of all SE issues (including ILS, Security and HF) in the design of the Combat System;

h) Consideration of whole warship design issues (transversal issues) which are common to all platform systems including the Combat System;

i) Consideration of the through-life cost of ownership and other affordability issues such as ‘spend profile’ (great importance is now attached to this as in-service costs are usually the dominant costs; often driven by decisions made early in the design process).

1.5.4 Current Challenges

1.5.4.1 The design of modern Combat Systems brings with it many significant challenges including:


Unclassified 6

a) The ability to specify a Combat System sufficiently for acquisition purposes without over-constraining the solutions preferred by suppliers;

b) The ability to predict and achieve overall capability in a cost-effective way;

c) The integration of the broad range of technical disciplines involved in Combat System design;

d) The identification and management of risk and risk contingencies;

e) Greater interaction between Combat System and warship (e.g. upper deck layout, hull/machinery) designers if novel hull forms and greater signature control are required;

f) The effective parallel development of the Combat System and new development equipments;

g) The ability to assess more rapidly the impact of changes in operational and other project requirements (including roles, threat and cost);

h) The intrinsic technical difficulty in addressing certain issues such as the specification and analysis of non-functional requirements;

i) The changing climate though which enabling technology is advancing. In the past, technological innovation was typically military-led. Nowadays, the commercial marketplace is driving technological advance. As a consequence, there is the need to consider the implications of adopting COTS technologies, open systems etc. including the consequent management of obsolescence and ‘technology refresh’;

j) Consideration of the highly interactive nature of each element of equipment within the Combat System;

k) Consideration of the capability of each equipment to be able to carry out additional functions to those that it would have traditionally carried out;

l) The need to partition the Combat System into specialist sub-systems for competitive procurement, followed by the requirement to assess competing software intensive solutions;

m) Increasing requirements for system inter-operability;

n) Successful Combat System integration with the equipment performing at least as well when integrated into the Combat System as it does when operated individually;

o) The need for architectural flexibility so that new combat capabilities can be rapidly inserted to meet new demands and other emerging threats;

p) Addressing issues which provide a benefit across several warship classes – such as common equipments or equipment modules, shared support arrangements and spares provisioning;

q) Timely and appropriate consideration of logistics and supportability issues including, as part of system development, Integrated Logistic Support (ILS) and Continuous Acquisition & Life-cycle Support (CALS);

r) The use of Incremental Acquisition procedures to enhance combat capability when equipment technology is both mature and required;

s) The use of an alliance-based procurement strategy involving a number of suppliers and a specialist project manager.

1.5.4.2 To address these current challenges requires the exploitation and harmonisation of a range of technical (both specialist and generalist) and project management skills.


Unclassified 7

1.5.5 Options for Combat System Acquisition

1.5.5.1 Conventional Approach (i.e. MoD as Design Authority (DA)

1.5.5.1.1 The conventional approach to the development of a Combat System essentially involves the subdivision of the Combat System design into its component subsystems and/or equipments in order to support separate development and procurement, e.g. an MLU requiring only changes to one specific system. With this approach, DPA acts as systems authority with responsibility for the overall integration of Combat System equipments/subsystems and integration of the Combat System in the warship.

1.5.5.1.2 A variant of this approach involves DPA contracting out responsibility for the successful integration of the Combat System equipments (but not for their procurement) to an external Combat System design authority.

1.5.5.2 Prime Contractorship and Whole Warship Procurement (i.e. DA placed with the Prime Contractor)

1.5.5.2.1 The aim of prime contractorship is to transfer risk (particularly the risk associated with the separate procurement and integration of individual Combat System equipments and, potentially, their life cycle support) from the MoD to the contractor.

1.5.5.2.2 Coupled with this transfer of risk is the transferral of design authority. Therefore, following the placement of a prime contract, DPA's ability to influence the design in areas such as the selection of specific equipments and consideration of the fleet-wide commonality of Combat System components is greatly reduced.

1.5.5.2.3 Prime Contractorship imposes a discipline on DPA in the need to:

a) Reduce the risk contingencies, which will be held by both the CSM and the prime contractor, to affordable levels. Tools for reducing these contingencies include applied research, technology demonstrators and rapid prototyping. These topics are discussed further in Sections 2 and 3;

b) Produce very carefully considered prime contractorship Invitation To Tender (ITT) documentation which, whilst taking care to avoid mandating equipment, sets requirements which can only be satisfied by acceptable design solutions;

c) Carefully select potential prime contractors in terms of their ability to handle the transferred risk, taking into account a broad range of factors including the financial viability of the organisation and the range of similar projects of comparable monetary value being undertaken;

d) Adopt a strictly ‘hands-off, eyes-on’ approach following prime contract award, relying upon audits and formal reviews to monitor the system design, including the fulfilment of the requirements and the maturity of the design solution.

1.5.5.2.4 Prime Contractorship may be extended to encompass the entire warship rather than just the Combat System. Its aim is to further transfer risk, including the risk associated with the integration of the hull, machinery and Combat System, from the procurer. This is known as whole warship procurement.

1.5.6 The Relationship between Combat System and Equipment Studies

1.5.7.1 To specify accurately the needs of equipments requiring development and to manage effectively issues such as their integration with the remainder of the Combat System, the relationship between Combat System and equipment studies needs careful consideration.

1.5.7.2 The MoD-funded research programme must reflect anticipated future warship projects, their requirements and design drivers, and current technology shortfalls. CSMs should influence the priorities and direction of the research programme, in the role of Associate customer, starting at concept phase or earlier.

1.5.7.3 Combat System studies should be timed to precede studies for new development equipments so that the full totality of the equipment requirement can be actioned. However, the development timescales for


Unclassified 8

new Combat Systems will require the conducting of system-level and equipment-level studies essentially in parallel.

1.5.7.4 Certain new equipments will be developed on a generic basis so that they can be applied to more than one platform. This places greater emphasis on the need to co-ordinate studies and requirements.

1.5.7 Future Challenges

1.5.7.1 Training

1.5.7.1.1 The Combat System includes the human element comprising the members of the warship complement who fulfil the fight function by utilising the Combat System. These personnel require training at various levels ranging from individual operator skills training through to Command Team Training (CTT). Training initially serves to develop skills, however it must be continued in order to retain and refine skills and to develop teamwork. Training Needs Analysis, which is part of the HF discipline, establishes the total training need for a new Combat System. Consideration of training and other areas of HF forms an intrinsic part of this design strategy.

1.5.7.1.2 Federated Training (or more precisely, Federation of Trainers) is training controlled centrally by a Combat Management System or associated equipment which serves to bring together a team or sub-team of Users and their equipment into a common training process at their place of work. Confederated Training is the linking of onboard training in multiple geographically-dispersed platforms. These solutions are likely to require consideration as alternatives to more conventional approaches.

1.5.7.2 Impact of Synthetic Environments

1.5.7.2.1 To stress the Combat System in a manner that the operator is unlikely to meet on a regular basis, but to which they must be trained to respond, has led to the development of synthetic environments. Synthetic Environments can be created involving a wide variety of training simulators, war game facilities and real equipment in live exercises all integrated through the extensive use of protocols such as Distributed Interactive Simulation (DIS), Aggregate Level Simulation Protocol (ALSP) or High Level Architecture (HLA). It enables an integrated and potentially global network of interacting systems to be used for training, mission rehearsal, system development, strategy and tactics development, and evaluation purposes. Synthetic Environments can comprise real warship combat systems, integration facilities, other land and air platforms (real or virtual), scenario generation and exercise analysis hubs and the networks which interconnect these elements. It does not replace ‘training in the field’ but can serve to complement it to develop tactical procedures and doctrine and the skills at military and, potentially, political, levels.

1.5.7.3 Command and Control Warfare (C2W) and Network Enabled Capability (NEC)

1.5.7.3.1 Command and Control Warfare (C2W) is the term used with the UK operational community to describe warfare in which new levels of speed, synchronisation, precision and flexibility can be achieved through the exploitation of battlefield and situation awareness. It relies on the availability of an increased volume of accurate information regarding both own and opposing forces (covering location, activity, status etc.).

1.5.7.3.2 Realisation of the ability to conduct C2W imposes requirements on all weapon, sensors and Command, Control, Communications, Computers and Intelligence (C4I) systems in terms of equipment characteristics (such as smart weaponry) and personnel training, particularly in the ability of commanders to react more quickly and decisively. Exploitation of such information enables the operational situation to be more accurately assessed, assets to be more effectively deployed and tactics to be more appropriately applied.

1.5.7.3.3 The advent of C2W will lead to new requirements being identified which the existing Combat System must be capable of supporting. The need to ensure that such flexibility and growth can be supported without extensive redesign will have an impact upon the SE process.

1.5.7.3.4 Network Enabled Capability (NEC) is MoD’s approach to improving the flow of information from its source to decision-makers, weapons platforms and operatives in the field who are best placed to deliver the action required. Its realisation will require developments in areas such as ‘shared situational awareness’,


Unclassified 9

‘distributed collaborative planning’, high-bandwidth communications links, resilient information structures with full information availability, mission-configurable assets and the synchronisation of effects.


Unclassified 10

1.6 Def Stan 21-59 Combat Systems Engineering Process

1.6.1 Introduction

1.6.1.1 Before introducing the Combat System design strategy, it is worthwhile summarising the underlying rationale on which it is based. This serves to set the structure of the document in context and aid an appreciation of its major themes.

1.6.1.2 Although the “Smart Acquisition” process approach serves as the procurement framework for exploring SE issues within Def Stan 21-59, the underlying engineering process is largely independent from the Concept/Assessment/Demonstration/Manufacture/In-Service/Disposal (CADMID) life cycle in which it operates.

1.6.1.3 To explain the SE process in context, it is necessary to briefly review a number of aspects which run throughout the Combat System life cycle and provide a useful means of highlighting issues and structuring information in the sections which follow. These aspects are:

a) Systems Engineering Process, introducing the core process, that is the cycle of design activities associated with developing the system definition;

b) Wider Perspective, which explains and explores the core process in the context of other key supporting management and technical functions;

c) Combat System Life Cycle, introducing the framework within which the generic SE process is conducted. This provides the process ‘template’ which is explored in detail in the main sections;

d) Other Major Themes of the Strategy, which provides an introductory overview of the key themes of Acceptance, ILS and Configuration Management (CM) which run throughout the Combat System life cycle;

e) Life Cycle Applications, which relates the generic approach to the specific emphases of the procurement strategy. This is followed by a brief survey of novel system life cycles and a discussion of how the generic process may be applied in support of recent developments, in particular, Smart Procurement.

1.6.2 Systems Engineering Process

1.6.2.1 Core Process

1.6.2.1 The core process which underpins the Combat System life cycle applies a consistent trade-off study framework that is equally applicable to concept evaluation as it is to decision-making during detailed design and beyond.

1.6.2.2 The core process is based on accepted SE and design theory (embodied in ISO 15288, Mil Std 499B, IEEE 1220 and EIA 632 standards) and reflects the iteration between the functional and physical domains, through analysis, synthesis and assessment.

1.6.2.3 This cycle is fundamental to design. It provides the basis for the analytical investigation of requirements and creative generation of possible implementations. Through repeated iterations of the cycle, design solutions are generated, developed, evaluated and subsequently consolidated. The result is an optimum design solution tailored to the requirements and constraints which bound it.

1.6.2.4 This iterative design cycle is depicted in Figure 1.2, which although presenting a simplistic view of what is a complex set of interacting design activities, illustrates the basic concepts. The design cycle is shown relative to the ‘information base’, the sum knowledge which is amassed during the design cycle. This is the suite of information, through which the design is defined, explored, expanded, and then subjected to trade-offs and consolidation.


Unclassified 11

AnalyseProblem

InformationBase

SynthesiseSolution(s)

GatherInformation

Assess(and Select) Synthesise

Solution(s)

GatherInformation


Solution(s)

GatherInformation


Solution(s)

GatherInformation

Assess(and Select)

Consolidation

Proliferation

Figure 1.2 The Iterative Spiral of Design Activities

1.6.2.5 The design cycle recognises that there is a continuum of design activities within which the design at one level determines the bounding requirements (or constraints) of the next. This infers that the definition of requirements cannot be entirely independent of the design solution. The scope of the requirements must also reflect the technology base from which the design solution is composed together with any constraints applicable to the solution’s operation and support.

1.6.2.6 Requirements Engineering

1.6.2.6.1 Requirements Engineering is the systematic process which transforms the needs, objectives and requirements of the customer in the context of system utilisation and identified system characteristics into a complete, consistent and precise form. It involves:

a) Identification of stakeholders;

b) Gathering of requirements and constraints;

c) Refinement of requirements and constraints to address inconsistency and ambiguity.

1.6.2.6.2 Requirements Engineering serves to define and maintain a statement of what is needed by the customer in terms of functionality and performance in the context of the overall system operational objectives. It is conducted iteratively with systems analysis and system design to develop realistic requirements which can be demonstrated as meeting the customer need.

1.6.2.7 Systems Analysis

1.6.2.7.1 Systems Analysis is the systematic consideration of a problem (or system) from complementary perspectives in order to:

a) Obtain a better understanding of it;

b) Identify the component elements, their inter-relationships and impact on key characteristics and other system issues;

c) Systematically identify and assess alternative courses of action.

1.6.2.7.2 Systems Analysis serves to generate models of the system requirement as a basis for investigating system functionality and exploring system partitioning and potential performance within the operational context of the system. It too is conducted iteratively, with requirements engineering and Combat System design, to ensure that customer requirements are satisfied and to support the refinement of feasible alternative solutions.


Unclassified 12

1.6.2.8 System Design

1.6.2.8.1 System Design serves to develop candidate physical architectural solutions, consistent with their position in the hierarchy of system development, which are likely to be capable of meeting both the operational and functional requirements. It involves:

a) Consideration of the non-functional/design/implementation requirements and constraints covering both features in the system and how they are to be developed and implemented;

b) Examination and adoption of one or more system structures/architectures;

c) Consideration of HF issues and mandated/candidate equipments.

1.6.2.9 Assessment and Selection

1.6.2.9.1 Where the creative design process generates many potential alternative solutions and options, some means of evaluating the candidates and determining the optimum is needed. Assessment serves to define the framework within which trade-off studies are to be performed. Typical criteria for comparison include technical performance, Availability, Reliability & Maintainability (AR&M), operational effectiveness, development and support cost and risk. Selection determines the design solution(s) to be taken forward, either for further detailed design or ultimately as a baseline for development / acquisition.

1.6.2.10 Document Production

1.6.2.10.1 Document Production is a fundamental activity, applicable to all stages of the life cycle. The documents produced form the total set of outputs from and inputs to each life cycle stage, providing the essential continuity throughout the overall project. Documents may take the form of textual or computer-generated reports or model-based representations.

1.6.3 The Wider Perspective

1.6.3.1 The generic SE process must be considered against the tasking activities which underpin and focus the SE activities described above. These include the SE (Technical) and project management functions, life cycle process factors, specialist discipline support and Combat Systems studies.

1.6.3.2 Systems Engineering (SE) (Technical) and Project Management

1.6.3.2.1 Overall co-ordination of the above is the responsibility of the SE (Technical) and project management function which provides:

a) SE Management, to ensure that work is identified and programmed within a defined schedule, that SE activities are conducted according to a defined and auditable process and that the engineering and support specialist disciplines are integrated within it and conducted in a coherent manner;

b) Project Management, to ensure that the project is controlled and co-ordinated according to an agreed suite of management and technical plans and policies the application of which can be demonstrated. Also, ensuring adequate arrangements are in place for the management of quality and risk across all contractual boundaries.

1.6.3.2.2 The medium for applying these separate disciplines in a co-ordinated manner demands the involvement of specialists working within an integrated, multi-disciplinary team. Skills are required in specialist technical (i.e. product specialists such as sonar and radar specialists, discipline specialists such as HF and ILS experts, and system specialists, for example in integration), generalist technical (i.e. SE) and project management domains.

1.6.3.3 Life Cycle Process Factors

1.6.3.3.1 Central to a systems-approach is the consideration of system issues at all phases. During early conceptual work, the implications of the evolving design on downstream activities should also be addressed and optimised to the appropriate level of detail. This is achieved through the generic SE process which serves as a problem-solving template at each step.


Unclassified 13

1.6.3.3.2 Thus whenever consideration of the system as a concept is necessary, the requirements/ analysis/design activities should address the consideration of downstream life cycle process factors including:

a) Produceability, the design should take advantage of optimal manufacturing processes and the potential for Concurrent Engineering;

b) Integration and Test, the design should be optimised for ease of integration and relevant issues dealt with in a pre-emptive fashion. This approach is commonly called Design for Integration;

c) Trials and Acceptance, due consideration of the most cost-effective route for acceptance consistent with the prevailing acceptance strategy (Design for Test or Design for Acceptance);

d) Operation and Support, the design needs to be evaluated for its usability and supportability in the field, otherwise known as Design for Support;

e) Removal and Disposal, sufficient effort should be applied to consider the strategy for eventual removal from service and disposal of the system. Design for Disposal will become a more important issue as environmental legislation tightens.

1.6.3.4 Specialist Discipline Support

1.6.3.4.1 A range of specialisms are central to the Combat System engineering effort, in equipment/ warfare technologies such as sensors, command and control, communications, infrastructure and networking, navigation, effectors (weapons, jammers, countermeasures), aviation, and warfare domain areas.

1.6.3.4.2 Furthermore, Combat System studies conducted throughout the life cycle require the consideration of the evolving design from a wide range of engineering discipline perspectives. This requires specialist support in the following areas:

a) Architectural design and performance;

b) AR&M;

c) Communications;

d) Information/Data Management, which reduces the risk and costs of system integration by the management of information transfer and the interfacing between the separately procured parts of the Combat System;

e) Environmental factors including Electro-Magnetic Environmental Effects (E3);

f) HFI and Operability;

g) Installation and Services;

h) ILS, Logistic Support Analysis (LSA) and CALS;

i) Naval Architecture (for naval systems);

j) Operational Analysis and Effectiveness;

k) Performance;

l) Signatures;

m) Software Development;

n) System Alignment;

o) System Safety;


Unclassified 14

p) System Security;

q) Shock and Vibration;

r) Training including training needs assessment and definition of training equipment;

s) Requirements Engineering;

t) Marine engineering, especially where it concerns ships services.

1.6.3.4.3 Naturally, there are trade-offs to be made between competing disciplines and contractual priorities. This is achieved through system studies introduced as a central activity of the generic SE process.

1.6.3.5 System Studies

1.6.3.5.1 System Studies are those activities which are continued throughout the system life cycle providing mathematical, physical or logical models of system emergent properties and predictive indications of system performance across a number of key disciplines. These can be used:

a) For specification through representation of certain facets of the design and verification of information flow across interfaces;

b) For analysis to further understanding of system behaviour and to provide confidence in the projected system operation and performance;

c) In support of the assessment and selection of a proposed design or comparison of options against defined criteria (impact analysis and trade-off studies);

d) To consider dynamic aspects of the system in operation;

e) To prove that technology is sufficiently mature to support the proposed implementation;

f) To support the optimisation of the design;

g) To support acceptance of design aspects which cannot be definitively or practically demonstrated (e.g. operational effectiveness);

h) To support subsequent analysis of test and trials results;

i) To predict the adaptability of the system against emerging requirements.

1.6.4 Combat System Life Cycle

1.6.4.1 General Framework

1.6.4.1.1 A general framework is useful as a reference to understand the relationships between SE processes, the system life cycles through which they are applied, and the contractual environment stages imposed as a result of procurement practice. Together, these aspects determine the role of SE within a project.

1.6.4.1.2 A Generic Combat System Life Cycle is very similar to the AMS CADMID cycle and is presented schematically in Figure 1.3. This shows the typical SE life cycle activities against the overall sequence of phases described below.


Unclassified 15

Definition Realisation Utilisation Disposal

SYSTEMS ENGINEERING (TECHNICAL) AND PROJECT MANAGEMENT

SDF/SIF HATs SATs

REQUIREMENTSENGINEERING

SUBSYSTEM/ EQUIPMENTDEVELOPMENT & PRODUCTION

COMBAT SYSTEMDEVELOPMENT

SYSTEMSANALYSIS

SYSTEMDESIGN

COMBAT SYSTEMTEST & TRIALS

EQUIPMENTTEST & TRIALS

SYSTEMINTEGRATION

IN-SERVICEOPERATION

REMOVAL &DISPOSAL

LIFE CYCLE PROCESS FACTORS

INTEGRATED LOGISTIC SUPPORT (ILS)

IN-SERVICESUPPORT

(THROUGH-LIFE) SYSTEMS STUDIES, SPECIALIST DISCIPLINE SUPPORT, CONFIGURATION MANAGEMENT

Figure 1.3 Generic Combat System Life Cycle

1.6.4.1.3 All technical systems may be considered in terms of a sequence of life cycle phases which may be described as:

a) Definition, where the system as an idea is developed and defined to the point at which it can be implemented;

b) Realisation where the system is implemented as previously defined;

c) Utilisation where the system is put into operational use and where it remains for most of its life time;

d) Disposal where the system has served its useful life, is no longer needed and is removed, dismantled, and disposed of.

1.6.4.1.4 The generic system life cycle embraces the established ‘V’ model of system development and shows this in relation to other notable SE themes which run across the generic phases. Though not explicitly shown on this diagram, system development comprises further levels of the ‘V’ model development cycle for subsystem, equipment, and component development as appropriate.

1.6.4.2 Definition Phase

1.6.4.2.1 The initial phase is primarily concerned with developing and refining the system as a concept, to a point where it can be efficiently and cost-effectively implemented. This refinement of the system concept is typified by the iterative application of the requirements engineering, systems analysis and system design activities central to the SE process model described earlier.

1.6.4.2.2 This phase may also see the design progressed to a level of detail from which development can commence. In this context, system development may entail the development of new or modified items, or the acquisition of Non-Development Items (NDIs) or the use of COTS technology. The system comprises architecture, elements of infrastructure and the applications and equipments delivering actual operational capability.


Unclassified 16

1.6.4.3 Realisation Phase: Implementation of the Physical System

1.6.4.3.1 Realisation of the system as a physical implementation is achieved through the acquisition of system components (referred to as configuration items) and their progressive integration and test. Component equipments are first tested as stand-alone items which are then brought together in groups through system integration. The process culminates in the test and trials of the fully integrated system as a whole.

1.6.4.3.2 The physical integration process is managed through a sequence of test and trials which ensures that the system is operationally acceptable before going to sea. This is achieved through a structured regime of shore-based trials (often conducted at a SDF/SIF), Harbour Acceptance Trials (HATs) and Sea Acceptance Trials (SATs), which collectively provide the basis for demonstrating compliance with the requirements for certification/acceptance.

1.6.4.3.3 The mix of equipments/components will inevitably dictate the overall approach to integration, test and trials, the policy for which would have been formulated prior to the realisation phase.

1.6.4.4 Utilisation Phase: In-Service Operation and Support

1.6.4.4.1 Following the certified hand over to the eventual user, the system will enter its operational lifespan. During this time it will be operated and maintained. Additionally, it may be subject to minor improvements and updates, to maintain its performance, or major upgrades to enhance its capability. Any additional development will need to be achieved through a comparable ‘V’ model development cycle as before, unless it has been planned for ‘ab initio’ as part of an Incremental Acquisition approach.

1.6.4.5 Disposal

1.6.4.5.1 The final phase of the life cycle is removal and disposal in which the system leaves operational service. This may be due to external factors such as changing government policy or platform/ equipment obsolescence or internal factors such as component failure/obsolescence.

1.6.5 Other Major Themes of the Strategy

1.6.5.1 Acceptance

1.6.5.1.1 As the Combat System Acceptance Authority, the DEC is responsible for outlining his Acceptance Strategy at the beginning of the Concept phase and ideally at Gate Zero. Factors that determine it include the Acquisition Strategy (e.g. Incremental, service or equipment oriented), the Acquisition Strategy (e.g. the placing of DA responsibility), when certain requirements need to be met in order to satisfy the Capability Gap and what Military Capability will be required to achieve the In-service Date. This Acceptance Strategy is developed by the IPT, with the DEC’s agreement, and captured within an Integrated Test and Evaluation Plan (ITEAP). System Acceptance occurs when the system acquired by the IPT satisfies the SRD. In-Service Date (ISD) is declared when the Military Capability provided by the system (across all Defence Lines of Development) is assessed as available for operational use.

1.6.5.1.2 The ITEAP should detail the process by which evidence of compliance with the requirements will be accumulated throughout all project stages and ultimately be presented to the DEC for his Acceptance and transfer to Customer 2 and the DLO. A typical acceptance process that could be adopted as a basis for a Combat System ITEAP illustrated in Figure 1.4.


Unclassified 17

Implementation

RE

SA

SD Integration

Test

Acceptance

Lower-levelSubsystem/Equipment

Requirements,partitioned by theSystem Design

Subsystem/EquipmentContracts

Subsystem/Equipment

Deliverables

Evidence ofCompliance with

Lower-levelRequirements for

Formal Acceptance

Combat System

RE

SA

SD Integration

Test

Acceptance

Verification

Verification of Equipment Design

Validation of Equipment/Compliance with Requirements

Subsystem/Equipment

Verification

Verification of System Design

Management of the Acceptance ProcessVisibility of progress towards Acceptance/

Progressive Acceptance

FormalCumulativeAcceptance

RequirementsAllocation andDecomposition

Validation of System/Compliance with Requirements

Figure 1.4 Acceptance Process

1.6.5.1.3 The acceptance process can be thought of as ‘closing the square’, satisfying Combat System design requirements cascaded down the subsystem/equipment hierarchy by the accumulation of acceptance evidence produced from lower level acceptance events. At each level, verification of the transformations between requirements, analyses, design, implementation and test activities ensures traceability of requirements to eventual products. This verification mechanism supports certification and acceptance.

1.6.5.1.4 Visibility of the progress towards acceptance is an important consideration. Reviews are a major element in the acceptance process, especially in determining whether acceptance evidence generated satisfies requirements. The acceptance process has therefore to deal with both success and failure of acceptance events and link in to general management processes to initiate and be involved in recovery programmes if defects and deficiencies arise.

1.6.5.2 Integrated Logistic Support (ILS)

1.6.5.2.1 The Through-Life Cost (TLC) of supporting an equipment is generally equal to or more than the initial cost of procurement. Through-Life cost is therefore a significant factor in acquisition decisions and hence needs to be managed in a disciplined way. ILS is the accepted discipline for managing support costs, for causing support considerations to influence the design or selection of equipment and for delivering and monitoring a consistent support environment for the fielded equipment. Figure 1.5 identifies the major themes of ILS which are shown relative to the project phases throughout which ILS activities are conducted.


Unclassified 18

IN-SERVICE UPKEEP

IN-SERVICE ASSESSMENT

LOGISTICS SUPPORT ANALYSIS (LSA) including the LSAR

LIFE CYCLE COSTING (LCCs)Final LCCRevise LCCLCCInitial LCC

TECHNICAL DOCUMENTATION

SUPPLY SUPPORT

MODIFICATIONS

INTEGRATED SUPPLY SUPPORT PROCEDURES (ISSP)

ScreenCodify Scale PlanProcurement

Order Invoice

DevelopSupport Strategy

MaintainSupport

TerminateSupport

Determine/DeliverSupport

FMECA - Failure Modes Effects and Criticality Analysis; RCM - Reliability Centred Maintenance; MTA - Maintenance Task Analysis;LORA - Level Of Repair Analysis; LSAR - Logistics Support Analysis Record; LCC - Life Cycle Costing

Package HandlingStorage & Transportation

Support & Test

Facilities

LORAMTA

PreventiveTasks

CorrectiveTasks

DesignUse

StudyReq FMECA

RCMSkills Training

Final LSAR

DocumentationProductionVerifyData Capture

& PreparationDoc Design

PlanningGenerate

Data Modules Verify Delivery Use Update Archive

Figure 1.5 ILS Activities

1.6.5.2.2 It is MoD policy that the discipline of ILS is applied to all system/equipment procurement and addresses a wide range of support areas. The ILS elements, as defined by the ILS Def Stan 00-60, are:

a) Maintenance Planning;

b) Supply Support;

c) Support and Test Equipment (S&TE);

d) Managing AR&M data;

e) Facilities;

f) Manpower and HF;

g) Training and Training Equipment;

h) Technical Documentation;

i) LSA.

1.6.5.2.3 The list however is neither exhaustive nor prescriptive and the applicability of each element will vary between projects.

1.6.5.2.4 The LSA process generates data which shall generally be held in a single relational database called a Logistic Support Analysis Record (LSAR).

1.6.5.2.5 Within this guide, ILS is considered as a project management function whereas LSA is considered as a composite of SE activities also addressed in their own right, for example HF, training etc. Though there may be some overlap between subject material, the underlying objective is to co ordinate a concerted set of activities which address all the main priorities as part of an overall SE strategy.

1.6.5.3 Configuration Management (CM)

1.6.5.3.1 CM is the process of identifying and managing changes to the system design as it evolves. The objective is to ensure that proposed changes are necessary and appropriate, that the integrity of the system is maintained and that correct records are kept.


Unclassified 19

1.6.5.3.2 CM during early stages of the life cycle serves to keep track of requirements, specification and plans, so that the design task is co-ordinated through an auditable process. It therefore provides the set of baselines against which change can be assessed, controlled and implemented. From the development and production stage onwards, CM becomes a significant issue due to the proliferation of requirements, specifications, physical deliverables, test and acceptance data.

SDF/SIF HATs SATs


SUBSYSTEM/ EQUIPMENTDEVELOPMENT & PRODUCTION


SYSTEMSANALYSIS

SYSTEMDESIGN



SYSTEMINTEGRATION

IN-SERVICEOPERATION

IN-SERVICESUPPORT

SUPPORT

VERIFICATION

PHYSICAL

REQUIREMENTS

FUNCTIONAL

ACCEPTANCE

Configuration Baselines

Figure 1.6 Configuration Management through the Life Cycle

1.6.5.3.3 CM provides the mechanisms through which the design can be taken through development and production in a controlled manner. This is achieved through imposition of CM baselines across each of the SE activities as depicted in Figure 1.6. The baselines provide a progressive scheme which records the status of the design, development, production, integration and test processes with rigour so that the design is supported by an auditable regime.

1.6.5.3.4 Each of the baselines should be reviewed at the point in the project review cycle at which they are first instantiated and thereafter audited at each formal review. The project review cycle provides the structure through which project baselines and change control processes are formally audited. The baselines provide a solid foundation for the impact assessment of engineering change requests through which change control can be effected. Each baseline represents a particular aspect of the design evolution and should be synchronised so that the design itself moves forward through the formal process of reviews.

1.6.5.3.5 CM is equally important during the in-service phase when technology refresh, technology insertion and major upgrade may be required. It serves to assist in ensuring that unchanged parts of the system perform as they did previously and that any changed parts of the system meet their respective requirements.

1.6.6 Life Cycle Applications

1.6.6.1 Flexibility of a Generic Approach

1.6.6.1.1 The acquisition approach adopted for a project inevitably influences the practical implementation of the SE process. In general terms, the SE approach is largely independent of the acquisition approach. This point is illustrated in Figure 1.7 which implies that the acquisition approach affects the organisational structure more than it affects the underlying process.


Unclassified 20

Concept ProjectDefinition

Development & Production

In Service Support

IntegrationTest & Trials

Feasibility DisposalORGANISATIONAL ARRANGEMENTS, Downey

Concept Stage Consolidation

Feasibility Stage Consolidation

PD StageConsolidation

Concept Stage Consolidation

MOD intra- mural studies/ rainbow team

MOD Project and three consortia conducting parallel studies

MOD Project andtwo consortiaconducting parallelstudies

Contract Award toa single consortia/organisation (orMOD retainedCSDA)

Hand Over to NSCISS Contract to single consortia/ organisation

EAC Approval EAC Approval EAC Approval

Concept Demonstration &Manufacture

In Service Support

(IntegrationTest & Trials)

Assessment Disposal

ORGANISATIONAL ARRANGEMENTS, Smart Acquisition

IPT needs to manage competition forCombat System Development and

Integration. IPT must devolveresponsibility for implementation to the

successful contractor

Integrated Project Team (IPT) formed from DEC, DPA, and Industry (where need for competition permits)empowered to conduct trade-offs within scope of authority

Consolidation conducted within the IPT through theSystems Engineering process

Operating Authority handover to DESO andagents in industry

Operating Authority handover to DESO andagents in industry

IPT moves under control of DLO supported by Industry

‘Initial Gate’ approval ‘Main Gate’ approval

DisposalOperation RealisationConcept APPLICATION OF THE GENERIC SYSTEMS ENGINEERING PROCESS

During the realisation and operational stages, the SE process provides the basis for change impact investigation and definition and implementation of design enhancements

and modifications

During the concept design stages, the SE process should beiteratively repeated to generate and investigate candidate

design solutions. These are then subject to trade-off study toidentify the optimum solution(s) for realisation

Figure 1.7 Systems Engineering Process and Procurement Approach

1.6.6.2 Acquisition Approach

1.6.6.2.1 Def Stan 21-59 provides guidance information on SE issues with reference to the Combat System life cycle. The equivalent Smart acquisition phases are also shown at the bottom of this diagram. This provides a useful framework for system acquisition using terms in common usage throughout the MoD and the defence industry.

1.6.6.2.2 The earlier stages (Concept and Assessment) are conducted to perform increasingly detailed design studies following the generic SE process introduced earlier. These consider a multiplicity of engineering issues and their implications throughout the subsequent stages of the life cycle. Each of the early stages is followed by consolidation activities which serve to rationalise the output products produced by competing consortia (where parallel stage activities are invoked).

1.6.6.2.3 Development and Manufacture, including integration, test, and trials are the combined stages during which the operational Combat System is implemented and accepted. Component equipment is many and complex. Control and co-ordination of separate system and equipment programmes and other Lines of Development demands the imposition of a formal project review cycle to synchronise inter related activities across the system hierarchy.

1.6.6.2.4 Once operational, the Combat System will be subject to continuing In-Service support throughout its operational lifetime. Finally, when it is no longer viable to support the system to provide the required operational capability it will be subject to removal from service, perhaps along with the platform, for eventual disposal.


Unclassified 21

1.6.7 Advanced Life Cycle Applications

1.6.7.1 Smart Acquisition and Def Stan 21-59

1.6.7.1.1 Since the underlying SE process which underpins the Combat System design strategy guide is generic, Def Stan 21-59 already embraces many of the principles of Smart Acquisition listed below and can be applied as described in Table 1.1.

Table 1.1 Smart Procurement Principal Themes and Def Stan 21-59

Topic Description Def Stan 21-59 Approach

Whole Life Systems Approach

A holistic approach whereby any new or modified system or capability is considered and justified in relation to the overall military capability.

Def Stan 21-59 advocates a ‘whole systems’ approach whereby the Combat System is considered in relation to the platform role(s) through Operational Analysis and associated Effectiveness modelling.

Furthermore, the SE Generic Process serves to consider equipment level issues and their impact on the overall system. Through-Life issues are considered from the earliest stage, when design freedom is greatest and issues with significant cost impact can be best addressed.

Project Initiation Formalisation of ‘concept’ to:

• Set out the project boundaries;

• Identify key stakeholders;

• Scope the requirement;

• Produce initial through-life cost estimates;

• Develop strategies for procurement, support, technology insertion and approval.

Def Stan 21-59 endorses the pre-feasibility objectives and activities as an integral part of the Concept stage.

Further emphasis is given on the need to tailor the SE process to meet the needs of the procurement strategy.

Co-operative Stakeholder Involvement

The formation of integrated teams (including industry where applicable) at the earliest stages of a project to agree and achieve common goals.

Def Stan 21-59 demands the identifying and consulting with project Stakeholders through the Requirements Engineering process.

The formation and constitution of integrated teams is addressed in Project Management.

Concurrency and Project Approvals Strategies

To combat delays in the approvals process which conflict with the desire to quickly field rapidly advancing technology.

This needs to be considered when tailoring the Guide to the individual project needs. Coherent, concurrent system and equipment development can be managed with Def Stan 21-59.

Improved Requirements Management

To assist project stakeholders in defining essential requirements and achieving a balance between performance, in-service date, costs, risks and through-life

Requirements Engineering is a central theme of the Strategy. The process though which requirements trade-offs are achieved is illustrated and explained by the Generic


Unclassified 22


supportability. SE Process described earlier.

Rapid Pull Through of Technology

To address and embrace emerging technology from both COTS and applied research domains.

Def Stan 21-59 encourages liaison with the MoD- and Industry-funded applied research and TDPs. Guidance on COTS insertion is provided in Annex B, Project Management.

Integrated Estimating and Predicting

Integrated Cost Forecasting and definition of In-Service Target Date (ISTD).

Whole-life cost estimation is advocated by Def Stan 21-59 from Concept phase so that n affordable system results.

Incremental Acquisition

Planned upgrade (particularly for electronics and software), to deliver early capability and exploit advancing technology through a series of manageable steps.

Incremental Acquisition requires the progressive delivery of capability in a manner which balances military need, technology maturity and budgetary considerations. Enabled by Design base lining, CM and other through-life technical and managerial processes advocated by this standard.

1.6.7.1.2 In addition, Smart Acquisition advocates the adoption of improved commercial practices to promote a more flexible approach to defence procurement as described below in Table 1.2.


Unclassified 23

Table 1.2 Smart Acquisition Commercial Practices and Def Stan 21-59


Partnering Approach/ Teamwork in Pricing

Development of a closer relationship between MoD and Industry:

• to improve flexibility and responsiveness particularly to support incremental acquisition;

• to improve the time taken to get to contract and improve estimating.

Def Stan 21-59 advocates closer working between MoD and industry though it does not explicitly suggest partnering since the precise arrangements are project specific

Performance Evaluation

To assess contractor performance against business process maturity for future reference.

Def Stan 21-59 encourages the adoption of business processes which are SE CMM compliant.

Enhanced Competition

Enhanced scope for competition for In-Service Support contracts as a result of improved definition of IPRs.

In-Service Support organisational considerations are discussed in In-Service Support.

Electronic Commerce

The use of Electronic Data Interchange (EDI) to improve tracking of orders and deliveries and streamline payment systems.

Electronic Data Management is central to the co-ordination and control of the Combat System and Platform design and build process. Other aspects of electronic commerce lie outside the scope of Def Stan 21-59.

Low Value Procurement

Improved efficiency of the purchase of low value items.

Not applicable to Def Stan 21-59.

Reproach through Liquidated Damages

Contractual remedies for unsatisfactory Performance though the informed use of Liquidated Damages (LDs).


Unclassified 24

This Page Intentionally Left Blank


Unclassified 25

Systems Engineering Guide for Naval Combat Systems - Concept Phase

2 Concept Phase

2.1 Introduction

2.1.1 The Acquisition Management System (AMS)

2.1.1.1 The prime source of guidance for the implementation of Smart Acquisition processes within the DPA, DLO and DCSA for EP and STP projects is contained in the AMS at www.ams.dii.r.mil.uk or www.ams.mod.uk. The content of this Def Stan uses the basic principles contained within the AMS and whilst these are unlikely to change, the implementation details may. The AMS is the subject of frequent update and should therefore be consulted for clarification of that detail.

2.1.2 Pre Gate Zero Responsibilities

2.1.2.1 Before starting the Concept Phase, i.e. prior to Gate Zero and the establishment of formal DPA procurement activity, Customer 1 (i.e. the DEC) is required to conduct several activities. These are listed in the DEC Desk Officers’ handbook. Primarily the DEC must establish a Capability Need based on a Capability Gap identified from egg a MILCAP shortfall or emerging threat. This Capability should be expressed in a Single Statement of (User) Need (SSON) that is traceable to the JETL or a METL (Joint / Maritime Essential Task List). This in turn should be decomposed into an initial set of User Requirements captured within an initial User Requirement Document (URD). Complementing this URD should be a Customer Supplier Agreement (CSA) which directs the IPTL or Project manager on how the DEC wishes the project to be managed within the principles of Smart Acquisition (egg for defining the Performance / Cost / Time (PCT) envelope). The CSA should state the budget, its spend profile, when the DEC expects project programme milestones to be met and with what confidence. The DEC must also have established initial coherency of his requirements within the overall MoD programme and established consensus across DECs for issues potentially affecting interoperability and integration into the battlespace.

2.1.2.2 As well as agreeing a CSA with the IPTL for the procurement of Equipment Capability, the DEC should also put in place a set of coherent CSA or funded Tasks with other potential Stakeholders to deliver the other Defence Lines Of Development (DLOD) that contribute to the total Military Capability (MC) to be satisfied by the URD. The Joint Customer 2 Handbook identifies the respective responsibilities for DEC, IPTL and Customer 2 (i.e. both Pivotal Management (PM) and Core Leadership (CL)) and their DLOD development activities. It is particularly important that the Operational Concepts and Doctrine element is started early. These should not just provide a justification for the URD but describe ‘HOW’ the User intends to exploit the capability between its expected In-Service Date (ISD) and Out of Service Date (OSD). Ideally, Analytical Concepts should have been produced by the appropriate Warfare Centre or Doctrine Cell before Gate Zero and delivered to the IPTL.

2.1.2.3 Analytical Concepts have limited value to procurement activities and it is essential that they are developed into Applied Concepts (a Concept of Employment (CONEMP) if relating to a single system, a Concept of Operations (CONOPS) for a number of systems) and regularly updated to inform each Project Phase. The output from this work stream must be produced in accordance with JDCC procedures and be coherent concepts being developed in related capability areas.

2.2 Purpose of the Concept Phase

2.2.1 The aim of the Concept phase is to identify which options are suitable to close an identified capability gap and should be considered further. On behalf of the IPTL, the CSM must gather evidence to inform the Investment Appraisal Board (IAB) at Initial Gate (IG) that there is good reason to proceed to the Assessment Phase and that the release of further public money to do so is justified. It is not the purpose of the Concept phase merely to ‘get through’ IG. The requirements of the IAB at IG are stipulated in the Smart


Unclassified 26

Approvals Handbook. The activities and outputs expected of a Project in the Concept phase of the CADMID cycle are detailed in the AMS.

2.2.2 The primary constraint on the project during the Concept phase is that the Customer (i.e. DEC) must not approve expenditure above 2% of the total procurement cost or £10m, whichever is the lower. The CSM must inform his DEC if this is likely to happen.

2.2.3 To progress Combat System design on a sound basis it is essential that a rigorous Systems Engineering (SE) process is applied to Project activities from the outset so that they are managed as coherent whole and form a firm basis for later decisions. It is important that:

a) A ‘whole systems’ approach is adopted;

b) Discipline is used in the conduct of all activities and the processing of project data;

c) Premature focusing on a single design is avoided.

It is also important that the following are clearly identified, understood and documented:

a) The User Requirement and in particular the Key User Requirements (KUR);

b) The operational scenarios;

c) The operational Concepts;

d) The project’s context within the wider MoD programme, including related equipment projects and contributing Defence LoDs;

e) Design constraints;

f) The overall Project programme;

g) The project Stakeholders;

h) The management processes to deliver the programme.

2.2.4 There is a significant impact on later phases if the Concept phase is not properly conducted or if issues are not effectively progressed.

2.3 Requirements Engineering

2.3.1 The Concept Phase has to concentrate on defining and refining the User’s Requirements and identifies Value For Money (VFM) issues that could affect the selection of design solution options in the next phase. In this predominantly Requirements Engineering process there needs to be a clear understanding of the division of responsibilities and the lines of accountability for the various aspects at this early stage. User’s Requirements are owned by the DEC who has ultimate responsibility them. He is assisted by his Requirements Manager (RM) who, although normally resident within the IPT and functionally responsible to the IPTL, is ultimately accountable to his DEC. The CSM however is normally directly accountable to his Project Manager or IPTL within the DPA. A contractor should never draft the URD.

2.3.2 During the Concept phase the RM assisted by the CSM, shall devise the Concept of Analysis (COA) to be applied during the Combined Operational Effectiveness and Investment Appraisal (COIEA) to be applied to the solution options in the Assessment phase. Critical to this is the identification of the Key User Requirements in the Concept Phase. KURs are those Military Capability requirements or constraints identified from within the wider set of user requirements which are assessed as key to the achievement of the mission, or which are for some reason assessed as of particular interest to management. The CSM shall ensure that any design solutions produced are in sufficient detail that the COEIA costings can be based on pertinent and technically viable solutions.

2.3.3 The COA should reflect the trading philosophy employed for associated Balance of Investment decisions and the strategy established by the DEC in his CSA for PCT envelope management.


Unclassified 27

2.3.4 Requirements Studies

2.3.4.1 To assist in the Requirement Engineering process it is usual to conduct one or more Requirements studies to:

a) formalise any earlier conceptual studies on the identification of a clear operational need;

b) implement a survey of any relevant activities in the MoD-funded research programme, any Generic Technology Demonstrator Programmes (Generic TDPs), Operational Analysis and other studies, and any other Industry Initiatives.

2.3.4.2 They also provide confidence that the technologies do, or will, exist to support the conclusions of the concept studies, provide justification for enhancements to research programmes (both intra murally and extra-murally) and underpin succeeding assessment studies.

2.4 Concept Phase Inputs

2.4.1 The AMS identifies initiators of the Concept phase to be:

a) Funding;

b) JDCC outputs;

c) Technical Opportunity;

d) Capability Gap;

e) Threats;

f) Opportunity;

g) Constraints.

Important sources of material to inform the Concept Phase include:

2.4.1.1 Existing Source/Background Material

2.4.1.1.1 Much existing, useful material is available which the CSM should collate. It includes:

a) Previously produced URDs for related and similar projects;

b) Any data already produced for the new/updated Combat System and its associated platform(s);

c) Baseline Combat System and platform data relating to the nearest comparable system (and from which the new system may potentially be evolved);

d) Joint and Naval staff papers (including white papers, intelligence briefings, The Equipment Programme (EP), related high-level studies etc.);

e) US, NATO, EU study results and position papers including, if appropriate, potential collaborative partner publications;

f) Published data available in the public domain (including defence publications, technical journals, industrial data sheets etc.);

g) Miscellaneous industrial information.

2.4.1.2 If either an evolutionary approach to the design of a new Combat System is envisaged or an update to an existing Combat System is being undertaken, then the following information should be collected on the baseline Combat System/platform and constituent equipments:


Unclassified 28

a) Acceptance Criteria;

b) Risk register, system specification(s), subsystem specifications and coherence specification(s);

c) URDs of related capabilities;

d) Technical information (BRs, CBs, specifications) on all relevant Combat System equipments;

e) Naval Weapon Harbour Trial (NWHT) and Naval Weapon Sea Trial (NWST) reports;

f) Alteration and Addition (As & As) details;

g) Link and interface documentation including Combat System-platform interfacing;

h) In Service Availability, Reliability, Maintainability (AR&M)/Integrated Logistic Support (ILS) data;

i) Design policy information;

j) Configuration status and management information (covering both hardware and software);

k) The assessment of the current contractors' capability to develop software as an initial and then ongoing element of a software process improvement programme within that organisation;

l) The assessment of contractors both as suppliers and as potential prime contractor to gain an understanding of the suppliers' capability, and therefore gain an understanding of the subcontract risks;

m) Software development metrics, if appropriate.

2.4.1.3 Agencies

2.4.1.3.1 Many agencies have potential to contribute to the input data. These include:

a) Sponsors of the ARP: DEC, FBG, RAO, and DEC as Equipment Capability sponsors

b) Defence Science & Technology Labs (Dstl), Qinetiq, Industry partnerships (e.g. ® NITEWORKS).

c) DPA (Project clusters, Support Groups; e.g. Integration Authority), DLO and DCSA.

2.4.1.4 Research Programme

2.4.1.4.1 Much work relevant to future Combat Systems and their design is undertaken as part of the MoD-funded research programme. Its use can limit risks, improve capability and contain development costs.

2.4.1.4.2 MOD-funded research is organised under 7 headings known as “Outputs” with the majority of the expenditure being on:

a) Output 3, the provision of specialist advice and analysis, including OA, to support capability management in the ECC area;

b) Output 6, to ensure the availability of appropriate and sufficiently mature technology in the UK defence supplier base to meet UK needs.

2.4.1.4.3 DCDS(EC) sponsors Output 3 research which is contracted through the Research Acquisition Organisation (RAO), Shrivenham. This research is often used to inform the User Requirement.

2.4.1.4.4 Output 6 research is owned by the DPA’s Deputy Chief Executive and is managed by the FBG with support from the RAO. Its priority is closely aligned to future capability requirements as defined by the ECC. It invests in ‘promising but low maturity’ technology (typical TRLs of levels 1-3) with a view to developing them typically to TRLs of level 6 in the supplier base - so that industry and IPTs can manage any remaining risk and exploit the technology. It does not take on technology development or specific project work that


Unclassified 29

may be more appropriately funded through the DPA Equipment Programme or the DLO Short Term Programme (STP).

2.4.1.4.5 MOD-funded research is undertaken either in-house by Dstl (for research of a sensitive nature or involving International Research Collaboration), by Universities or by industry, including QinetiQ. Advice on the scope of research already undertaken which may be of relevance to a project and the contracting of new research in support of a project can be obtained from the RAO and FBG.

2.4.1.4.6 Output 6 does not take on technology development and specific project work that may be more appropriately funded through the EP or DLO STP. Thus project funding is likely to have to be used to fund any technology development beyond TRL 6. Technology demonstration programmes can be undertaken to mature technologies to TRLs 4 to 7. Where it is undertaken exploitation plans should be drawn up which address issues including: the area and extent of applicability; risks and benefits if used; alternatives and considerations if not used; how industrial involvement, exploitation and acceptance of the programme will be achieved; and alignment with the project's technical, managerial and programme objectives.

2.4.1.5 Use of Research and Generic Technology Demonstrator Programme Output

2.4.1.5.1 TDPs are programmes, which have the aims of producing hardware to bridge the gap between research and development and to demonstrate the feasibility of using new technology to meet a specific operational requirement. They can range from Combat System-level to equipments or significant equipment functions. TDPs also reduce the risk contingencies associated with introducing the new technology down to affordable levels.

2.4.1.5.2 Generally, successful TDPs are exploited by industry through the development of new or enhanced equipments. TDPs often serve as the basis for equipments for inclusion as part of the Combat System design. DECs, RMs and CSMs should make themselves aware of the currently funded and anticipated TDPs.

2.4.1.6 International Collaboration

2.4.1.6.1 Outside the UK there are extensive military programmes. DECs, RMs and CSMs must make themselves aware of such programmes, through the government to government mechanisms such as the US/UK information exchange programmes, and evaluate the applicability of such programmes to the development of UK Combat Systems.

2.4.1.6.2 On behalf of UK MOD, Dstl participates in a range of International Research Collaboration (IRC) programmes, including at the US/UK bipartite, NATO level, TTCP, AUSCANZUKUS, and other multinational arrangements. Information on technical exchanges relevant to a particular project can be sought from Dstl Naval Systems.

2.4.1.7 Industrial Technology Development Programmes

2.4.1.7.1 For non-military purposes there are industrial Research and Development (R&D) programmes, both within the UK and abroad that could be sensibly applied to meeting the defined operational needs. Through a programme of meetings with industry, DECs, RMs and CSMs should make themselves aware of the development of such products and should explore their applicability to Combat System development. This will include, but not be limited to, monitoring the progress of open systems and Commercial Off The Shelf (COTS) technology.

2.4.1.7.2 The S+T Director in MoD is the owner of Output 5 of the research programme whose remit is to undertake technology-watch activities and provide advice across MoD on global S+T advances and to interpret and communicate their relevance to MOD. This work is undertaken by Dstl.

2.4.1.7.3 In order that the MoD can be made aware of emerging industrial developments in technology, the MoD personnel may be required to enter into commercial confidentiality/non disclosure agreements with industry. Before entering into such arrangements, the CSM must seek advice and approval from his IPT Commercial Officer.


Unclassified 30

2.4.1.8 Scenarios for Operational Analysis

2.4.1.8.1 Scenarios for analysing and determining the operational requirement and assessing candidate solutions should be derived from those devised by the Studies Assumptions Group (SAG). These represent the range of operations in which the UK might expect to be involved and cover peacetime, non-warfighting operations and warfighting operations, rapid reaction scenarios, short duration and enduring operations. The most relevant scenario(s) should be selected for exercising the capability under consideration. Detailing of the scenarios is required for specific OA studies in terms of geographical area, timeframe, ORBATs, operational demand such as concurrency of capability, etc.

2.4.1.9 Combat System Engineering Database

2.4.1.9.1 The Combat System Engineering Database (CSED) is a MoD database generated circa 2000 with information on legacy submarine and surface ship weapon and sensor equipments. It covers:

a) Performance parameters (outfit level);

b) Engineering data (physical and ships services);

c) Trials;

d) Cost (outfit level);

e) Documentation;

f) Hazards, including safety and RADHAZ;

g) AR&M.

2.4.1.9.2 It constitutes a source of information, which may be useful for Combat System studies. Further information may be obtained from DLO/TES WLS.

2.4.1.10 Contributory Factors

2.4.1.10.1 All appropriate factors that contribute to the emergence of the need for a new or updated Combat System should be documented. These can include:

a) Need to replace obsolete equipment and provide a necessary capability;

b) New defence policy;

c) New/enhanced actual or potential threat;

d) Weakness in present equipment;

e) New equipments available which offer significant benefits (e.g. support cost) over existing equipments;

f) Advances in technology;

g) Need to fulfil defence roles at lower overall cost;

h) Industrial proposals;

i) Foreign collaboration.

2.4.1.11 Shared Data Environment

2.4.1.11.1 The IPT should set up a Shared Data Environment (SDE) during the Concept phase to serve as the central repository of, or signpost to. all relevant Combat System design information. The SDE should provide ready and easy access to information such as:


Unclassified 31

a) The AMS, BRs, CBs, Def Stans, SSPs, STGPs etc.;

b) JSPs, CESG documents, Central Staff documents, DCIs, SSIs etc.;

c) Industry publications from individual companies;

d) Technical publications etc.;

e) Relevant yearbooks and defence periodicals;

f) Symposia and conference proceedings etc.;

g) International and national standards (BS, EU, ISO etc.);

h) Relevant Government documentation, Acts etc.

2.4.1.11.2 It is essential that all this information is under configuration management and access controlled to allow the appropriate visibility. It must not only allow visibility of the latest information available but also to historical versions which have been used for requirement specification and interoperability with legacy systems. Where a conflict arises in precedence of specific documentation, the CSM should resolve it by consultation which must be auditable.

2.5 Concept Phase Outputs

2.5.1 The Concept phase should produce the following outputs:

a) A compelling IG Business Case for IAB submission justifying a value for money recommendation as to whether or not to proceed to the next Project phase;

b) Whole Life Cost Plan / Cost Of Ownership (COO);

c) A stable URD with clearly identified KURs;

d) Capability Model within MODAF;

e) An outline SRD with initial Key System Requirements;

f) A Through-Life management Plan (TLMP);

g) Assessment phase PCT analysis/Plan;

h) A Risk Management Plan and Risk Register;

i) Draft Systems Engineering Management Plan (SEMP) defining how technical activities are to be integrated;

j) An Interoperability Strategy;

k) A Standardisation Implementation Plan;

l) A COA;

m) A Master Data and Assumptions List (MDAL) and any other Design Assumptions Register;

n) Initial Through Life Costings (N.B. (b) above);

o) System Options;

p) Collaboration opportunities with Industry;

q) Outline definition of the Combat System acquisition/approvals strategy;


Unclassified 32

r) Draft coherence specification identifying common Combat System design issues; reference to all transversal issues e.g. HF, security, safety etc.;

s) Mandated equipment list preferably within the SRD. The mandating of any equipment needs to be justified. There are frequently circumstances (e.g. security systems, or for support reasons i.e. commonality with other ship classes) where there is a clear case for defining equipments that should be incorporated in the new Combat System. Equipments should only be mandated if it is known for certain that they will form part of the Combat System;

t) An initial set of policy papers (either drafted or scoped) to formulate and define how Combat System design issues should be progressed;

u) High-level computer models exploring the feasibility and detail of practical Combat System solutions;

v) An initial assessment of the Combat System design options in terms of technical performance, AR&M and operational effectiveness;

w) Combat System design options record documenting design solutions, problems encountered and resolved, any assumptions made, the results of design reviews etc.;

x) Anomaly reports specifying problems encountered in the analysis of requirements and Combat System models, their resolution and action;

y) A clear definition of the work required during the subsequent assessment phase together with supporting ITT documents (assuming that the work will be placed by a competitive tender) and associated assessment criteria;

z) Draft phase review documentation containing ‘MoD eyes only’ information on potential contractors (especially prime contractors);

aa) A first approximation of the implications in terms of cost and timescale of development, procurement and through life support of the Combat System;

bb) First pass estimates of manning levels for the Combat System design options.

2.5.2 During Concept phase, the project file should be initiated. This serves as a configuration-controlled repository of all significant project-related information.

2.5.3 Additionally projects are required to satisfy the joint DPA/DLO’s Technical Assurance criteria for Project Review and Assurance (PR&A), the IAB’s general requirements (see SMART Approvals Handbook) and any specific IAB requests.

2.5.4 Organisation

2.5.4.1 Approach

2.5.4.1.1 Concept Phase studies may be undertaken:

a) Internally within DPA complemented by uniformed User representatives and supported by intra-mural studies and industrial specialist contractors;

b) By a ‘rainbow’ team of contractors led by DPA and complemented by other specialists (e.g. uniformed user representatives). Rainbow teams are composite teams of Combat System design staff drawn from the range of contractors with relevant experience and capability and who may be formalised into an Integrated Project Team;

c) By an industrial contractor following competitive tendering and overseen by a small MoD team supported by uniformed user representatives and, if necessary, contractor project support.


Unclassified 33

2.5.4.1.2 In accordance with the aims and principles of Smart Acquisition, normally only one team would conduct concept studies.

2.5.4.2 The Integrated Project Team (IPT)

Under Smart Procurement principles the IPT Leader (IPTL) is empowered to manage and conduct procurement activities to deliver products that meet the URD and requirements of his Customer 1 as stipulated in their CSA. The IPTL is not allowed however to act unilaterally in areas that involve other Stakeholder involvement, in particular where trade-offs are being considered and interoperability is affected. The IPTL’s approach must involve the participation of all stakeholders including user, customers 1 and 2, and industry representation (where competition permits) from the earliest opportunity. Whilst Def Stan 21-59 acknowledges the need for co-operative stakeholder involvement, it is the responsibility of the CSM to ensure that organisational requirements and constraints are wholly appropriate to the needs of the project within the prevailing management regime.

2.5.4.3 Team Capability

2.5.4.3.1 Within an IPT, there should, as a minimum, be members capable of supporting the following functional roles for Combat System management:

a) Requirements Manager. An expert User responsible for representing the DEC, Customer 2 and the interests of the eventual Combat System users. As well as taking the lead for requirement issues he should also be responsible for advising the DEC and IPTL on Service Acceptance. This expert would typically be an experienced Naval Officer with authority to engage other IPT RMs on both Maritime and Joint User issues;

b) Combat System Manager. The CSM has responsibility for the whole project. Management of certain elements may be delegated, either internally or externally via a contract, but the CSM still retains responsibility for the project as a whole;

c) The Technical Co-ordinator. Responsible for controlling and directing the SE process and monitoring and reporting of the technical quality of the work typically across the engineering disciplines;

d) ILS Manager. Responsible for co-ordination and monitoring of all life cycle support activities and specifically the AR&M and training programme;

e) Programme Manager. Responsible for advice and assistance in planning, maintenance of administrative controls against schedules and budget, and monitoring quality of deliverable plans, schedules and contractual matters to the extent to which he is so qualified;

f) Quality Assurance (QA). Responsible for ensuring that the correct quality control processes are in place both internally to the MoD and external in any subcontract organisation. This QA role will also encompass any on-site Quality Assurance Representative (QAR), role required by the contract.

2.5.4.3.2 These roles must be contained within the team, even when the study is largely undertaken by a contractor so that concept studies can be correctly scoped, design issues and anomalies resolved, results interpreted, technical progress monitored and an informed customer base developed and maintained. However, several roles are normally fulfilled by a small group of individuals who each wear ‘more than one hat’.

2.5.4.4 International Collaboration/Co-operation

2.5.4.4.1 The consideration of all new requirements and projects should include an evaluation of the possibilities for establishing joint requirements and common development and/or production projects with other countries. The DEC has responsibility for initiating and setting up international agreements for information and technical exchange.


Unclassified 34

2.6 Key Concept Phase Processes

2.6.1 The Concept Phase serves to develop the URD, which the required Combat System seeks to address. This need is reflected through requirements, which are subjected to analysis to identify all significant design issues. Design studies are conducted to draw up and assess a broad range of potential candidate design solutions, which can be subjected to COEIA.

LIFE CYCLEPROCESS FACTORS

Produceability

Integration and Test Strategy

Acceptance and Trials Policy

Support Plans

Disposal Plans

SYSTEMS ENGINEERING (TECHNICAL)and PROJECT MANAGEMENT

Management Policies and PlansSystems Engineering Management Plan

NEEDSDEFINITION


SYSTEMSANALYSIS

CONCEPTUALDESIGN

DESIGN ASSESSMENTand SELECTION

Technical Performance

AR&M Assessment

COEIA

Design Selection

Design Validation

Definition of Systems DesignConcept Options

SYSTEM STUDIES

DOCUMENTATION

SPECIALIST DISCIPLINE SUPPORT

URD Production

Configuration ManagementHuman Factors ILS Safety Security

Prototyping and Modelling

Figure 2.1 Concept Phase Activities

2.6.2 Important activities in the Concept phase are to:

a) Clarify, identify and refine the need for a new or replacement capability;

b) Gather requirement statements and all relevant data (including applicable policies, objectives and constraints) for the proposed Combat System from all parties with a legitimate interest in it; all requirements acquired should be recorded in a DOORS database;

c) Document and analyse the operational scenarios within which the Combat System will have to provide a capability and how the User intends to employ it within those scenarios as stated in his Applied Operational Concepts;

d) Better define the Performance Cost Time (PCT) envelope by gaining and promoting a thorough understanding of the problem-space, potential solutions and the interactions between their different issues;

e) Generate an initial range of candidate design solutions which could potentially satisfy the identified need, to assess these solutions, their associated Costs and Risks, ascertain the design drivers and shortlist the most promising options for subsequent COEIA. The design solutions must meet the requirements, be technically viable and result in an affordable Combat System;

f) Identify any possible changes or enhancements required in the capability of other systems or subsystems with which interoperability or integration may pose a risk. This may lead to new equipment and funding requirements for these other project;

g) Produce an initial SRD and other documentation to support the development of the URD;

h) Identify Needlines and produce draft Information / Service Exchange Requirements.


Unclassified 35

Conceptual design should identify architectural issues and whether possible equipments (or whole system solutions) are available.

2.6.2.1 Through Life Management Plan

2.6.2.1.1 The Project must produce a TLMP i.a.w. the AMS. Prior to Gate Zero the DEC should have produced an embryonic TLMP that is aligned with his CSA agreed with the IPTL.

2.6.2.2 Stakeholder Responsibility Matrix (SRM)

2.6.2.2.1 The IPTL and CSM shall produce a Stakeholder Responsibility Matrix and form funded business agreements with those stakeholders to ensure delivery of the DLODs. This is particularly important to ensure that Government Furnished Assets (GFA) associated with Combat System interfaces are managed.

2.6.3 Systems Engineering (Technical) and Project Management

2.6.3.1 Use of a SEMP is central to the Combat System engineering strategy. This document defines the intended conduct and implementation of SE (Technical) and project management activities on a Combat System design project. Many of the activities of technical and project management are common to each of the phases covered in this guide. The SEMP should form part of the TLMP.

2.6.3.2 Risk Management Plan

2.6.3.2.1 As part of the joint DPA /DLO Project Review & Assurance (PR&A) process, the IPT must show that they have identified and managed Project Risk sufficiently to provide confidence that further public money can be released.

2.6.3.3 Approvals Strategy

2.6.3.3.1 The adoption of a streamlined approval process consistent with the Smart Approvals Handbook is mandatory.

2.6.3.4 Costing

2.6.3.4.1 All Risks and Assumptions should be associated with an entry in the MDAL.

2.6.3.5 Standards

2.6.3.5.1 The imposition of specific standards on a particular supplier, particularly retrospectively, can cause high costs. However, for successful networking and eventual NEC, adherence to common standards is essential. The CSM should aim to structure the project such that the costs and benefits of standardisation are identifiable. He should comply with the JSP 600 policy statements on CIS Standards and produce an initial Standardisation implementation Plan prior to IG. Advice can be sought from PDG/DStan, the Integration Authority and real-time combat system networking experts.

2.6.3.6 Needs Definition

2.6.3.6.1 Needs Definition is driven by VCDS’s Capability Working Groups and their capability management processes. Director Equipment Capability (Customer 1) is responsible for establishing the Operational need in liaison with service Operational and Engineering branches, and scientific advice including Operational Analysis. MILCAP is a primary source for the identification of the need for new or enhanced capability.

2.6.3.7 Concept of Operations (CONOPS) and Concept of Employment (CONEMP)

2.6.3.7.1 As part of the URD, it is essential that the operational Applied Concepts are developed and delivered to a programme that supports the Equipment Capability procurement. It is the DECs responsibility to facilitate this e.g. by funded tasks or CSAs with Customer 2. The Joint Customer 2 Handbook identifies the respective responsibilities for DEC, IPTL and Customer 2 (i.e. both Pivotal Management (PM) and Core Leadership (CL)) and their activities. The updating of Combat System capability implicitly changes the Ship’s or Submarine’s capability and possibly its role. The production of new Applied Concepts (e.g. CONOPS and


Unclassified 36

CONEMP) is the responsibility of the appropriate Warfare Centre/Doctrine Cell, under JDCC procedures and guidelines. It must not be placed with a Contractor. It involves:

a) Outlining of the warship roles (a role is the prime operational objective of the warship at any one time) and missions (a mission is the series of roles performed from leaving base to returning to base and the durations that those roles are performed);

b) Outlining of typical scenarios (a scenario is a description of the circumstances (features and events) external to the warship within which key attributes may be tested);

c) Considering the threat of likely opposing forces.


2.6.3.8.1 Requirements Engineering is the integrated and systematic process which transforms the needs and wishes of the Customers for an MC into a complete, consistent and precise form for a System design.

2.6.3.8.2 Requirements Engineering combines the activities of Collection, Collation and Control – to promote Sufficiency. The process is Iterative and Interactive – it initiates the design process and adds numeracy to requirements statements in formal documentation. The working framework (organisation/structures) must encourage a pro-active attitude, offer flexibility and support multiple domain perspectives.

2.6.3.8.3 Requirements Capture, Requirements Engineering and Requirements Management are now all mainstream activities under Smart Acquisition for both functional and non-functional requirements using the preferred MoD toolset. Further information is available on the AMS and guidance can be sought from the DPA Procurement Development Group who sponsor a Requirements Managers focus Group (PDG PM2 ABW x34188).

2.6.3.8.4 The RM shall place both the Requirement Database (RDB) and requirements definition documents (e.g. URD, SRD) under project configuration control from the earliest phases of the project to ensure that the evolution of the Combat System requirements can be subject to a formal design audit.

2.6.3.9 Scenario Analysis

2.6.3.9.1 Within the context of requirements engineering, scenario analysis serves to identify the events (in the environment) to which the system must respond and the envisaged operational usage of the system (in terms of tasks, which must be performed). It can therefore be used to assist in the acquisition of requirements (e.g. those concerning system-environment interaction) or with their verification/validation.

2.6.3.10 User Requirements Specification (URS)

2.6.3.10.1 Historically a URS has been used to provide more detail to the requirements for Command Systems/Combat Management Systems. The Command System/CMS provides the crucial Control Room/Ops Room Command Team interface at the heart of any combat system. The production of a separate URS for this very complex and User-dominated system was regarded as a distinct activity. Under Smart Acquisition this distinction is no longer recognised.

2.6.3.10.2 The URS has not been replaced by the URD. A summary of what the URS’s intended purpose was, is included here to illustrate the issues that still need to be addressed in some way, either by MoD or by Industry. In the area of Human Factors, Style guides (Ref STG Publications 10 & 11) and techniques such as Task Analysis are now recognised as constraints on a CMS design.

2.6.3.10.3 Level 00 of a URS (also termed the ‘scoping document’) was produced to cover:

a) High-level explanation of the host platform roles;

b) High-level analysis of the command task capability;

c) Combat System description and boundary definition;


Unclassified 37

d) Context Diagram (a context diagram graphically indicates the boundary of a system and identifies the interaction between the system and elements of its environment. It is a standard notational device of systems analysis and is a feature of several methods including the Yourdon Structured Method), top-level processes (man and machine) and supporting explanation.

2.6.3.10.4 Subsequently Level 0 of a URS was produced for a later project phase to cover:

a) Statement of the command task capability requirement;

b) Definition of the high-level tactical data capacity;

c) Consideration of equipment fit implications;

d) Consideration of HF;

e) Analysis of implication for the whole warship project.

2.6.3.10.5 The URS was a major source of both User and System requirements for the Combat System.

2.6.3.11 Capability Modelling

2.6.3.11.1 Modelling techniques can be employed to address experimentation, completeness and consistency of User requirements. Advice on modelling tool-sets and techniques and their relative suitability should be sought from established MoD sources, as some may be freely available to the project through corporate licence agreements. Lock-in to proprietary modelling tools must be avoided. NITEWORKS is available for project use.

2.6.3.11.2 User Needlines and if possible Information Exchange Requirements (IER) should be modelled e.g. in JIFM or iSMART (or earlier TULIP).

2.6.3.11.3 Whatever modelling is conducted for the User’s MC requirements an output from that modelling must feed into Integration Authority’s MODAF to provide a high level view of the ‘system of systems’ architecture.

2.6.3.12 System Requirement Specification

2.6.3.12.1 The Concept phase should identify the scope of possible, wide ranging solutions to the MC being sought and how they will be investigated and assessed during the Assessment Phase. As part of this exercise an initial view of what is required of the new/updated system can be obtained. This can start the formal specification of system requirements in an SRD. This will need to bound the scope of the (equipment) system being acquired and the contributions required from other Defence LoDs.

2.6.3.13 Concept of Analysis (COA)

2.6.3.13.1 The Concept Phase must produce a COA. This should detail how the various Combat System design solution options will be assessed in the Combined Operational Effectiveness and Investment Appraisal (COEIA) during the Assessment phase and what criteria will be used. Responsibility for the COA and COEIA is shared between the DEC and IPT.

2.6.3.14 Specialist Discipline Support

2.6.3.14.1 A number of specialist activities which have significant downstream implications on the cost and effectiveness of the evolving Combat System need to be considered during the Concept phase as follows:

a) Air Engineering;

b) Alignment;

c) AR&M;


Unclassified 38

d) CM;

e) Cost;

f) Documentation;

g) EMC, RADHAZ, Tempest, Magnetic and Nuclear Survivability;

h) Environmental Factors;

i) HF/Operability;

j) ILS/Supportability;

k) Installation and Ship Services;

l) Information Management, Interface/Data Management;

m) Modelling;

n) Mutual Interference;

o) Naval Architecture;

p) Operational Performance/Effectiveness;

q) Safety;

r) Security;

s) Shock and Vibration;

t) Software Development;

u) Structural Engineering;

v) Training;

w) Integration into the Battlespace;

x) C4I;

y) ISTAR.

2.6.3.15 Configuration Management (CM)

2.6.3.15.1 CM is a central theme of the strategy, important during the early phases to keep track of requirements, specification and plans which define the Combat System baseline design.

2.6.3.15.2 Def Stan 05-57 is the overriding standard for CM in the MOD.

2.6.3.16 Data Management

2.6.3.16.1 To ‘design for integration’, an initial consideration of how information passing between systems is to be managed standard definitions of application data elements, co-ordinate systems, network interfaces, network protocols, and application protocols reduce both implementation risk, integration risk, and test tool costs. An initial review is needed to establish whether currently established standards and documentation processes are likely to apply or whether there would be benefit in moving to a new regime.


Unclassified 39

2.6.3.16.2 The information and interface management issues will be reflected in the phase outputs in the draft coherence specification and they may also require specific studies to be undertaken in the Concept Phase. These issues are unlikely to impact on the costings for this phase, but they will impact on risk.

2.6.3.17 Human Factors (HF)

2.6.3.17.1 The Combat System is an integration of personnel and technology. Consideration of the human contribution to Combat System capability is required, starting at Concept phase.

2.6.3.17.2 The approach defined in STG publications 10 & 11, Human Factors Guide for Management and Design in Royal Navy Combat Systems, is intended to ensure that greater account is taken of the human element in generating the URD by emphasising that warships and their systems (including the Combat System) are to be designed to be compatible with their users. The aim of this approach is to enhance system effectiveness, increase manpower retention, reduce manning, enhance safety and reduce cost.

2.6.3.17.3 The Concept phase should produce an initial Human Factors Integration Plan (HFIP).

2.6.3.18 ILS Issues

2.6.3.18.1 An initial draft of an ILS plan and ILS Task 201 (Use Study) should be produced in accordance with Def Stan 00-60.

2.6.3.19 Information System Security (INFOSEC)

2.6.3.19.1 Security can have a profound impact on the Combat System design, e.g. use of COTS. Combat System security must be addressed in the Concept Phase.

2.6.3.19.2 An infosec appraisal should be drafted during the Concept Phase and early liaison established with both the Security Accreditor, and CESG.

2.6.3.20 Safety Requirements

2.6.3.20.1 The adoption of a safety management procedure is now a mandatory and legal requirement on all IPTLs. It is required to encompass both platform and Combat System aspects.

2.6.3.20.2 For systems which will contain Ordnance, Munitions & Explosives, the following apply:

a) JSP 520 Pt 1 – OME Safety Management System;

b) JSP 520 Pt 2 - Operational Procedures.

2.6.3.20.3 The Defence Ordnance Safety Group (DOSG) web site: www.ams.dii.r.mil.uk/content/docs/dosgweb contains further guidance.

2.6.3.20.4 For issues concerning Platform safety, the DOSG Technical support (DOSGTS) Safety Management Office (SMO) should be consulted. Relevant standards are as follows:

a) Ships – JSP 430;

b) Air – JSP 553;

c) Land – JSP 454.

2.6.3.20.5 The procedure to generate the ‘Safety Case’ is described in the AMS and will be devised at the warship level.

2.6.3.20.6 The mandated procedures are contained in:

a) POSMS – Project-Oriented Safety Management System;


Unclassified 40

b) POEMS – Project-Oriented Environmental Management System.

2.6.3.20.7 The safety case serves to demonstrate that the whole warship meets agreed safety objectives. It should ensure that a clear audit trail exists from design through procurement, operator and upkeep to disposal. In addition, a summary of the safety case, known as the safety case report, is prepared to assist in the reviewing of performance and the provision of authority to proceed to the next phase of the life cycle.

2.6.3.21 Concept Phase Consolidation

2.6.3.21.1 The purpose of the consolidation phase is to ensure that all initial threads are brought together to form a firm basis for the Assessment Phase. It is generally dependent on the implementation of the SE process as determined by the project SEMP and the extent of Concept Phase activities undertaken by Industry.


Unclassified 41

Systems Engineering Guide for Naval Combat Systems - Assessment Phase

3 Assessment Phase

3.1 Introduction



3.1.2 The Assessment Phase serves to develop candidate Concept Phase designs in more detail, subjecting them to trade-off studies considering time, risk, cost and performance, until the detail is sufficient to take a single design solution forward into Demonstration & Manufacture. Following consolidation, the result is a Systems Requirements Document (SRD) and systems and subsystem specifications together with a framework of policies and plans to support the Business Case for Main Gate (MG) approval by the IAB.

3.1.2.1 Assessment is sometimes split into two or more Assessment Phases. This is usually done in order to manage risk and implement the Procurement Strategy which down-selects from say, three bidders / industrial partners / consortia in Assessment Phase 1, down to two in Phase 2 and ending in the selection of one design to take forward to Demonstration & Manufacture (D&M).

3.2 Purpose of the Assessment Phase

3.2.1 The aim of the Assessment Phase is to down-select to a single technological option and reduce risk to an acceptable level for Main Gate (MG). The CSM, on behalf of the IPTL, must gather evidence to inform the Investment Appraisal Board (IAB) at MG that there is good reason to proceed to the D&M Phase and that the release of further public money to do so is justified. It is not the purpose of the Assessment Phase merely to ‘pass through’ MG. The requirements of the IAB at MG are stipulated in the SMART Approvals handbook. The activities and outputs expected of a Project in the Assessment Phase of the CADMID cycle are detailed in the AMS.

3.2.2 The primary constraint placed on the Assessment Phase is that expenditure shall not exceed the limits set at the Initial Gate approval without resubmission for a change to those limits. The CSM must inform his DEC if this is likely to happen.

3.2.3 For the “Acquisition of equipment capability by conventional means”, the AMS Handbook describes the activities to be carried out during Assessment as follows:

• Produce the SRD, defining what the system must do to meet the User needs as stated in the URD;

• Establish & maintain the linkage between the User and System requirements;

• Identify the most cost-effective technological and procurement solution;

• Develop the SRD, trading time, cost and performance to devise a balanced technological and timely solution;

• Reduce risk;


Unclassified 42

• Refine the TLMP including detailed plans for the Demonstration Phase and through-life management;

• Produce the Main Gate Business Case.

3.2.4 For a Combat System, the Assessment Phase aims to build on the achievements of Concept Phase and reduce risk by:

a) Producing credible system solutions to meet the requirements;

b) Validating these solutions through extensive modelling;

c) Documenting the implications in revisions to the requirements definition and enhancement to the system specification;

3.2.5 Establishing a realistic Cost Of Ownership (COO) and Through Life Cost. This includes developing the MOD’s spend profile to take account of all the remaining project phases.

Figure 3.1 Assessment Phase Activities

3.2.6 Figure 3.1 shows Assessment Phase activities based on the generic Systems Engineering (SE) process. This is common between the Concept and Assessment Phases and provides a consistent basis for activities prior to contract award. During the Assessment Phase, the SE process conducted during Concept is repeated. This time the emphasis is on developing and defining candidate designs in sufficient detail, subjecting them to a trade-off studies, in order to select one design solution to take forward into D&M.

3.2.7 Assessment Phase activities

3.2.7.1 The activities during the Assessment Phase follow-on from those initiated in the Concept Phase but with increasing levels of detail until the first indications of physical solutions become available and the baseline Combat System design is established. This is reflected by the finalisation of the User Requirement Document (URD), its Key User Requirements (KURs), the System Requirement Document (SRD), its Key System Requirements (KSRs), an initial specified design (system and subsystem specifications) and mature Through Life Management Plan (TLMP). Throughout the Assessment Phase, the evolving design solution is explored using through-life models, which provide confidence that the design will achieve the required performance to time and budget within project constraints. Most assessment work will be conducted by contractors on behalf of MOD; the major MoD activity is to retain full awareness of the progress, to enable the correct choice for Demonstration & Manufacture in terms of both design solutions and participating contractors.

Assessment InputsURD

Contract Management Plan Risk Assessment

Requirements Database/ Requirements Definition

Document System Specifications

Concept System models Assessment Tasks

Consolidation & Tender Selection

ASSESSMENT Stagee.g. MoD Project and two competing consortia

SelectedContractor for

Demonstration &Manufacture

Assessment OutputsSRD

Contract Management PlanRisk Management Plan

SEMP (Update)Updated Requirements

Database/ RequirementsDefinition Document

Mature System & Subsystem

SpecificationsITT & Related Documents

for Demonstration &Manufacture


REQUIREMENTS ENGINEERING

SYSTEMSANALYSIS

SYSTEMDESIGN

DOCUMENTATIONASSESSMENT & SELECTION

Assess and Select

Documentand Report

Gather Information

SynthesiseSolution(s)

AnalyseProblem

InformationBase

TENDER

SELECT’N

ITT for D&M

CONSOL-

IDATION


Unclassified 43

3.2.7.2 Following consolidation, the result is a definitive specification of the Combat System, its subsystems and equipments and its platform interfaces. This is supported within a hierarchy of plans defining the development and production, integration, trials, operation, and support strategies. In accordance with the Smart Approvals Handbook the package will be submitted for approval by the Investment Approval Board (IAB) and Secretary of State if necessary, and forms the basis for the Development and Manufacture contract.

3.2.7.3 The detailed activities are all iterative and include:

a) Expanding the definition of the Combat System requirements and design activities to produce a set of acceptable physical options that are good candidates for meeting the SRD;

b) Documenting the candidate designs, and any issues associated with mandating any part of the equipment solution, highlighting technical, programme and cost risk and the implications of the mandated levels of safety on the proposed designs;

c) Assessment of the performance of the proposed Combat System designs within real world scenarios - this will require the use of computer modelling techniques, thereby laying the groundwork for Combat System acceptance trials;

d) Ensuring that the correct level of emphasis is given to Integrated Logistics Support (ILS) activities by considering supportability issues as an integral part of the system design process;

e) Obtaining COO and Whole Life Costs for each candidate design including the costs associated with contributing Lines of Development and any impact on other system and capabilities;

f) Conducting cost trade-off analysis on the system design options to ensure that the most cost-effective solutions are given priority and that the Combined Operational Effectiveness and Investment Appraisal (COEIA) for the MG business case is supported;

g) Exchanging information with the platform design team to ensure compatible development;

h) Developing the integration strategy;

i) Developing the Interoperability strategy (aka CIS Integration Strategy (CISIP)).

3.2.8 Political Decisions

3.2.8.1 Combat System costs, and those of the platform itself, are likely to make their procurement the subject of intense media and political interest. In such cases all IPT staff, including the CSM, must not exceed their remit given to them by previous IAB approvals and their CSA with the DEC. The division of procurement responsibility between CDP and VCDS is fundamental to the success of Smart Acquisition. The IPTL and DEC should act in accordance with directions from their respective line managers.

3.3 Inputs

3.3.1 The level of detail collected during the Concept Phase will determine how much additional data collection effort is required during the Assessment Phase. As a minimum, all the assumptions and conclusions from earlier work should be reviewed. If the requirements exist in a structured database, under full Configuration Management, only periodic updates will be required.

3.4 Outputs

3.4.1 Competing Consortia

3.4.1.1 As part of their tenders for D&M, the competing consortia’s deliverables should include:

a) Contract Management Plan (CMP);

b) Risk Management Plan and initial Risk register;


Unclassified 44

c) Quality Plan;

d) Project Management Plan;

e) System Engineering Management Plan (SEMP);

f) Development and Manufacture Management Plan;

g) Integration, Test and Trials Management Plan;

h) Acceptance Management Plan (sometimes linked in with the management of requirements as a Requirements and Acceptance Management Plan (RAMP));

i) Upkeep and Support Management Plan;

j) Availability, Reliability and Maintainability (AR&M) Management Plan;

k) Safety Management Plan;

l) Security Plan for the production of the Accreditation Documentation Set (ADS);

m) Requirements Database (RDB)/Acceptance Database, DOORS;

n) System Specification, (including those for Integration, Development and Training Facilities);

o) Coherence Specification;

p) Subsystem Specifications;

q) Interface Specifications and draft Interface Control Documents;

r) Test and Acceptance Criteria against each System and Equipment requirement supported by some form of Verification Cross Reference Index (VCRI);

s) Integrated Support Plan (ISP);

t) Training Plan;

u) Capability Roll-Out Plan;

v) A list of Government Furnished Assets (GFA);

w) Standardization Implementation Plan (SIP);

x) Payment Milestone Plan;

y) Compliance Matrix referencing their response to each ITT and Statement of Work requirement.

3.4.2 Outputs - CSM’s Team

3.4.2.1 The Assessment Phase will produce documents from several sources, from which the MoD assessment team will generate:

a) ITT Response assessment criteria;

b) COO and Whole Life Costs;

c) Evidence to give a high level of confidence that the required capability of the Combat System design is achievable;


Unclassified 45

d) Documentation to support the clear definition of the system required by the MoD which can be written into a comprehensive SRD;

e) Draft subsystem specifications sufficient to issue to Industry for the Demonstration & Manufacture phase. These must not disallow any satisfactory Combat System design;

f) Thorough project and SE management controls and plans for the remainder of the project, including funded business agreements (e.g. with other IPTs) to support interface management and the supply of GFA;

g) The SEMP which shall address the ‘system-wide’ or transversal issues such as:

• Requirements,

• System Analysis and Design, including design configuration management,

• Integration,

• Interoperability,

• Human Factors (HF),

• Performance,

• Availability Reliability and Maintainability (AR&M),

• ILS,

• Risk,

• Safety,

• Quality,

• Security, et al;

h) A draft integration strategy identifying options for integration and development activities;

i) Draft Combat System acceptance trials policy document plans;

j) A clear definition of, and programme for, the work required during the D&M Phase;

k) Comprehensive cost and timescale estimates for Combat System development, including COEIA data.

3.4.3 Outputs - Lines of Development

3.4.3.1 There are eight recognised ‘Defence Lines of Development’ (DLODs) (reference Joint Customer 2 Handbook) which must be addressed in order to deliver the MC required by the URD. They are:

a) Training;

b) Equipment (the subject of most of this section);

c) Personnel;

d) Infrastructure;

e) Concepts and Doctrine;

f) Organisation;


Unclassified 46

g) Information;

h) Logistics.

3.4.3.2 Equipment Capability (EC) is the primary focus of this Def Stan, however it is essential that an SE approach is applied to all DLODs, both individually and collectively. Delivery of EC and elements of Logistics is the responsibility of the IPTL and CSM. Delivery of other non-EC lines is the responsibility of other stakeholders outside the DPA and who the DEC should have tasked. The DEC normally delegates the responsibility of managing the programming and coordination of all the DLOD deliverables to the IPTL. He, in turn, may subsequently delegate similar responsibilities to the CSM for all DLOD issues related to Combat System user requirements.

3.4.4 Outputs - Consolidation

3.4.4.1 The multitude of data (some conflicting) resulting from the Assessment Phase must be compiled into one set of consistent, coherent documents.

3.4.5 Organisation - Assessment Team

3.4.5.1 It is not the intention of this document to mandate a particular composition of the Assessment Phase teams. Traditionally the CSMs have looked to MoD departments, agencies or consultants for independent technical support. However, the CSM must recognise that all organisations may be competitors to other industrial subcontractors in the commercial market place. The position of any organisation (i.e. whether ‘supplier’ or ‘customer friend’) should be made clear and any track record taken into account.

3.4.5.2 The CSM should also be well aware of potential commercial problems and possible conflicts of interest. The guidance contained in this document may help to significantly reduce such problems.

3.4.5.3 Participation of foreign consortia members is potentially complex particularly in a joint nation development. Lessons learned from any previous co-operative projects should be sought.

3.4.5.4 Assistance from the IPT Commercial Officer should be sought at all stages.

3.4.6 Organisation - Integrated Project Teams (IPT)

3.4.6.1 IPTs empowered to conduct system design studies and trade-offs are essential to Smart Acquisition. The IPT approach involves the participation of all stakeholders including user, customer, and industry representation (where competition permits) from the earliest opportunity. Whilst Def Stan 21-59 acknowledges the need for co-operative stakeholder involvement, it is the responsibility of the CSM to ensure that organisational needs and constraints are wholly appropriate to the needs of the project within the acquisition regime.

3.4.6.2 The IPT Leader (IPTL) shall build upon the core team assembled for the functional capability assembled during the Concept Phase. However, as the Assessment Phase is conducted principally by contractors, the CSM will require a larger team with a specific emphasis on contract and technical management. Under normal circumstances this will be by using either MoD personnel or members of companies that have as little involvement as possible with the constitution of the competing consortia.

3.4.6.3 On occasions the CSM, and the other competing consortia, may need very specialised assistance or expertise that is vested in only one consortium. Dependent upon the type of goods and services in question the contract for Assessment Phase work must be very carefully drawn up to ensure that:

a) The CSM identifies, any specialist goods and services required for the Assessment or D&M Phases that are effectively ‘single source’ so that appropriate commercial negotiations can be entered into;

b) The MoD has the right to use any information that it owns for MoD procurement purposes subject to any commercial caveats agreed with that supplier;

c) All avoidable and unavoidable Government Furnished Assets (GFA) are identified and their associated risks managed.


Unclassified 47

3.4.7 Organisation - Consortia

3.4.7.1 In the selection process for Assessment phase consortia, the CSM shall ensure that:

a) The consortia’s organisation contains the required functional capability in depth for the duration of the work. The functional capability of the CSM’s own team is a good guide to the structure required of the consortia’s organisation;

b) Key individuals, with the necessary functional skills, are nominated to work on the contract. (The CSM’s best opportunity of ensuring that the correct capability is being offered is during the assessment of the consortia’s proposals.)

3.5 Technology Demonstration Programmes / R & D - Technology Survey

3.5.1 As the system requirements are devised and system options are considered, a more detailed technology survey should be considered. This should aim to clarify the extent to which equipment/technology options satisfy the system requirements, their design maturity and the extent of any further development required. Overall the aim of the Assessment Phase is for the equipment/technology underlying options to be at least at TRL 4 (reference TES-SSG Maritime System Maturity Issue 3).

3.5.2 Where commercial sensitivity permits, the IPT will have also reviewed relevant industrial technology developments including R&D programmes. These could include tools and methodologies for system/software development, approaches to adopting modular/open systems architectures for software-intensive systems, and the capabilities of Commercial Off-The-Shelf/Non Development Item (COTS/NDI) software and hardware.

3.5.3 It is the responsibility of the CSM to disseminate the results of any technology surveys and inform of relevant MOD-funded R&D to other MoD and industrial participants in the IPT where security and intellectual property considerations permit.

3.5.4 Rapid prototyping, Simulation, Synthetic Environments and other simulation techniques can be employed to explore and demonstrate equipment capabilities, including their man-machine interfaces. Synthetic environments and other simulation and integration technologies (such as wide area integration) can be used to explore and demonstrate equipment interworking and operations team teamworking issues. Such technologies should be considered if risks are significant in these areas.

3.5.5 Modelling and simulation including synthetic environments also have a potential role to play in system acceptance.

3.6 Key Assessment Phase Processes

3.6.1 Inheritance from the Concept Phase

3.6.1.1 The CSM must conduct the Assessment Phase work in accordance with the approval granted by the IAB at IG. The CSM team will inherit the Shared Development Environment (SDE) and deliverables from the Concept Phase for use as the basis of data for the Assessment phase. While the design process is iterative, the team must not simply accept the work carried out previously without question, neither should Concept Phase be repeated.

3.6.1.2 Inherited documents can be divided into those that will, and those that will not, be updated during the Assessment Phase.

3.6.1.3 Documents which will not be maintained through-life shall be reviewed by the project team and modified in the light of any new/revised information to confirm their suitability for subsequent phases. This will lead to the applicability of key documents being confirmed.

3.6.1.4 The CSM shall make the contents of the project file visible to the competing consortia within the constraints necessary to uphold commercial confidentiality. However, the CSM must remember that certain documentation may well be subject to commercially confidential caveats, which may limit the MOD’s freedom of action.


Unclassified 48

3.6.2 Assessment phase activities

3.6.2.1 Activities are shown on a generic ‘process template’ below. This serves as a guide for implementing the Combat SE strategy. The process is presented as a ‘seamless’ set of SE activities. The precise split between the MoD and the competing consortia for conducting the Assessment Phase studies must be defined by the CSM consistent with the general MoD policy to transfer risk, but not all IPR, to industry.

3.6.2.2 The division of MoD and Supplier responsibilities is such that the MoD assumes the MC requirement definition, system specification and acceptance roles, while the competing consortia assume the proposal of their EC Design Authority and implementation role. Each being responsible for the documentation and deliverables associated with their role.

3.6.2.3 SE (Technical) and Project Management is an activity split between MoD and the consortia contractors. Planning and policy review including the monitoring and review of system designs and documentation should be performed by the MOD. Operational assessment, Whole Life Costs (WLC), contractor assessment and risk assessment are specialist activities which will also be performed by the MoD and for which parallel activities will be performed by the contractor within ‘design selection and assessment’.

3.6.2.4 The overarching MC requirements engineering activities (i.e. requirements management, acquisition, review and endorsement) must be performed by the MOD. The interface between these and contractor responsible requirements and design activities needs to be clearly defined by the Procurement Strategy and SEMP.


Unclassified 49

LIFE CYCLEPROCESS FACTORS

Produceability

Integration and Test

Acceptance and Trials Policy

Support Plans

Disposal Plans


Management Policies and Systems Engineering Management


SYSTEMSANALYSIS

SYSTEM DESIGN

DESIGN ASSESSMENT and SELECTION Technical Performance AR&M Assessment COEIA

Design Selection Design Validation Definition of Systems DesignAssessment Options

SYSTEM STUDIES

DOCUMENTATION


System Requirements Document ITT for Demonstration & Manufacture

Configuration ManagementHuman Factors ILS Safety

Prototyping and Modelling

Tender Production for Development & Manufacture

AR&M

Figure 3.2 Assessment Phase Process

3.6.2.5 The primary activities in the Assessment phase are to:

a) Further develop URD, SRD and all relevant data (including applicable policies, objectives and constraints) for the proposed Combat System from all stakeholders; all requirements acquired should be recorded in the computer-held RDB (i.e. DOORS);

b) Refine and develop the CONOPS/CONEMP and operational scenarios within which the Combat System provides a capability and to indicate what that level of capability is required to be;

c) Gain and promote a thorough understanding of the capability gap requiring solution and the interactions between its different issues;

d) Devise and refine potential candidate design solutions and assess their technical assessment and suitability;

e) Rationalise the range of candidate design solutions which potentially satisfy the identified need and shortlist the most promising options for more detailed consideration;

f) Assess the function and performance of candidate design solutions and ascertain the design drivers and technical risks;

g) Prototype any significant enhancements required in the capability of specific subsystems (which may lead to new development equipments being required) as part of an overall risk reduction exercise ( See DPA PDG Support Group);

h) Further develop the system specification and other documentation to support the generation of the Invitation to Tender (ITT) for Demonstration/Manufacture;

i) ITT Response assessment criteria;

j) Project Review and Assurance (PR&A). Submission of the project’s products and processes to the scrutiny and advice of DPA and DLO Assurance Groups. PR&A serves to clearly define tangible progress made to the benefit of both management and Combat System design team members. It


Unclassified 50

does not diminish the need for conventional project management and other SE management procedures.

3.6.2.6 In order to monitor the completion and review of SE activities, and the endorsement and, where appropriate acceptance of SE products (output products or other standardised project documents/data), it is suggested that an Activity Check List drawn from the TLMP for the Assessment Phase, be used.

3.6.3 Sources of Combat System Engineering Information

3.6.3.1 The Combat System design process involves compliance with a wide range of engineering standards and standard engineering practice. When specifying standards to be applied during later phases, it should be recognised that the majority of existing/mandated equipments will have been developed in accordance with commercial standards. Standards are frequently updated and the CSM must establish the baseline for each standard that he wishes to contract for. The CSM should produce a Standardisation Implementation Plan (SIP). Advice can be sought from PDG. For CIS standards the CSM must complete the JSP 602 Checklist for which advice is available from the Integration Authority.

3.6.3.2 The imposition of specific standards on suppliers, particularly retrospectively, can cause high costs. Some standards will be essential, for example for networking/interoperability reasons, safety, or to make the systems suitable for Operational use. The CSM should aim to structure the project so that the costs and benefits of standardisation are identifiable.

3.6.3.3 The IPT should have already set up a Shared Data Environment (SDE) during the Concept Phase to serve as the central repository of, or signpost to. all relevant Combat System design information. The SDE should provide ready and easy access information such as:

a) The AMS, BRs, CBs, Def Stans, SSPs, STGPs etc.;

b) JSPs, CESG documents, Central Staff documents, DCIs, SSIs etc.;

c) Industry publications from individual companies;

d) Technical publications etc.;

e) Relevant yearbooks, defence equipment catalogues and defence periodicals;

f) Symposia and conference proceedings etc.;

g) International and national standards (BS, EU, ISO etc.);

h) Relevant Government documentation, Acts etc.

3.6.3.4 It is essential that all this information is under configuration management and access controlled to allow the appropriate visibility to each consortia. It must not only allow visibility of the latest information available but also to historical versions which have been used for requirement specification or contract action. Where a conflict arises in precedence of specific documentation, the CSM should resolve it by consultation in a manner which must be auditable.

3.6.3.5 The extent of data to be held in an SDE should not be underestimated. Failure to obtain certain documentation, particularly any which relates to equipments that are either in-service or under development, can lead to project delay.

3.6.4 Systems Engineering Management Plan

3.6.4.1 Smart Acquisition has introduced Systems Engineering practice to all projects.

3.6.4.2 The Systems Engineering Management Plan (SEMP) is central to the Combat System engineering strategy. It defines the intended conduct and implementation of SE technical and project management on a Combat System design project:

a) Establishes the activities and goals of the Assessment Phase;


Unclassified 51

b) Manages the conduct of all technical activities, through either direct action or subcontracts on industry, including their integration to meet the overall technical objective;

c) Ensures that phase objectives are met in programme (timescale) and cost terms;

d) Completes the formal project documentation, which is a pre-requisite for a project to progress to the following phase.

3.6.5 Mandatory Processes

3.6.5.1 Main Gate (MG) approval by the Investment Approvals Board is required before funding is released to the Combat System project to move to the Demonstration and Manufacture Phases.

3.6.5.2 The precise implementation of the generic process must to be tailored to meet the needs and priorities of the project. The SEMP is the most appropriate vehicle for detailing the SE process implementation, which must demonstrate how the approach will effect concurrency and support the overall project business case.

3.6.6 Combined Operational Effectiveness Investment Appraisal (COEIA)

3.6.6.1 This is the formal process by which the Combat System design solution options are compared. It balances the operational value, (primarily performance and programme) of each option with its cost. The approach and process of how this is to be done, should have been established in a Concept of Analysis (CoA) produced in the previous Concept Phase.

3.6.6.2 The outcome of the COEIA is the basis on which the CSM submits to the IAB not only his recommendation for the technical solution but also his estimate for the public funds he is proposing the MoD should commit to spend.

3.6.6.3 The COEIA predicts the achievable MOEs of the design options against agreed scenarios which will have been derived by a cascade process from higher level agreed scenarios through the conduct of operational analysis.

3.6.6.4 The MOEs in terms of which system effectiveness is judged will have been documented as part of the system specification activity, as discussed in the paragraphs on Operational Analysis in this section.

3.6.7 Development, Cost and Risk Assessment

3.6.7.1 The overall technical maturity of warship (and combat system) options should be assessed against the AMS on System (Integration) Readiness Levels and the more domain-specific guidance from the Technical Enabling Services – Sea Systems Group (TES-SSG).

3.6.7.2 System options should be assessed and compared in terms of their Whole Life Costs (WLC) taking into account contributors across all Defence Lines Of Development (DLODs). Cost accounting rules (e.g. using net present value costs, using resource accounting, ignoring sunk costs etc.) must be used.

3.6.7.3 There are always uncertainties in the capture and attribution of costs (especially indirect costs such as those associated with reliability, maintenance and training), in performance data and effectiveness assessment, in development and production risk, and exchange rate variation. This issue can be addressed using a variety of means such as:

a) by employing three point estimates to represent the spread of cost confidence;

b) translating risks into the variation of programme activity schedules and/or cost;

c) employing sensitivity analysis to determine the effect of uncertainties in input data and the robustness of a solution to any variation in assumptions.

3.6.7.4 Further details on the policy which should be employed can be found in the AMS guidance on Business Cases.


Unclassified 52

3.6.7.5 The COEIA implicitly involves Cost trade-off analysis to answer the questions:

a) Can the MoD afford the Combat System as presently envisaged?

b) If so, what is the most cost effective design that meets the requirements?

c) If not, what actions are required?

3.6.7.6 Balance of Investment (BoI) and Cost trade-off activities must never be done in isolation or the absence of affected Stakeholders. Their impact particularly on interoperability must be understood and costed. Such trade-off activities may have started in Concept but a significant difference in the Assessment phase is that a great deal of the source information will have come from design activity carried out by the competing consortia. Each subcontractor will require to conduct mini cost trade-off exercises at each phase of the design. These should be co-ordinated and integrated by their prime contractors at appropriate phases, to enable the final submissions at the end of the Assessment Phase to meet the most stringent cost constraints.

3.6.7.7 In theory, if a particularly costly requirement or problem is identified, the consortia making this discovery should refer this to the CSM and RM who can consider the issue of a change to all Assessment contractors.

3.6.7.8 In practice, competition will normally restrict the willingness of any one consortium to show evidence of problems any earlier than they are required to do so by their contract. The CSM should therefore encourage as free an exchange as possible in this critical area. It is important to appreciate that the cost increases or decreases may affect or be caused by decisions outside the Combat System boundary.

3.6.7.9 The IPTL and CSM are allowed to trade only within the boundaries set by the DEC in his CSA. The RM must be involved in all trading.

3.6.8 System Costing Support

3.6.8.1 The MoD PFG can provide assistance with:

a) Selection of the project cost-trade-off database to be populated by project information so that data can be readily exported to SPS models for cost-of-ownership calculations;

b) Advice on which elements of cost information should be collected, and the granularity and accuracy required;

c) Detailed cost analysis at appropriate project phases when sufficient information is available;

d) ‘Quick-look’ analysis - either to address particular design decisions or to balance technical options.

3.6.8.2 Detailed cost trade-off analysis is required at certain phases of the design and when major decisions regarding direction are required. This is particularly true of inputs to the EP and decisions on affordability. The quality of the costing advice provided by PFG is significantly dependent upon the quality of raw cost data available both within the project and more widely in the MoD. However, if SPS provides the means and techniques to assess the cost trade-off analysis there must be no expectation given to any consortia that any inaccuracies in their supplied cost are accepted by the MoD and that the associated risk is deemed to rest with MOD.

3.6.8.3 In general, the assistance of PFG in addressing cost issues will require anticipation so that suitable resources and appropriate data can be used. The CSM will have difficulty in ensuring that SPS assistance is available precisely when the prime contractors/subcontractors need it. CSMs should strive to make the service available for their projects despite the potential problems identified above.

3.6.9 Acceptance Strategy

3.6.9.1 The DEC is the Acceptance Authority and he alone is responsible for his Acceptance Strategy. He should have identified an initial Acceptance Strategy in the CSA, prior to Gate Zero. He can of course be assisted in its production and development by the Capability Working Group, the IPT and the CSM. The


Unclassified 53

CSM is responsible for proposing to the DEC, usually via the RM, the IPT’s Integrated Test Evaluation & Acceptance Plan (ITEAP) in which he will state how he will present evidence to the DEC so that the DEC is suitably informed to make the decision whether or not to grant System Acceptance. The CSM will establish the Acceptance criteria against which the MC, EC and project deliverables will be accepted. The will include the overall system, documentation, equipment hardware, software or firmware, integrated system, operability, training manning levels and skills, AR&M, support, etc. To achieve this, an Acceptance Strategy will need to be developed during the Assessment Phase.

3.6.9.2 The ITEAP shall be produced in accordance with the AMS. The ITEAP must reflect the project acquisition and design strategies and take into account the spend profile for the project. It should consider the contributions of the other Defence LoDs towards System Acceptance and In-Service Date.

3.6.9.3 The ITEAP should include a funded plan to address modelling, integration, test and trials and how the acceptance process will utilise data generated by other activities. The CSM must define the role that modelling will play in the acceptance process. Modelling can be used in two phases:

a) To predict the system performance that the Combat System can achieve;

b) To compare that predicted performance with the product’s achieved performance.

3.6.9.4 The CSM must provide advice as to how the modelling process is linked to the acceptance process by the test and trials plans.

3.6.10 Modelling, Integration, Test and Trials Strategies

3.6.10.1 In addition to helping system development, Modelling and Simulation (M&S) including synthetic environments also have potential roles to play in addressing integration issues and, as mentioned above, in supporting system acceptance.

3.6.10.2 Integration is a crucial combat systems engineering activity. The CSM should consider the Integration Strategy in terms of:

a) Integration responsibilities;

b) The extent to which integration should be conducted ashore and onboard ship;

c) The need to use dedicated shore facilities, for initial integration and possibly through-life in support of continuing system development - such facilities may use a combination of real and simulated equipments together with stimulators, scenario generation, and recording, replay and analysis capabilities;

d) The time needed for conducting combat system integration and the impact this has on equipment procurement schedules;

e) The potential need to order an additional equipment shipset (or use a shipset intended for a later vessel in the class) for conducting integration;

f) The potential use of virtual integration facilities using synthetic environment technologies to link development/actual equipments, for example, located at manufacturers’ premises (i.e. ‘wide area integration’).

3.6.10.3 M&S should be used to address equipment interworking, interface and integration issues for the combat system options under active consideration during the Assessment Phase. The use of M&S to consider such ‘design for integration’ issues should be reflected in the M&S Plans.

3.6.10.4 M&S, demonstration, inspection, test and trials are all methods for determining whether a combat system performs as required. Each has its part to play and offers distinct advantages (e.g. veracity, low cost) and disadvantages (e.g. high cost, reduced veracity). There will be certain elements of combat system functionality which only M&S can be used to confirm system acceptability and in these cases more traditional tests and trials can be used to verify and validate the behaviour of the models and simulations.


Unclassified 54

3.6.10.5 The CSM should consider the scope of the system requirements and how satisfaction of each requirement will be ascertained, individually and collectively. The CSM must define the role that M&S will play both in the acceptance process and in predicting the system performance that the Combat System can achieve.

3.6.10.6 The CSM shall ensure that the Combat System modelling activities are compatible with the requirement on him to supply and maintain a model of the Combat System’s capability to the Integration authority for inclusion in MODAR and MODAF.

3.6.11 Requirements Engineering (RE)

3.6.11.1 KURs are those Military Capability requirements or constraints identified from within the wider set of user requirements which are assessed as key to the achievement of the mission, or which are for some reason assessed as of particular interest to management. The CSM shall ensure that all KURS are met.

3.6.11.2 KSRs are requirements (normally restricted to EC) critical to system cost, performance, time or risk, which provide management indicators of overall system performance. Again the CSM shall ensure that all KSRs are met.

3.6.11.3 A purpose of RE during the Assessment Phase is to generate sufficient detail until such time as physical solutions emerge and stabilise into a definitive set of requirements. All requirements activities will be continue to a degree commensurate with the design activities, as determined by the RM. For guidance on the specifics of related activities refer to Section 2 - Concept Phase.

3.6.11.4 The constituent elements of the RE process needed for the Assessment phase are documented in the AMS .

3.6.11.5 The Requirements Database (RDB) produced by the CSM during the Concept phase will be an important method of technical communication between organisations and will require the use of an SDE.

3.6.11.6 The CSM should regularly update the RDB to reflect any changes that have been agreed with the DEC or RM, e.g. those resulting from related sub-system and equipment development programmes that may be progressing in parallel. Whilst the DEC is ultimately responsible for ensuring harmonisation of URDs and SRDs across these programmes, it is the CSM’s responsibility to bring to his attention any inconsistencies. A comparison and reconciliation exercise is recommended to ensure commonality and compatibility of assumptions and design goals.

3.6.11.7 DOORS is the MoD’s preferred structure, format and toolset for Requirements Databases. Depending on its implementation during the Concept Phase, (see the CSM should mandate DOORS as the RDB within the contract. The RDB held by the CSM’s team, must be the single repository for:

a) User and System Requirements;

b) Traces between requirements and the candidate design solutions;

c) Criteria against which the requirements will be offered for acceptance;

d) Acceptance processes to demonstrate that the acceptance criteria have been met.

3.6.11.8 The CSM shall require a soft copy of each contractors RDB to be delivered as part of the output of the Assessment Phase and shall insist that all such deliverables are compatible with the format and structure of the original MoD RDB. The CSM shall ensure that the contractors accept that it is their responsibility to ensure such compatibility.

3.6.12 Requirements Management

3.6.12.1 The URD is owned by the DEC. The SRD is produced by the IPT and CSM but it is endorsed by the DEC. The RM, as the DEC's representative within the IPT, must be involved in all requirement planning and change management.


Unclassified 55

3.6.12.2 Requirements Management is concerned with maintaining the integrity of User and System requirements as they evolve over time. It involves maintaining an audit trail of requirement changes, assessing the impact of requirements changes and identifying the inter-relationships between requirements.

3.6.12.3 Requirements Engineering Planning involves the support and maintenance of the RDB scheme, the preparation of the information and configuration control system and the definition of the requirements engineering procedure (and associated tools, if used).

3.6.12.4 Requirements Change Management is concerned with the formal and rigorous management of Combat System requirements once they have been established. It is closely associated with project change/CM procedures. During the Concept phase the RDB is placed under the project CM regime to track the evolution of the requirement. Once the URD has been formally issued at the end of Concept Phase, the RDB is frozen under formal configuration and change control procedure to act as the endorsed baseline for the subsequent phases. The CSM shall establish a project change control board to process and approve all subsequent changes to the RDB during the subsequent phases of the project.

3.6.12.5 Any modifications of the requirements definition need to be promulgated to all interested parties in a disciplined and well-documented manner.

3.6.12.6 Requirements Change Management involves:

a) Operating the project change control board, including obtaining authorisation to change any specific requirements statement;

b) The implementation of endorsed changes to captured requirement statements and their associated data (such as contextual information and classification fields) in a disciplined and controlled manner;

c) Authorising the release of the updated baseline RDB. The CSM shall only authorise such a release via the IPT Commercial Officer.

3.6.12.7 Requirements Traceability Management - this ensures that requirement statements are actioned by the relevant systems engineering activities and disciplines, and implemented in the Combat System or its design process, as appropriate. It involves the recording of links between the RDB and models, documents etc. as used in relevant activities. Typically these links are expressed in the form of trace tables or additional fields in an expanded RDB. Trace mechanisms should ideally facilitate tracing in both directions.

3.6.12.8 Effective requirements change management requires traceability links - to facilitate both the impact analysis of changes to requirements and also the subsequent actioning of the impact of a change in requirements (for example, by the amendment of the corresponding elements in analysis, design and assessment models). if anomaly reports are be used to initiate changes to requirements, then these can be linked-in to requirements change management.

3.6.12.9 As a result of the design activities certain requirements may prove to be impractical to achieve either due to technical or cost constraints. The CSM and RM are responsible for reviewing the implication of these results with the originator of that particular requirement and, if agreed, changing the contents of the RDB and promulgating changes to all relevant authorities.

3.6.12.10 The CSM should, where practical, avoid re-issuing the RDB during the later stages of the Assessment Phase. Throughout the life cycle phase and particularly in circumstances where re-issue is unavoidable, the CSM shall implement strict configuration control through the project configuration control board, remembering that:

a) The RDB must be authorised by the Acceptance Authority (especially where a relaxation of the endorsed requirement is being proposed);

b) Competing consortia are most likely to have developed individual, specific instantiations of the original RDB, which may now require amendment. Such activities may be reflected in additional costs/risks/programme delays to MOD. (Any contractor-specific instantiation of the RDB should conform to DOORS in terms of structure, schema and consistency).


Unclassified 56

3.6.13 Stakeholder, Domain and Scenario Analysis

3.6.13.1 The IPTL and CSM shall produce a Stakeholder Responsibility Matrix and form funded business agreements with those stakeholders to ensure delivery of the DLODs.

3.6.13.2 During the Concept Phase, CONEMP/CONOPS and subsequent scenario analysis will have identified high level scenarios and associated events. These can be used to provide context and clarification of user (and system) requirements. During the Assessment phase, further consideration and refinement of these scenarios assists the requirements engineering and systems analysis activities (particularly Operational Analysis (OA)).

3.6.14 Acceptance Evidence

3.6.14.1 The CSM shall require the competing consortia to provide acceptance data related to each requirement :

a) Acceptance evidence comprising:

i) Evidence that will be offered to show that the requirements has been achieved,

ii) Supporting information,

iii) Specific test limitations;

b) Acceptance event, which will be classified as one of the following types, at which the acceptance evidence will be presented:

i) Review,

ii) Audit,

iii) Analysis (including calculation and mathematical modelling),

iv) Inspection,

v) Demonstration,

vi) Test (including Factory Acceptance Trials),

vii) Shore Facility Trials,

viii) Harbour Acceptance Trials (HATs),

ix) Sea Acceptance Trials (SATs).

3.6.15 Systems Analysis

3.6.15.1 Systems Analysis is the application of scientific principles to the study of a problem domain or system in order to:

a) Obtain a better understanding of it;

b) Identify the component elements, their inter-relationships and impact on key characteristics and other system issues;

c) Systematically identify and assess alternative courses of action.


Unclassified 57

DecompositionAggregationAbstraction

BehaviouralAnalysis

Consistency &CompletnessAnalysis

RequirementsStructuring

LogicalModelling

Refinementof OperationalScenarios

Refinement ofSystem MOEs

FurtherParametricPerformance Analysis

OperationalAnalysis

Production ofLogical Model

Production ofSystemSpecification

Definition ofSystem MoPs

SystemSpecification

SystemsAnalysis

Figure 3.3 Composition of Systems Analysis Activity

3.6.15.2 Systems Analysis during the Assessment Phase is principally concerned with developing an in-depth understanding of the problem domain and potential solution space using modelling and analysis techniques. The composition of the Assessment Phase systems analysis activity is illustrated in Composition of Systems Analysis Activity. Each of the constituent activities is described in turn below.

3.6.15.3 Logical Modelling

3.6.15.3.1 Logical modelling can be performed of the functional user and/or system requirements to help establish their consistency, (internal) completeness, structure/level and absence of ambiguity.

3.6.15.3.2 A number of techniques (methods) are suitable for the conduct of logical and functional modelling. Up-to-date advice on all aspects of tool support should be sought from experts in the field. Computer support environments are prone to obsolescence and the MoD is littered with models which can no longer be supported due to the demise of particular hardware or Operating System. Consortia will always press for models and support tools to be subject to IPR restriction. The CSM should insist that all work funded by MoD has full MoD-user rights so that it can be used without hindrance throughout the project lifecycle.

3.6.15.4 Operational Analysis (OA)

3.6.15.4.1 Operational Scenarios - Further refinement and specialisation of vignettes taken from SAG scenarios will be used to evaluate different system options within the levels of operational capability required and the CONEMP. The resulting vignettes shall be used during subsequent project phases to provide a defined framework for evaluating deliverable capability concerning the vessel’s warfare and non warfare-related roles. These scenarios will also be used as the benchmark for the equipment and Combat System test, trials, evaluation and acceptance programme to provide the link between Equipment and Combat System performance and operational effectiveness as described in Measures of Effectiveness (MOEs) for the COEIA.

3.6.15.4.2 For the purposes of warfighting effectiveness assessment, the following information relating to vignettes will need to be defined:

a) Roles and tasks of the vessels within the scenario;

b) Tactics to be employed within the scenario together with Rules Of Engagement;

c) Geometry of the engagement;

d) The physical environment (land topography, meteorology, oceanography etc.);


Unclassified 58

e) The characteristics of vessels, e.g.; signature: broadband, narrowband, tonals; speed and endurance; weapons and countermeasure capability; sensors and command management systems performance, physical operational constraints such as blind arcs.

3.6.15.5 Measures of Effectiveness/Performance

3.6.15.5.1 Initial MOEs will have been developed during the Concept Phase and shall be developed in line with scenario definition and URD completion. The aim should be to quantify all areas of required capability in terms of specific, verifiable MOEs.

3.6.15.5.2 Measures Of Performance (MOPs), and other quantified system requirements, shall be devised as the SRD is developed. These should at least initially aim to satisfy the URD MOEs but may need to be traded as the potential solution space is refined. Means of MOP verification will need to be determined.

3.6.15.5.3 Parametric performance analysis techniques can be used to define the overall solution space and potential trade-offs between the performance characteristics of different system options. They can be a powerful technique to use within the Assessment Phase.

3.6.15.5.4 Within the sets of individual user and system requirements, those which are assessed as key to the achievement of the mission need, or which are important for project progression and management, will be assessed as Key User Requirements (KURs) and Key System Requirements (KSRs) respectively. Numbers of KURs and KSRs should be kept quite low, with the precise number depending upon the range of operational roles the vessel performs and its range of functional capabilities. KURs and KSRs provide an indication of critical system elements and the boundary limits to trade-offs.

3.6.15.6 Operational Analysis techniques

3.6.15.6.1 Advice on the use of Operational Analysis tools and techniques which are suitable for their intended purposes can be obtained from Dstl Naval Systems.

3.6.16 System Design

3.6.16.1 Scope

3.6.16.1.1 System Design is the process of developing candidate design solutions, capable of meeting both the operational and functional requirements as stated in the system specification.

3.6.16.1.2 It also involves the examination and adoption of one or more system structures/architectures, consideration of non-functional requirements, constraints, HF issues and mandated/candidate equipments.

3.6.16.1.3 To fulfil the requirements of the IAB at MG, it is usually necessary to demonstrate that, at least initially, the Assessment Phase (if not done previously in the Concept Phase) has considered a wide range of system design solutions. Thereafter it is normal to concentrate on the most viable, capable, affordable and cost-effective solutions and explore variants of the preferred design (or designs) concentrating on progressively lower-level design issues.


Unclassified 59

SystemVariants

SystemInformationExchange

Equipment DataCollection

Specificationof Software

SystemFunctional

Design

SystemArchitecture

Non FunctionalAnalysis

SystemInterfaces

InstallationEngineering

SystemStudies

DesignVerification

Policy Paper Refinement

Coherence Specification

Subsystem Specifications

DesignDocumentation

SystemDesign

Figure 3.4 Composition of System Design Activity

3.6.16.1.4 The composition of the system design activity is illustrated in Figure 3.4. In practice, iteration between requirements engineering, systems analysis and system design, will be required.

3.6.16.2 System Functional Design

3.6.16.2.1 By the end of the Assessment Phase, each competing consortia should provide sufficient confidence that their solution will work and meet the KURs and KSRs within their estimated Whole Life Costs. This should be demonstrated by a prototype of each system. However in many cases a comprehensive series of computer models will have to be acceptable.

3.6.16.2.2 The procedures and disciplines for the Concept Phase design process should be carried through to the Assessment Phase. However the emphasis changes from Logical Modelling of the requirement by mapping to a Physical model of a proposed implementation. The CSM is responsible for monitoring the process.

3.6.16.2.3 This work should be carried out using tools and techniques similar to those identified in earlier stages. The CSM shall make available to the competing consortia computer-readable copies of the models developed during the Concept Phase or Assessment Phase 1, but shall avoid mandating their use.

3.6.16.2.4 The final selection of tools and techniques for the Assessment Phase is a decision that the CSM shall leave to the competing consortia. However, the CSM shall require each competing consortia to deliver a detailed justification for the tools and techniques proposed, including information on the selection and assessment criteria applied.

3.6.16.2.5 Functional Design is the application of functional modelling techniques supported by other SE disciplines e.g. HF; Performance Engineering; AR&M Engineering etc. to derive one or more physical models of system design concepts. It should be noted that specific design approaches (e.g. design for performance, design for reliability, design for supportability, design to cost) can be applied where corresponding non-functional requirements are particularly demanding.

3.6.16.2.6 The derivation of a physical model from a logical model together with design and implementation requirements and constraints (as contained in the RDB) uses a mixture of two distinct but complementary techniques:

a) Candidate equipment matching - the matching of required functionality to that of candidate and / or mandated equipments;


Unclassified 60

b) ‘Greenfield’ design - the application of a translation procedure to derive the functionality for man and machine – viz: operators and new development. Translation involves the allocation of function and must take HF considerations into account.

3.6.16.2.7 The result of functional design is a number of physical models, each of which represents the functionality and real-time behaviour of candidate design options. Each option should be practical and potentially capable of implementation.

3.6.16.2.8 For a new Combat System, the physical models should reflect both ‘off-the-shelf’ (possibly with tailored interfaces) and new development equipments together with the functionality performed by specified members of the ship’s complement who are to operate and utilise the Combat System.

3.6.16.2.9 In the case of an update to an existing Combat System, the physical model may reflect the entire Combat System or be limited to broadly the extent of the update. The extent of the update must be addressed as it sometimes proves to be greater than first apparent when equipment design characteristics, performance issues etc. are taken into consideration. Elements of the Combat System, which interface with areas covered by an update generally, require some degree of modelling.

3.6.16.3 Variants of Combat System Physical Models

3.6.16.3.1 The Assessment Phase is initially aimed at producing feasible options from which a preferred design will emerge. The design process will generate a number of different physical models, each one representing a different view of the functionality of the required combat system. The lowest level assembly physical models are grouped to form equipment models, Subsystem models and ultimately the single global system, the total Combat System model.

3.6.16.3.2 Physical models enable the tracing of requirements through to specifications. They support the numerate translation of requirements to specifications by identifying the ‘actual' hardware, software and human components of the overall system. Control of these models enhances the traceability of the overall design and permits the generation of variants of the design for mixes of equipment on different platforms.

3.6.16.4 Information Exchange Requirements (IER)

3.6.16.4.1 With the completion of the physical model solutions the high-level information flow between the candidate subsystems can be identified. The CSM shall capture the detailed system Information Exchange Requirements (IER) and data exchange requirements (DER) to ensure that the architecture studies can be completed.

3.6.16.4.2 Ideally IERs should initially be captured in both JIFM and iSMART and associated assumptions captured. All terms used should be lodged with CDMA for inclusion in the Defence Data Repository (DDR). Advice on the formal documentation of the system data / information exchange requirements by the CSM is available from (ref to www.fdm.dii.r.mil.uk).


Unclassified 61

3.6.16.5 Equipment Data Collation

3.6.16.5.1 For mandated or candidate equipment identified within the functional design the CSM shall assist the competing consortia in collating the equipment data. Equipment data should not be restricted to functionality, performance, AR&M, space, weight, required services, heat etc. The information gathered during the Assessment Phase should build upon that previously obtained at Concept Phase.

3.6.16.5.2 Where a specific equipment (e.g. navigation radar) has been identified, a market survey should be undertaken to identify candidate equipments. Existing project databases and defence publications (yearbooks, defence equipment catalogues and periodicals) are useful in conducting this.

3.6.16.5.3 For each of these equipments, the following information should be collated:

a) Test and Acceptance Criteria;

b) URDs, SRDs;

c) Equipment Specifications;

d) Interface Specifications and draft Interface Control Documents (ICD);

e) Comprehensive technical documentation covering functionality, performance, AR&M, weight, space, through life cost etc.;

f) Interface documentation;

g) Model and database data;

h) Trials Reports;

i) Alteration and Addition (A&A) details, including variants.

3.6.16.5.4 This material will be useful in conducting candidate equipment matching within the functional design activity. The Combat System Engineering Database may be a useful source of information.

3.6.16.5.5 Based on the system studies, the CSM shall require the competing consortia to translate the functional design into a candidate architecture, i.e. physical solution. For the proposed physical solution to the Combat System requirement it is necessary for the competing consortia to:

a) Conduct detailed evaluations on proposed solutions in order to resolve errors of incompatibility where the functionality offered by the recommended solution does not provide requisite system effectiveness within the scenarios provided;

b) Optimise the proposed designs to ensure that maximum capability is provided for the minimum investment in time, cost and areas of risk;

c) Feedback the results of the evaluations into the other areas of system design.

3.6.16.5.6 The level of detail required of the candidate architecture shall be sufficient to allow all of the necessary design documentation to be produced without imposing unnecessary constraints on the equipment solutions. That is, existing equipment can be characterised by its BRs and CBs as its implementation is known and accepted. However, new development equipment should be characterised in terms of its specific requirements. In this manner the implementation risks are flowed through the competing consortia to equipment suppliers, who in time will become the equipment design authorities.

3.6.16.5.7 The CSM shall require that the competing consortia deliver all functional and physical models, and any variants thereof, in both hard copy and computer-readable format complete with a description of the operating environment required to host the models.


Unclassified 62

3.6.17 System Studies

3.6.17.1 Systems Analysis & Modelling

3.6.17.1.1 Systems analysis and modelling techniques can be employed to address the completeness and consistency of a set of requirements such as a logical description of a system. These techniques tend to concentrate on functionality and leave non-functional issues to manual analysis. Checklists can be used to ensure that non-functional issues have been addressed.

3.6.17.1.2 Systems Analysis can be described as the application of scientific principles to the study of a problem domain or system to obtain a better understanding and identify the component elements, their inter-relationships and impact on key characteristics and other system issues. It also helps to systematically identify and assess alternative courses of action.

a) The complexity of modern Combat Systems makes production of prototypes extremely costly both in time and financial terms. For many aspects of Combat Systems design, modern computer technology and SE techniques offer a cost-effective alternative to prototyping by providing the means to `model' the system;

b) Modelling is the process of deriving a representation of some aspect, or aspects or a real-world entity, being representative to a particular level of fidelity, i.e. accuracy, detail etc. sufficient for the purposes for which it was constructed.

3.6.17.1.3 Computer Modelling can be used to:

a) Structure functional requirements for analysis of consistency and completeness and;

b) Promote greater understanding of what functions the Combat System is required to perform.

3.6.17.1.4 Logical modelling provides a logical, structured, unambiguous, descriptive, complete, consistent and correct statement of required functionality in the form of a logical model. Ideally, nothing in a logical model states how the Combat System is to be implemented.

3.6.17.1.5 Needlines and Information Exchange Requirements (IER) should be modelled e.g. in JIFM or iSMART (or predecessor i.e. TULIP).

3.6.17.1.6 Whatever modelling is conducted for the Combat System an output from that modelling must feed into MODAF.

3.6.17.1.7 Advice on modelling tool-sets and techniques and their relative suitability should be sought from established MoD sources, as some may be freely available to the project through corporate licence agreements. Lock-in to proprietary modelling tools must be avoided.

3.6.17.2 System Requirement Specification

3.6.17.2.1 Having conducted logical modelling of the User capability required, a clear and consistent view of what is required of the new/updated system should be obtained during the Assessment Phase. The purpose of the system requirement specification activity is to specify this formally in a SRD.

3.6.17.3 Combat System Concept Design

3.6.17.3.1 Combat System Concept Design is the process of developing candidate high-level Combat System design solutions, which are likely to be capable of meeting both the functional and non-functional requirements captured in the SRD). It also involves:

a) Consideration of the non-functional/design/implementation requirements and constraints covering both features in the Combat System and how they are to be developed or implemented;

b) Examination and adoption of one or more system structures/architectures;


Unclassified 63

c) Consideration of HF issues and mandated/candidate equipments and all other transversal (e.g. safety, security) issues.

3.6.17.3.2 During Assessment Phase, the aim is to generate as wide a range as possible of potentially viable Combat System design solutions. In the later life cycle phases, the system design solutions will concentrate on the most viable (capable, affordable and cost-effective) of these solutions and explore variants of the preferred design (or designs) concentrating on progressively lower-level design issues.

3.6.17.3.3 In practice during the course of this work, some iteration between Assessment Phase design, systems analysis and requirements engineering, will be inevitable.

3.6.17.4 System Structure Studies

3.6.17.4.1 Structures are the ways system components may be assembled. Currently there are three structures in Naval Combat System data handling which are either in use or under consideration, as follows:

a) Centralised structure which comprises one (or a small number) central functional processor(s) carrying out all data handling compilations and control functions;

b) Federated structure which comprises a number of subsystems which are autonomous in data handling to a certain degree, but can still be controlled to a certain degree by one (or a small number) controlling computer(s);

c) Distributed structure which divides the data processing load via the processors in the system without using fixed or central points.

3.6.17.4.2 In practice, a continuum of hybrid structures exist.

3.6.17.4.3 There are also several initiatives on Open Systems Architectures sponsored by various DECs and applicable to naval combat systems and their constituent equipment. Advice should be sought from DEC CCII who has core DEC responsibility for Interoperability.

3.6.17.4.4 System Structure Studies should take into account:

a) Design and implementation requirements and constraints;

b) Mandation of specific equipments;

c) Vulnerability considerations;

d) Data management considerations;

e) Information processing considerations including load, flexibility and re-configurability.

3.6.17.4.5 Other SE considerations:

a) HF to ensure that functionality is allocated correctly between man and machine functions;

b) Performance engineering;

c) AR&M engineering - Def Stan 00-41 gives guidance on such matters.

3.6.17.5 Formulation of Design Policy

3.6.17.5.1 Formal project documentation should include:

a) Design Policy Papers and Coherence specifications - to identify issues and their constituent factors, formulate policy and to state guidelines, rules and requirements. They adopt a management and high-level technical perspective;


Unclassified 64

b) All design policy papers should at least be scoped and where possible progressed to draft issues during Assessment Phase.

3.6.17.5.2 From the above work, the CSM will have one or more candidate structures which can be used for assembling system components in functional design.

3.6.17.5.3 The CSM shall conduct, or shall ensure that competing consortia conduct, architecture studies to:

a) Create possible physical solutions that meet the logical requirements for the Combat System;

b) Subject these possible solutions to modelling studies that resolve the question of ‘how well does solution A meet the requirements and is it better than solution B?’.

3.6.17.5.4 Trends in design, including trends in systems architecture (e.g. open and modular architectures) and COTS equipment, and trends in the military marketplace must also be considered. Within the design time of the project will specific solutions, which are not achievable at present, become possible? This is particularly critical in areas such as database techniques, processing power and display technology where MoD and Industry can have very similar requirements.

3.6.17.5.5 The results of these studies feed into other areas of the design effort, and contribute to the other outputs of the Assessment Phase.

3.6.17.6 Architecture Modelling

3.6.17.6.1 By the end of the Assessment Phase, the technology to be used to implement the Combat System will be reasonably firm. The CSM shall require the competing consortia to carry out, and deliver the results of, thorough modelling studies to demonstrate that their chosen solution will meet all the system requirements. Architecture modelling should concentrate on the proposed physical designs and investigation of infrastructure and software architecture issues.

3.6.17.6.2 Architecture modelling will validate the system physical models. The competing consortia, guided by the project team, will be required to evaluate the capabilities of these designs, within the agreed scenarios.

3.6.17.6.3 Architectural modelling should study the following to expose potential architecture difficulties, operability requirements, and system interface, communications and computer power requirements:

a) Communication bandwidths;

b) Processor speeds and capabilities;

c) Trade-off analysis of bandwidth, data volumes and processor responses;

d) Data transfers and boundary conditions between subsystems;

e) Safety implications of system architectures;

f) AR&M implications of architecture;

g) Operability conditions at subsystem interfaces;

h) Data grouped by different security classes;

i) Real time data transfer around the system;

j) Response times;

k) Data storage requirements;

l) Human computer interactions;


Unclassified 65

m) Data communication bandwidths;

n) Trade-off analysis of bandwidth, data volumes and processor responses.

3.6.17.6.4 The degree to which the above aspects can be considered will depend on the maturity of the supporting data and design configuration. It is likely that the necessary detail will only become available during the course of the Assessment Phase, and even then with voids in specific areas which will require documented assumptions.

3.6.17.6.5 Analysis of this type permits review of potential architecture difficulties, operability requirements, and system interface communications and computer power requirements.

3.6.17.6.6 Each physical solution must be analysed for dependencies such as processor bottlenecks, communication delays and dataflow transfers. To understand fully a system design, it is necessary to stimulate the architecture and monitor the various dependent elements. The design detail available is likely to be restricted and incomplete at this stage, hence the level of firm detail for modelling studies is correspondingly limited, requiring assumptions to be made. Such an approach is both valid and useful, as long as the assumptions made are both realistic and fully documented, so that the modelling studies can be refined as further design detail emerges.

3.6.17.6.7 Certain high-level requirements, principally reliability, maintainability and performance, will be budgeted in the form of lower level targets, where satisfaction of all the lower level targets would be equivalent to satisfaction of the high-level requirement. The CSM shall ensure that AR&M and performance budgeting, is undertaken through the apportionment and allocation of the overall project budgets, such that the results can be assigned to each of the candidate solutions developed. This is an iterative process.

3.6.17.6.8 While much of this work will be carried out by the competing consortia, the initial identification and decomposition of the budgets, as well as the assessment that the subsequent apportionment and allocation does comply with these budgets will be conducted by the CSM’s team and tracked through the RDB.

3.6.17.7 System Interfaces

3.6.17.7.1 Following the completion of the system architecture studies the CSM shall document the resultant system interfaces. For the Assessment Phase it is sufficient to identify the system interfaces by type, i.e. RS422, Combat System LAN, etc. Detailed interface specification should be produced at this phase in readiness for their translation into Interface Control Documents (ICD). Assumptions should be based upon de-facto MoD documentation schemas (e.g. Def Stan 21-13) where tools are already in place for through-life support.

3.6.17.7.2 The CSM shall map the system information exchange to the system interfaces to provide an indication of the complexity of the data exchange across the interface. Understanding the complexity of the data exchanges is a key factor in the risk assessment of the system architecture. A good system design requires robust protocols which will continue to work during fault conditions and with faulty data. It is recommended that Data Management and Tactical Data Link (TDL) expertise be sought early (e.g. www.fdm.dii.r.mil.uk).

3.6.17.8 Installation Engineering

3.6.17.8.1 During the Assessment Phase preliminary installation engineering is limited to:

a) Developing layouts for operational spaces;

b) Developing layouts for the outboard equipment fit;

c) Collating basic equipment data and budgets to ensure that the platform systems are adequately designed: e.g. Chilled water; Electrical power; Ventilation and Air Conditioning; Hydraulics; Pneumatics; Weight, space, maintenance access;

d) Hydrodynamic studies for below water Combat System equipments;

e) Electromagnetic Environmental Effects (E3) studies for Electromagnetic sensors and emitters (Def Stan 08-136 applies and guidance should be sought from DCSA E3 section);


Unclassified 66

f) Structural studies to ensure that the equipment can be installed within the platform.

3.6.17.8.2 Various modelling tools are available for these activities; choice should take into account the databases, models and techniques used for requirements capture, structured design, architecture design and Human Factors.

3.6.17.9 Design Verification

3.6.17.9.1 Design Verification is the checking that the design concepts (expressed as physical models) and resulting from the functional design activity, correctly reflect:

a) User and System requirements, including agreed trade-offs;

b) Approved documentation describing the functionality and design characteristics of mandated and candidate equipments where adopted as part of the Combat System design;

c) Formal design decisions.

3.6.17.9.2 The verification of activities and related project documents/data should be undertaken only when they have been completed and brought under formal CM. The execution of design verification should be recorded.

3.6.17.9.3 A cycle of project reviews shall be used by the CSM as the mechanism for ensuring that the competing consortia offer their Combat System design for design validation.

3.6.17.9.4 The adoption of formal mechanisms such as trace tables for linking requirements and constraints on system designs to the designs themselves greatly facilitates the conduct of design validation. Design validation, i.e. the process of confirming that the offered design is 'fit for purpose' in the sense that it is capable of fulfilling the roles envisaged, is a much more complex and open-ended process than design verification and is best addressed in an ongoing manner through formal design review mechanisms required by the SEMP.

3.6.17.10 Design Documentation

3.6.17.10.1 In parallel with design solutions, design issues should be identified, policies formulated and implementation rules stated. The CSM will document during assessment:

a) Policy Papers; used to identify issues and their constituent factors, formulate policy and to state guidelines, rules and requirements. They adopt a management and high-level technical perspective;

b) Combat System Coherence Specification; (1) the possession of consistency and integrity, (2) the adoption and maintenance of system-level rules which define key features of operating procedures and interfaces within a system. More than one may be needed depending on the diversity of system options. They should lead to a more uniform, integrated and complete Combat System. The coherence specification should evolve into a formal technical specification to accompany the subsystem specifications;

c) Combat System Subsystem Specifications; updated as part of the system analysis activity. Following on from this, draft subsystem specifications based on the candidate architectures shall be developed;

d) Combat System Anomaly Reports; produced as the formal mechanism for documenting and managing the resolution of problems and issues;

e) Combat System Design Variants Record. - The process of deriving and assessing design options, key assumptions, the conduct of system design activities (including the modelling techniques used), the derivation, evolution, evaluation and iteration of design options and discussions held, and a record of decisions taken and any associated studies which have affected the course or conduct of the Assessment Phase.


Unclassified 67

3.6.17.11 Platform Installation Studies - Budgets

3.6.17.11.1 At the start of the Assessment Phase, the CSM shall negotiate with the DEC and platform project in respect of Combat System budgets in the following areas:

a) Space available in specific compartments;

b) Weight budgets;

c) Power budgets by type and standard, i.e. 24V, 115V 60Hz, essential supplies, non essential supplies, etc.;

d) Chilled water budgets;

e) Hydraulic power budget;

f) Pneumatic power budget;

g) Internal and external constraints on the Combat System;

h) Heat budget;

i) Timing budgets;

j) Accuracy budgets;

k) Alignment budgets;

l) Yaw, Pitch and Roll budgets.

3.6.17.11.2 Throughout Assessment Phase, the CSM shall keep both the platform project and the contractors appraised of the developing system budgets, with particular emphasis upon (perceived) design compliance and acceptance of those budgets as the candidate Combat System architectures develop. The CSM shall attempt to retain a margin between the budgets agreed with the platform authority and the budgets made available to the competing consortia, especially if there are emerging requirements whose budgets are not yet defined. The CSM shall require that the competing consortia complete the Combat System design within such budgets.

3.6.17.12 Weapon Equipment Schedule (WES)

3.6.17.12.1 The CSM shall require that the competing consortia document the Combat System budgets in a WES. Further, the CSM shall require that the WES be delivered at intermediate stages during the Assessment Phase to facilitate negotiations with the platform project to meet the Combat System requirements. The WES is one of the controlling documents for the Combat System.

3.6.17.13 Internal Arrangements

3.6.17.13.1 The CSM shall draw upon earlier Human Factors (HF), complementing and platform general arrangement studies to:

a) Inform the platform project of the proposed internal arrangements for the Combat System operational spaces (recognising DEC and RM responsibilities);

b) Agree with the platform project the interface details of the Combat System operational spaces with respect to the platform.

3.6.17.13.2 The CSM shall provide the platform project with sufficient detail to allow conflicts between Combat System equipments and other platform systems to be identified and resolved.

3.6.17.13.3 The CSM shall require the competing consortia to deliver draft layouts for the operational spaces within the platform indicating the fitting arrangements for the Combat System. These layouts shall


Unclassified 68

include sufficient detail for the CSM to have confidence that the Combat System can be fitted in to the available space.

3.6.17.14 External Arrangements

3.6.17.14.1 The CSM shall advise the platform project of the Combat System requirements concerning its external arrangements and shall agree the final layouts for each competing consortium’s Combat System solution. The CSM shall provide the platform project with sufficient details to allow the resolution of conflicts between Combat System equipments and other platform layout constraints.

3.6.17.14.2 The CSM shall require the competing consortia to deliver draft layouts for the fitting of the Combat System equipment to other parts of the platform, e.g. flight decks on carriers, areas external to the pressure hull on a submarine. These external arrangements shall include sufficient detail for the CSM to have confidence in the Combat System platform interface requirements.

3.6.18 Life Cycle Process Factors

3.6.18.1 During the Assessment Phase, there is a need to build upon policy and plan documentation co-ordinated through the TLMP and SEMP. At this phase, the following topics need to be revisited to document policy in line with the evolving Combat System design:

a) Procurement strategy;

b) Implementation and production issues, including adoption of COTS, and approach to software-intensive system components;

c) ITEAP;

d) ILS/Logistic Support Analysis (LSA) strategy; Further advice should be sought from the Support Solutions Envelope / Maritime Support Strategy (DLO / TES);

e) Capability Roll-out plan;

f) Disposal plan.

3.6.19 Life Cycle Process Factors - Integration Strategy

3.6.19.1 The contractors have to formulate their integration strategy plan to the point where firm price development of the chosen method can be undertaken. In the extreme case of a new Shore Integration Facility (SIF) the site must have been established and evaluated as fit for purpose. All buildings and services must have been designed to a state where they can be accurately costed. The environment for the Combat System must have been designed to a sufficient level of detail. This detail will establish the design of the scenario generators, the recording and analysis equipments and the simulation / stimulation requirement for each Combat System equipment. There must also be a coherence specification and subsystem specifications for each equipment’s respective element of the SIF. There also has to be a programme showing that the SIF will be built, integrated, and accepted before its first usage in the Combat System integration plan.

3.6.19.2 A Combat System integration plan has to be formulated to show the development of integration test/trial orders and the sequence of integration tests that are to be undertaken to ensure the integrity of the Combat System design and to prepare the Combat System for acceptance; this plan should include both shore and ship tests/trials. It is most important that the Combat System contractor can make all equipment suppliers aware of the extent of their obligations to support Combat System integration when negotiating equipment contracts.

3.6.19.3 The issues in the integration strategy plan should be further explored. Whilst avoiding integration at a SIF may appear to be the cheapest ‘ least work’ option, this also represents the highest risk to the Platform project. New technology solutions may also have to be explored and recommendations made regarding their adoption and their impact upon a traditional SIF-based approach. Wide-area networking at ‘slower than real time’ may offer an opportunity to test protocols and basic functionality thereby reducing the risk of poor design.


Unclassified 69

3.6.19.4 The intended longevity of the SIF is an important issue. It may be required for the initial development only, or it may be required for through-life support of the class.

3.6.19.5 The SIF test environment will contain some form of scenario generator to drive simulators or simulators interfaced to the Combat System equipments. The simulators and stimulators may be included in the equipment contracts, and if so, there is an opportunity to use them for on-board training, maintenance diagnostics, and system set-up. As already implied, a SIF also needs extensive data recording and analysis facilities. The integration programme may also benefit from having several networks available in the SIF to enable parallel testing and so shorten the trials programme.

3.6.19.6 An outline Combat System integration plan should show the sequence of integration tests to be undertaken to ensure the integrity of the Combat System design and to prepare the Combat System for acceptance; this plan should include both shore and ship tests/trials.

3.6.19.7 The method and plan must be taken into account in this phase's Combat System costings and risk assessment. The form in which they are passed into the next phase will depend upon MOD's intended relationship with the competing contractors - an integration method might be mandated, indicated as preferred, or left to the contractor. If not mandated, any other method or plan offered will be compared with the reports of this phase to see if there are any shortfalls in a contractor's offering.

3.6.20 Assessment and Selection

3.6.20.1 A suggestion for the assessment and selection process of the candidate design solutions is shown in Figure 3.5. All the following factors must be taken into consideration:

a) Operational effectiveness;

b) Technical assessment and performance;

c) AR&M;

d) Development, cost;

e) Risk;

f) Affordability;

g) Supportability.

3.6.20.2 The preferred options will be subject to formal COEIA evaluation and a recommendation made to the IAB at MG. The IAB should not being asked to select a solution development.

DesignSelection

Technical Risk

Identification of Scientific&Technology Difficulty

TechnicalFeasibility

Assessment

Definition ofEquipmentPerformanceCharacteristics

TechnicalPerformanceAssessment

ARMAssessment

OperationalEffectivenessAssessment

OverallOptionAssessment

Development,Cost & RiskAssessement

Refinementof COEIA

Design DevelopmentFeasibility Option Record

InterfaceDocumentation

Outline SystemAgreed Characteristics

Outline SystemAcceptance Questionnaire

System DesignSpecification

DesignAssessment& Selection

Figure 3.5 Design Assessment and Selection


Unclassified 70

3.6.20.3 System design, design assessment and selection are iterative processes to `close the design loop' and identify any options which would benefit from further exploration and refinement to effect improvements.

3.6.20.4 Formal documentation is completed to reflect the design assessment and selection process and to document the most promising design options (i.e. the highest ranking options in the COEIA).

3.6.20.5 Design Selection

3.6.20.5.1 From the functional perspective the CSM shall compare the solutions against a defined set of criteria, such as those identified below;

a) Duplication of functionality between subsystems;

b) Degree of Subsystem autonomy;

c) Resilience to External Failures;

d) Complexity of Interfaces;

e) Complexity of Data Exchange;

f) Inter-equipment Data Rates;

g) Data Storage Requirements;

h) Size and complexity of the Resultant Subsystems;

i) Operator numbers, workload and skill levels;

j) Interoperability.

3.6.20.6 Technical Assessment

3.6.20.6.1 In support of the recommendation to the IAB it must be shown that project and programme risk has been reduced to a manageable level that ensure successful ISD delivery. This requires the Assessment Phase to provide a comprehensive measurement of the technical risk through scientific and technological assessment of the candidate design solutions in terms of Technology Readiness Levels (TRLs). Such assessment should be thorough and captured in a ‘risk register’ relating to each design. Technological assessment will cover hardware and software areas and shall consider if:

a) The technology is available now;

b) The technology is being developed as part of another programme;

c) The technology will be developed as part of another programme, which is yet to be approved;

d) The maturity of system-wide and integration issues should also be addressed using System Readiness Levels (SRLs).

3.6.20.7 Performance and Effectiveness Assessment

3.6.20.7.1 The achievable performance of candidate Combat System designs against MOPs can be determined by aggregating the lower-level performance achievable by individual equipments taking into account interworking and interference factors as necessary. The degree of satisfaction of the user requirement MOEs will need to be ascertained either through understanding the correlation between MOPs and MOEs or by directly assessing the achievable MOEs using Operational Analysis tools.

3.6.20.7.2 Such performance assessment is often conducted as part of an integrated approach to performance and effectiveness assessment whereby performance measures can be defined which if satisfied confer a level of capability, which leads to satisfaction of the Combat System effectiveness


Unclassified 71

requirements. This serves to facilitate the specification of required system capability in performance rather than effectiveness terms for procurement and acceptance purposes.

3.6.20.8 AR&M Assessment

3.6.20.8.1 Each candidate design solution will be subjected to AR&M assessment. AR&M prediction techniques should be employed.

3.6.20.8.2 Design Options - If design variants are being considered, a Design Development Assessment Option Record documents the assessment and shortlisting of variants of one or more baseline Combat System designs. It is therefore often known as the Combat System. This document provides a historical record of the conduct of a study which should be appended to the design options/variants record, identifying:

a) Key assumptions made;

b) Conduct of Combat System design activities;

c) Limitations of any techniques used;

d) Development of output products and other standardised documents/data;

e) Design baselines (a system baseline is the documented, approved description of the system at a specific time) and other relevant configuration control issues;

f) Relevant discussions held, decisions taken and associated studies which have affected the conduct or course of the study.

3.6.20.9 Interface Documentation

3.6.20.9.1 The CSM shall develop preliminary interface documentation for the Combat System. During the Assessment Phase, this shall be limited to:

a) Operational spaces layouts for the candidate designs. Sufficient detail to confirm that it is viable to fit the design within the platform;

b) Information Exchange Requirements. These identify the information needs of each subsystem without the need to complete the detailed Interface Specifications (IFS) and Data Exchange Specifications (DES).

3.6.20.10 Acceptance Documentation

3.6.20.10.1 By the end of the Assessment Phase, the ITEA Plan should:

a) Meet the maturity level expected by PR&A;

b) Embrace all of the options being presented in the Main Gate Business Case;

c) Be sufficiently robust to underpin the WLC and schedule estimates presented in the MG Business Case;

d) Inform the SRM and Asset Delivery Schedule;

e) Be tailored to the options remaining at MG;

f) Identify risks, cost and schedule drivers;

g) Identify demands on test resources and other Government-Furnished Assets (GFA);

h) Define any further Test & Evaluation (T&E) activity to be performed prior to supplier selection (if applicable);


Unclassified 72

i) Address interoperability issues;

j) Define the distinction and relationship between Contract and System Acceptance.

3.6.20.10.2 BR 4050 contains full guidance on Acceptance matters.

3.6.20.10.3 Validation Criteria relate to the user requirements, concern military capability (across all Lines Of Development) and are expressed in terms of MOEs. Verification Criteria relate to system requirements, concern singular DLODs such as equipment provision and are typically expressed in terms including MOPs. It is the Verification Criteria which will form the basis for System Acceptance and associated with each of the Verification Criteria shall be the mechanisms by which the Combat System MOPs (and other system measures) will be offered for acceptance. Acceptance may include the use of M&S techniques and, in the case of areas of capability for which model-based acceptance is going to be employed, the validation of those models will need to be explicitly addressed.

3.6.21 Specialist Discipline Support

3.6.21.1 A number of specialist activities which have significant downstream implications on the cost and effectiveness of the evolving Combat System need to be considered during the Assessment Phase as follows:

a) Air Engineering;

b) Alignment;

c) AR&M;

d) CM;

e) Cost;

f) Documentation;

g) Electromagnetic & Environmental Effects (E3);

h) Environmental Impact Assessment (see ‘Environet’ on the MoD intranet);

i) HF/Operability;

j) ILS/Supportability;

k) Installation and Ship Services;

l) Information Management, Interface/Data Management;

m) Modelling & Simulation (M&S), including Synthetic Environments;

n) Naval Architecture;

o) Operational Performance/Effectiveness;

p) Safety;

q) Security;

r) Shock and Vibration;

s) Software Development;

t) Structural Engineering;


Unclassified 73

u) Training.

3.6.21.2 Configuration Management

3.6.21.2.1 Configuration Management (CM) is an important theme of the AMS, essential to keep track of requirements, specification and plans which define the Combat System baseline design and the acquisition and support processes through which it will be realised. The emphasis of CM during Assessment activities is to enhance and develop the framework within which trade-offs can be managed effectively, whilst enabling/retaining the flexibility to explore options and evolve solutions. Consideration is required as to how CM will be applied and managed across the various levels of the combat system (from system-level, through equipments to software, hardware and firmware components) and through-life. Def Stan 05-57 mandates high level CM requirements for all projects.


3.6.21.3.1 The contractors have to formulate and declare their "design for integration" approach, with a detailed explanation of how information passing between systems is to be managed. The data management standards for the definitions of application data elements, co-ordinate systems, network interfaces, network protocols, and application protocols have to be included in their coherence specifications.

3.6.21.3.2 Convincing supporting evidence for their choices should be supplied, demonstrating their suitability for the Combat System real-time environment. Details of the intended tool support for application data definition, communications profile compliance testing, and interface application data testing should also be provided. In particular, the performance and characteristics of all chosen network profiles should be exposed in considerable detail to minimise the risk of an inappropriate choice.

3.6.21.3.3 The contractors have to formulate and declare:

a) Their overall management of the development of the inter-equipment interface specification documentation set;

b) The way in which an equipment's compliance with the coherence specification, Data Exchange Specifications (DES), and Interface Specifications (IFS) will be assured during Development;

c) How they will undertake related compliance testing in factory tests, thus ensuring each equipment is "fit for integration".

3.6.21.3.4 The contractor's stage outputs are a coherence specification and a stated method of managing interfaces. The coherence specification must be judged on its "completeness" and the credibility of the supporting evidence for the choices therein. The management of interfaces must judged on its credibility and its visibility to MoD during development.

3.6.21.3.5 Current standards for the definitions of application data elements, co-ordinate systems, network interfaces, network protocols, and application protocols can be used as benchmarks to reduce the risk of a replacement being inappropriate to the Combat System real-time environment. The consequences of replacement standards upon interface documentation practice and upon tool support need to be carefully assessed (i.e. application data definition tools, communications profile compliance testing tools, and application data testing tools). An indication of the performance required from any networks should emerge from the various modelling processes and indication of required integrity characteristics will drive from the Combat System AR&M and vulnerability requirements. The process for the overall management and development of the inter-equipment interface specification documentation set and its compliance testing needs to be established.

3.6.21.3.6 These issues will impact on the costings and the risk assessment made in this phase. The results of these investigations will be reflected in the phase outputs. The form, which they take, will depend upon MOD's intended relationship with the competing contractors and the prevailing MoD policy on standards. Selected standards may appear directly in the coherence specification as being mandated or preferred. The coherence specification may merely state the areas in which standardisation is recommended (in which case the work of this phase would be in reports and used by MoD to assess a contractor's choice). Similarly the consideration of the management of IFS documentation and interface compliance testing may be passed to the Assessment contractors or it may be retained by MoD for use in assessment of the contractor's proposals.


Unclassified 74

3.6.21.3.7 To permit effective competition for in-service and through-life activities, the MoD should ensure that all data exchange standards are freely available to all equipment suppliers, support agencies and other stakeholders without IPR constraints which might limit their use. This also applies to data management support environments, test equipment, data recording & analysis equipment, data exchange specifications and the data / information itself.

3.6.21.4 Human Factors Integration

3.6.21.4.1 Assessment Phase HF studies should include:

a) Suitability of designs in the light of HF developments over the course of the project;

b) Optimising cost trade-off between HF (including training and skill levels) against system and subsystem designs;

c) Carrying out subsystem operability and inter-operability evolutions.

3.6.21.4.2 Embracing HF integration into Combat Systems design provides a number of benefits, including improved technical solutions and visibility within the DPA of through life costs.

3.6.21.4.3 The approach defined in STG Publication 11, Human Factors Guide for Management and Design in Royal Navy Combat Systems, is intended to ensure that full account is taken of the human element in system design. Systems which are an aggravation to their users not only require huge resources to ameliorate them, but are also highly detrimental to morale and ultimately Naval recruitment. HFI therefore has a large impact on overall effectiveness. Historically, Ships’ Staff have had to endure system designs which would not be tolerated in other commercial industries.

3.6.21.4.4 The Human Factors Integration Plan (HFIP) is crucial and specifies both the planning, co-ordination and technical design work for HF and is divided into two parts:

a) Part 1 - The Management Plan;

b) Part 2 - Technical Information and Specifications.

3.6.21.4.5 The HFIP is compiled during the Concept Phase (see Human Factors) for implementation in all subsequent phases.

3.6.21.4.6 During Assessment, the HFIP needs to be expanded and enhanced to include:

a) Review of the applicability of tools chosen during Concept Phase;

b) Cost trade-off analysis is required to validate the location of functionality within the system. The conclusions of the HF studies require to be integrated with those of other aspects of the design.

3.6.21.4.7 Following the STGP 11 guidelines HFIP Part 1 is to be updated and HFIP Part 2 progressively compiled.

3.6.21.4.8 Also of particular concern to the CSM at this phase of the design process is the impact of equipment design on the operator numbers and skill levels. This has significant external effects as well as the internal project considerations.

3.6.21.4.9 External effects include:

a) Platform considerations on accommodation size and balance between ratings/senior ratings/officers;

b) Impact on ship size, engine power, fuel, endurance etc.;

c) Impact on training/recruitment/retention and availability of training billets; effects on sea/shore ratios and long term branch numbers.


Unclassified 75

3.6.21.5 Operability Analysis and Assessment - The CSM shall require that, with the system design process carried out by the competing consortia, analytic operability assessments of the candidate designs be carried out.

3.6.21.6 Operability analysis shall be of sufficient depth to provide the following information concerning each of the candidate designs:

a) Platform Procedures;

• Operations Areas,

• Readiness States,

• Watchkeeping Policies,

• Manning Levels/Outline Scheme of Complement;

b) User Characteristics;

• Anthropometric Dimensions,

• User Levels,

• Cognitive Capability,

• Specific Skills,

• Naval Experience and Training;

c) User Role Definition;

• Operator Positions,

• Combat System Roles,

• Combat System Role Interaction,

• External Role Interaction,

• Training Roles;

d) User Interface Standards, (to be included within the coherence specification);

• Displays,

• Dialogue Style,

• Environmental Standards.

3.6.21.7 This analysis shall be based on the operational performance scenarios described earlier in this Section under Performance and Effectiveness Assessment. This analysis shall be carried on into D&M.

3.6.21.8 Integrated Logistic Support (ILS)

3.6.21.8.1 ILS shall be applied to all future system/equipment procurement, major upgrades, off-the-shelf procurement etc. in accordance with Def Stan 00-60.

3.6.21.8.2 During the Assessment Phase, ILS activities include:

a) The formation of an LSA management team (if appropriate);


Unclassified 76

b) Updating COO and WLC estimates. These should be indicative, allowing an informed judgement on the solution, together with its support package ensuring optimum value for money;

c) Update and further refine the ILS strategy documents started during Concept Phase;

d) Conducting LSA supportability assessment tasks and in particular trade-off analysis. This will contribute significantly to the decision on the preferred contractor;

e) Attending the bidders conference. Producing ILS ITT/Statement Of Work (SOW) for contract award for full development.

3.6.21.8.3 Wherever practicable, demonstrations of support capabilities should occur.

3.6.21.8.4 The availability of data suitable to populate the tailored Logistic Support Analysis Record (LSAR) will begin to arrive, maintenance policies will be emerging and remaining support risks will be being managed.

3.6.21.9 Availability, Reliability and Maintainability (AR&M)

3.6.21.9.1 AR&M issues identified in Concept Phase outputs form the basis of Assessment Phase activities. During the early part of Assessment, the AR&M concept panel should agree assessment criteria for marking the AR&M sections of the full development tenders. The AR&M project panel then takes over after formal endorsement of the URD and has many of the same members as the concept panel, but is chaired by the IPT Leader.

3.6.21.9.2 During Assessment the contractor must:

a) Prepare preliminary AR&M plans including component selection control procedures, initial AR&M predictions, subsystem AR&M targets, AR&M development plans, sub-contract AR&M plans, and identified AR&M resources and costs;

b) Carry out detailed design and related AR&M activities including listing approved and non-approved components, applying component control procedures, devising an initial critical items list and life limited items list, applying a component de-rating policy, refining AR&M models, refining and updating AR&M predictions, and checking design redundancy;

c) Define the AR&M strategy for full development, including planning development phase AR&M tests.

3.6.21.9.3 The terms of reference and therefore main tasks/activities of the project panel are:

a) To assist the IPTL in defining the Reliability and Maintainability (R&M) aspects of the specification including the assessment criteria for the tender proposals and the method of gaining sufficient assurance that the R&M requirements have been satisfied;

b) Provide suitable input to the IAB MG submission;

c) To recommend amendment or adoption of formal R&M plans, programmes and trials, prepared by the contractor;

d) To ensure that the relevant details from these plans and programmes are included as integral parts of the overall development cost plans;

e) To recommend verifiable milestones to monitor progress in R&M;

f) To monitor R&M aspects of the results of development and user trials and of maintainability and testability assessments and contractor progress against achieving R&M contract requirements;

g) To agree an assessment of achieved R&M levels for acceptance purposes;

h) To enable trade-off studies between AR&M and cost to be undertaken, with the aim of striking the optimum balance;


Unclassified 77

i) To provide advice to both IPTL and DEC on any proposed amendments to project timescales or targets subject to contract and cost factors. To highlight any conflict found to exist between the various R&M requirements and detailed performance requirements, or where significant shortfalls become apparent, and to discuss the in-service effects of any amendments;

j) To assess the results of in-service R&M demonstrations, where these are called for as part of the Acquisition or Procurement Strategy.

3.6.21.9.4 The costs associated with AR&M activities require to be fed into the cost trade-off analysis as other results of the design activities.

3.6.21.9.5 The outputs from the AR&M activities form inputs into ILS. AR&M is a key input into the IAB submission documents.

3.6.21.9.6 The Reliability and Maintainability (R&M) Panel performs the following functions:

a) Analysis of environmental and operating (mission) conditions;

b) Definition of the maintenance concept and support constraints;

c) Investigates AR&M contracting strategy and demonstration methodology;

d) Reviews AR&M assessments, predictions and trade-off studies conducted by the consortia;

e) Refines AR&M requirements and their numerical values, environmental and operating conditions for inclusion in the SRD.

3.6.21.9.7 R&M studies are undertaken within other life cycle processes (principally in system design and design assessment and selection) concentrating on analysis, design for R&M, and prediction/assessment (hardware and software). Trade-offs between AR&M, performance, cost and in-service date are conducted. The results of such investigation feed into the COEIA and then into the MG Business Case.

3.6.21.10 System Security

3.6.21.10.1 Security is normally considered as a platform-wide subject. However, only the Combat System aspects are addressed here. Security can have a profound impact on the Combat System design, for example concerning the adoption of COTS technology.

3.6.21.10.2 During the assessment Phase the regular liaison initiated during the Concept Phase with the DSSO Accreditor should continue.

3.6.21.10.3 The system security issues shall be fully addressed by the CSM who shall provide the Accreditor with a plan as to how and when the Accreditation Document Set (ADS) will be provided either by the IPT or the competing consortia e.g. by provision of :

a) An InfoSec scoping appraisal;

b) A draft System Electronic Information Security Policy (SEISP);

c) A draft Security Operating Procedure (SyOPs).

3.6.21.11 System Safety

3.6.21.11.1 The adoption of a safety case procedure is a mandatory requirement for all future warships. It is required to encompass both platform and Combat System aspects.

3.6.21.11.2 The safety cells within MoD offering advice on system safety and software safety is DOSG. See JSP 430 Annex F - Specialist Authorities for Ship Hazard Areas.


Unclassified 78

3.6.21.11.3 The “SAFETY CASE” - The adoption of a safety case management procedure is now a mandatory and legal requirement on all IPTLs and projects. It is required to encompass both platform and Combat System aspects.

3.6.21.11.4 For systems which will contain Ordnance, Munitions & Explosives:

a) JSP 520 Pt 1 – OME Safety Management System;

b) JSP 520 Pt 2 - Operational Procedures;

3.6.21.11.5 The Defence Ordnance Safety Group (DOSG) is now part of the TES organisation. The web site: www.ams.dii.r.mil.uk/content/docs/dosgweb contains further guidance.

3.6.21.11.6 For issues concerning Platform safety, the DOSG Technical Support (DOSGTS) Safety Management Office (SMO) should be consulted. Relevant standards are as follows:

a) Ships – JSP 430;

b) Air – JSP 553;

c) Land – JSP 454.

3.6.21.11.7 The procedure required to generate the ‘Safety Case’ is described in the AMS

3.6.21.11.8 The mandated procedures are contained in:

a) POSMS – Project-Oriented Safety Management System;

b) POEMS – Project-Oriented Environmental Management System.

3.6.21.11.9 The safety case serves to demonstrate that the whole warship meets agreed safety objectives. It should ensure that a clear audit trail exists from design through procurement, operator and upkeep to disposal. In addition, a summary of the safety case, known as the safety case report, is prepared to assist in the reviewing of performance and the provision of authority to proceed to the next phase of the life cycle.

3.6.21.11.10 The CSM shall require each competing consortium to produce a safety case for their proposed warship Combat System solution. This shall be produced in accordance with JSP 430.

3.6.21.11.11 After selection of the Combat System solution, but still during Assessment, the CSM shall update and re issue the Combat System safety case in accordance with JSP 430 - Ship Safety Management System Handbook.

3.6.21.11.12 The CSM should assure himself of the following as a minimum during the Assessment Phase:

a) No significant changes have occurred in legislation or in related documents that mandate a change in the safety policy for the Combat System;

b) No significant additional tools or methods have become available in the interim since the start of the Assessment Phase that would ease the task of safety validation;

c) That all the consortia are giving the correct emphasis to ensuring that the system will not do something unsafe but also that when required it will fulfil its defined function.

3.6.21.11.13 The cost and timescale implications of the need for high integrity safety critical software, and the standards, against which it is to be identified and developed, need to be reviewed. These standards are available from the Director of Standardisation, www.dstan.dii.r.mil.uk.


3.6.21.12.1 The CSM shall produce the Combat System data management policies to manage the digital and analogue data interfaces within the Combat System. When developing these policies the CSM shall take into account that these polices are to be applied to:


Unclassified 79

a) New subsystems - can be applied in total;

b) Modified subsystems - can be applied if change is cost effective;

c) Un-modified subsystems - can only be applied if change is cost effective and may well define what polices can be implemented.

3.6.21.12.2 Advice should be sought from current practitioners in this specialised field. For those with access to the MoD Intranet, www.fdm.dii.r.mil.uk provides further reading.

3.6.22 ITT for Development and Manufacture

3.6.22.1 The CSM shall ensure that the MoD intention to issue an ITT for development and manufacture is included in the MoD contracts bulletin in accordance with the departmental procedures.

3.6.22.2 During the Assessment Phase, the CSM team, including the commercial and finance functions, shall prepare the ITT for development and manufacture.

3.6.22.3 The IPT Commercial Officer will issue the ITT for development and manufacture.

3.6.22.4 Tender Selection

3.6.22.4.1 The CSM’s task is to ensure that the competing consortia, and particularly the lead contractors, have properly assessed the scope of the Combat System development and are suitable organisations to undertake the contract. The CSM’s tasks are to effect:

a) Contractor Assessment;

b) Risk Assessment;

c) Requirements Compliance;

d) Refinement of the COEIA.

3.6.22.4.2 Further details on tender assessment can be found on the AMS.

3.6.22.5 Contractor Assessment

3.6.22.1 The CSM should undertake an assessment of the potential prime contractors to ensure that they have the financial stability necessary to undertake the contract with all its attendant risks.

3.6.22.2 The CSM should require that the competing consortia conduct similar assessments of their potential sub-contractors. The CSM shall require that the results of such assessments be provided to assist in the assessment of the potential prime contractors.

3.6.22.3 Contractors who fail to convince the CSM of their suitability as a prime contractor should not be provided with the ITT for D&M. This is a very contentious issue and the advice of director of contracts should be sought at all times.

3.6.22.4 Down selection of competing consortia can occur at any point in the assessment process unless this would leave a single tender situation with outstanding points of clarification. Final down selection should not occur until the CSM has finalised all points of clarification.

3.6.22.5 In a competitive situation, the CSM shall avoid cross fertilisation of the details of the solutions proposed by the competing consortia.

3.6.22.6 Risk Assessment

3.6.22.6.1 CSM shall produce a project risk assessment in accordance with the Contract Management Plan (CMP) for Assessment. This assessment can draw on, but must not be limited to, the risk assessments produced by the competing consortia.


Unclassified 80

3.6.22.6.2 The CSM’s risk assessment shall include the risks associated with the installation and integration of the Combat System within the platform. As a rule, the competing consortia are usually unable to address this aspect of the risk assessment to sufficient depth.

3.6.22.7 Requirement Compliance

3.6.22.7.1 The CSM shall assess the degree of compliance offered by the competing consortia against the RDB/acceptance database. This assessment shall include:

a) Management Proposal;

b) Technical Solution;

c) Software Architecture, Development Process, Metrics and Progress Monitoring;

d) Performance Assessment;

e) AR&M Assessment;

f) Shipfit Aspects;

g) Compliance with Standards;

h) Integration, Test and Trials Process;

i) Acceptance Philosophy;

j) External Dependencies;

k) Any other commercial factors (e.g. IPR issues).

3.6.22.7.2 If in doubt, the CSM must seek clarification from the competing consortia, rather than make an assumption.

3.6.22.8 Problems with Lowest Cost Compliancy

3.6.22.8.1 If one system is significantly cheaper than another, then the CSM must question why.

3.6.22.8.2 Once the CSM has established this, he can then advise properly on the final choice of the Combat System. If it is not possible to determine why a particular bid is cheaper than another is, there is may be a significant risk to the project.

3.6.22.8.3 There may be many reasons why consortium may produce a low price for a system. Included are:

a) More efficient solution and/or better technology;

b) Lower profit margins than competitors;

c) Lower labour rates than the opposition;

d) Lower corporate overheads;

e) Willingness to accept lower contingency for risk;

f) Under-estimate of scope of task;

g) Over-estimate of scope of task by competitors;

h) Deliberate loss leader for related business and/or subsequent anticipated profitable changes to specification. This would be common where the platform and Combat System were both being bid for by the same consortium;


Unclassified 81

i) Profits in other sales e.g. exports.

3.6.23 SRD Production

3.6.23.1 Initial Preparation of SRD

3.6.23.1.1 Initial drafting of the SRD may be conducted in parallel with the work of the competing consortia during assessment Phase activities.

3.6.23.2 SRD Drafting

3.6.23.2.1 The draft SRD will pass through a number of drafts and iterations before issue. Industry should be permitted to comment on its contents.

3.6.23.3 SRD Completion

3.6.23.3.1 The SRD is generally produced on a whole warship level, being compiled from the Combat System and platform-related drafts. Harmonisation of material and re-drafting/answering of specific comments can be expected as the draft passes through progressively higher level review.

3.6.24 Assessment Phase Consolidation

3.6.24.1 Introduction

3.6.24.1.1 Two competing consortia will most likely conduct the Assessment Phase. Each consortium will be working in isolation from the other although each may be utilising common subcontractors (particularly for mandated equipments). The IPTL and his team will work to ensure that each consortium is provided with appropriate information and access to expertise.

3.6.24.1.2 To minimise risk and to ensure the cost of the Assessment Phase is kept to budget, the IPTL will not be in a position to alter the scope of the task during the Assessment Phase.

3.6.24.1.3 On completion of the Assessment Phase, the IPTL will receive a complete set of the output products from each consortium. These outputs should be a complete and very comprehensive design of the Combat System.

3.6.24.1.4 During the consolidation period the two designs and their associated costs are assessed and then a firm/fixed price contract is awarded to one of the two consortia to proceed to the development and production stage with their design and programme.

3.6.24.1.5 The CSM will have estimated the resource required to complete consolidation and identified the specialist resources necessary in the Assessment Phase MoD CMP, as part of the submission for the Assessment Phase.

3.6.24.2 Development and Manufacture Stage Input

3.6.24.2.1 During consolidation all the documentation for a properly structured contractually binding input to the development and production stage has to be defined. Many of these will be based upon the products of the Assessment Phase.

3.6.24.2.2 The inputs to development and production assume that the chosen consortium will implement their design from Assessment.

3.6.24.3 Consolidation Stage

3.6.24.3.1 The aims of the consolidation stage are as follows:

a) To assess the two consortia's offerings and to select a winner;

b) To provide a comprehensive assessment of risk (for MoD use);


Unclassified 82

c) To develop the MG Business Case in order to get funding approval for the D&M Phase;

d) To incorporate MoD initiated changes to requirements, scenarios, contractual, financial, or procedural aspects;

e) To incorporate impact of the platform design evolution;

f) To formally publish the User and System Requirements Documents;

g) To provide details of the scope of and the programme for the acceptance process;

h) To identify a preferred contractor;

i) To de-brief the failing consortia on the reasons for their rejection;

j) Identify the capabilities and resources required within the MoD for the succeeding D&M Phase.

3.6.24.3.2 Timescale - The consolidation stage will have a realistic minimum duration dependent on the number of appropriately skilled personnel available and the work to be carried out. Early completion is to be encouraged otherwise the consortia may be forced to disband their teams and assign them to other work, thereby prejudicing the validity of their bids.

3.6.24.3.3 There is no reason why the consolidation stage cannot start before the Assessment Phase completes. With phased deliverables from consortia there will be a large amount of data for early work to progress.

3.6.24.3.4 Tasks - During consolidation, the CSM is to make no major adjustments to the designs but he is to ensure that the consortia have included everything that is needed. The CSM shall also ensure that commercially sensitive information from one consortia is not revealed to the other. The documents must properly describe the total Combat System, contain its development and integration programme, and supply the cost information for the design.

3.6.24.3.5 The prime task of this stage is the assessment of outputs from the consortia. Prior to the commencement of this assessment, the CSM must have developed a comprehensive marking scheme to be used in the process. The scheme must be fair and objective; it provides the justification for selecting the winning consortia and for rejecting the losing consortia; it will be the main source of information for de-briefing the losing consortia; it must withstand close scrutiny if the losing consortia challenge MOD's decision.

3.6.24.3.6 Following the identification of the preferred consortium, all the necessary contractual documentation for the next phases must be prepared. This will also involve some negotiations with the chosen consortium.

3.6.24.3.7 It may be necessary for the CSM to incorporate MoD initiated changes to requirements, scenarios, contractual, financial, or procedural aspects or to incorporate the impact of the platform design evolution. In both cases this might involve renegotiating the price offered by the preferred consortium when they submitted their costs for providing the Combat System.

3.6.24.3.8 Another task to be completed is preparing the submission for funding for the next two stages, the winning consortia's cost estimates will be one of the inputs, the CSM will have to make a judgement of any contingency required and any other costs that may be incurred which are not the consortia's responsibility.

3.6.24.3.9 The CSM must identify the programme of work, size, and make-up for the MoD team for the next two stages along with any supporting resources (e.g. computers, accommodation). He will also have to generate the MoD CMP to cover these stages.

3.6.24.3.10 Consolidation Stage Team –

The PM should set up a team that encompasses the following functions:

a) CSM;


Unclassified 83

b) Technical Co-ordinator (including system modelling and acceptance expertise);

c) Project Planning;

d) Risk Specialist;

e) Requirements Manager;

f) Administrative Co-ordinator;

g) Software Specialist;

h) ARM Specialist;

i) ILS Specialist;

j) Quality Specialist;

k) Safety Specialist;

l) Security Specialist;

m) Commercial Officer;

n) Finance Officer;

o) Secretarial Assistance.

3.6.24.3.11 If any personnel are seconded or contracted from industry there must be confidentiality agreements and suitable arrangements to ensure that there will be no conflict of interest.

3.6.24.4 Outputs from the Consolidation Stage

3.6.24.4.1 Internal Review Documents - Objective and subjective evidence should be gathered into the SDE to provide the audit trail for prime contractor selection and MG submission. With the complexities of modern procurement it is considered that all suitable data is collected to assist in the correct eventual decision. With the very best contract staff in the world the selection process will still involves an element of "can we trust the contractor to do what he has promised?".

3.6.24.4.2 With the prospect of more than one IPTL and many team members over the lifetime of the Combat System evolution it is important that the relationship of trust is built up and documented during the project.


Unclassified 84

This page intentionally left blank


Unclassified 85

Systems Engineering Guide for Naval Combat Systems – Demonstration and Manufacture Phase

4 Demonstration and Manufacture Phase

4.1 Introduction

4.1.1 Demonstration and Manufacture phases are rarely serial activities, particularly in a complex system acquisition where they are hugely interdependent. Therefore this document addresses D & M phases together. In addition Integration, Test and Trials activities are supportive of Demonstration, Manufacture and In Service phases and are covered therefore in a separate section. At the commencement of the D&M Phases, a Combat System design will have been proposed either through the previous Assessment Phase (AP) phase or as part of the tender for a whole warship design and build contract.

4.2 Purpose of the Demonstration and Manufacture Phase

4.2.1 The purpose of the D&M phases is to de-risk and co-ordinate the realisation of the proposed system design. This is achieved through the acquisition of subsystem and equipment components which can be integrated together to provide the required system functionality and performance.

4.2.2 The objective of these phases is to realise a design of the Combat System that will satisfy CONEMP , URD and SRD and to further the arrangements necessary to meet all Lines of Development (LODs) when in-service.

4.2.3 The Shift in Emphasis

4.2.3.1 In terms of the Combat System as a whole, the D&M Phase is typified by a shift of emphasis, from the ‘top-down’ system design perspective, to a ‘bottom-up’ implementation perspective. This is illustrated in Figure 4.1 which depicts the shift in emphasis compared to previous phases.

4.2.3.2 During Concept and Assessment Phases, the main thrust is to explore and define the system in terms of its operational need, ensuring that the system-level requirements are reflected by lower level subsystem and equipment performance. During these phases, lower level subsystem and equipment design characteristics (such as functionality, physical implementation constraints and performance) are predicted or quantified using representative data generated from modelling and prototyping activities.

4.2.3.3 Once the Combat System design baseline is established at the end of Assessment Phase/early in the demonstration and manufacture phase, the acquisition process can commence in earnest. As subsystems and equipments become available for integration, actual subsystem and equipment characteristics will be confirmed through test and evaluation.

4.2.4 System Design and System Implementation Perspectives

4.2.4.1 The ‘top-down’ system design perspective is appropriate when considering change to the system requirement and assessing potential impact on the system design. This perspective, through which the requirements engineering, system analysis and system design elements remain key, applies when considering changes to the system. During these phases, system design activities are expected to be strictly contained. Changes to the requirement arising from operational or implementational issues must be investigated through the formal Systems Engineering (SE) process as for previous phases.

4.2.4.2 The ‘bottom-up’ perspective becomes increasingly prominent during the implementation of the system design. This is manifested through the need to assess, manage and co-ordinate the characteristics of subsystem and equipment components as they are acquired, developed and produced. The ‘implemented performance’ of individual subsystems and equipments must be considered in terms of overall


Unclassified 86

system performance and actions taken to ensure that system-level objectives and requirements can be met. This is the prevalent perspective during this and the following/related integration, test and trials.

CONCEPT ASSESSMENT 1

SYSTEM IMPLEMENTATION

ASSESSMENT 2 DEVELOPMENT andMANUFACTURE

SYSTEMS ENGINEERING PROCESS

The emphasis of the System Design perspective dominates in the early stages of theLife Cycle since the design is being ‘driven’ at the system-level. Information relating topredicted performance of constituent (candidate) subsystems and equipments is usedto predict overall system characteristics.

INTEGRATION TEST andTRIALS

Requirements

Acquire Equipment

Analysis/Design

Acceptance

Integrate/Test

SE Process

DPSS

SDSA

RE

The SE process ‘drives’ system design and implementationthrough Requirements Engineering (RE), Systems Analysis(SA) and System Design (SD) activities. These define thesystem design to progressively lower levels of detail. SystemStudies (SS) and Documentation Production (DP) support theinvestigation of system characteristics and documentation ofthe design.

SYSTEM DESIGN

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

The emphasis of System Designis predominantly ‘top-down’,considering system level issuesand how these can beaddressed by subsystem andequipment implementation.

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

The emphasis of SystemImplementation is ‘bottom-up’,assessing and managing thecharacteristics of equipmentcomponents as they are acquired,developed and produced.

The ‘implemented performance’ ofindividual subsystems andequipments must be considered interms of overall systemperformance and actions taken toensure that system-levelobjectives and requirements canbe met.

Following contract award for D & M, there is achange in emphasis towards the System Implementation perspective. Thischange of emphasis continues through Integration Test and Trials up toCombat System Acceptance.

Figure 4.1 Shift in Emphasis of Demonstration and Manufacture

4.2.4.3 Both perspectives can be expressed through the application of the generic SE process introduced in Section 1. The system implementation perspective, which is the dominant view during this phase, invokes general SE and project management activities, supported through system studies and documentation production.

4.2.5 Demonstration and Manufacture Phase Activities

4.2.5.1 Combat Systems engineering during these phases is mostly concerned with managing the relationship between the ‘top down’ system design perspective, driven by whole system requirements, which is imposed on the ‘bottom-up’ system implementation, typified by the real world characteristics of subsystem and equipment components. This is in order to achieve the optimum compromise through trade-offs between what is required and what is actually possible under the prevailing circumstances.

4.2.5.2 The elements of the core process within its wider SE context are developed as the basis for the demonstration and manufacture phase process as illustrated in Figure 4.2. This shows the life cycle activities of particular importance during this phase in relation to key inputs and outputs.


Unclassified 87

Figure 4.2 Demonstration and Manufacture Phase, Overall Process

4.2.5.3 Combat System demonstration and manufacture activities are summarised as follows:

a) Systems Engineering (Technical) and Project Management - the over-arching management process addressing contract, risk, quality, project and SE aspects throughout these phases;

b) Life Cycle Process Factors - anticipation of the needs of downstream activities, such as preparation for the integration, test and trials activities, liaison with the platform design process and Through Life Management;

c) Design Change Management - effective management of change and impact assessment against Combat System design baselines;

d) Combat System Development - influence across equipment/subsystem and platform programmes to maximise the achievement of Combat System-level objectives and requirements;

e) Subsystem Equipment Development and Manufacture - visibility of the acquisition of subsystem/equipment components including the selection of existing equipment and demonstration and manufacture of new or modified equipments;

f) Acceptance - progressive generation and review of acceptance evidence to predict compliance with the requirements and provide confidence that the implementation is proceeding according to plan;

g) System Studies - modelling, simulation and related analyses supporting evaluation of the evolving design implementation, assessment of engineering trade-offs and a basis for the investigation of change;

h) Integrated Logistic Support - co-ordination of Integrated Logistic Support (ILS)/ Logistic Support Analysis (LSA) activities associated with the demonstration and manufacture of Combat System components;

Combat SystemDEVELOPMENT and MANUFACTURE

LIFE CYCLE

PROCESSFACTORS


DESIGN CHANGEMANAGEMENT

SUBSYSTEM/EQUIPMENT FDP


DOCUMENTATION


CONFIGURATION MANAGEMENT

SYSTEM STUDIES

INTEGRATED LOGISTIC SUPPORT

ACCEPTANCE

Supporting DocumentationEquipment Design Information

Equipment Test, Trials &Acceptance Information

Equipment Operation & SupportInformation

Models & Simulations Cost Models

Effectiveness Models Functional Models

Architectural Models HCI Simulations AR&M Analyses

Key Documents Contract Documentation

Management Plans Requirements Database

System Design Documentation

Acceptance Documentation

Equipment for Integrationin Shore Facilitiesin FOC Vessels

in Follow-on Vesselsin Training Facilities


Unclassified 88

i) Documentation - maintenance of key documents used to co-ordinate the detailed design of the Combat System, and the demonstration of lower-level documents defining the implementation;

j) Specialist Discipline Support - integration of engineering and support specialist disciplines within the SE process;

k) Configuration Management - implementation of the regime through which all demonstration and manufacture products are controlled and co-ordinated.

4.2.6 The Transition from Demonstration and Manufacture to Integration, Test and Trials

4.2.6.1 The early life cycle phases (Concept and Assessment) are discrete phases typified by a distinct transition from one phase to the next. This is not the case for the latter phases where there is seldom a distinct single boundary between what are effectively notional phases.

4.2.6.2 From the system perspective, the transition between demonstration and manufacture, and related integration test and trials is the series of events associated with the delivery of subsystems and equipments for integration. Though this is not necessarily a specific milestone in overall Combat System programme terms, it is a convenient point by which to separately discuss issues associated with acquisition and integration.

4.2.7 Platform Integration

4.2.7.1 The Combat System cannot be considered in isolation. The integration of the Combat System components into the platform is a major facet of the system demonstration process, which has to be considered throughout all design and integration activities. This includes:

a) The realisation of the installed Combat System components in terms of their physical characteristics (e.g. size, shape, weight, service requirements);

b) The interactions between the Combat System and the platform which characterise the warship as a fighting unit (e.g. blind arcs, alignment);

c) The interactions between the Combat System components themselves (e.g. mutual interference, interfaces, physical arrangements).

4.2.7.2 The progressive integration of equipments into a coherent system, where the system comes together as a whole, is treated as a subject in its own right in Integration, Test, Evaluation and Acceptance.

4.3 Inputs

4.3.1 Inputs from Previous Phases

4.3.1.1 Inputs to the demonstration and manufacture phase include all the output products generated from the previous Assessment Phase or equivalent pre-contract work undertaken in support of responding to an ITT including:

a) Systems Engineering Management Plan (SEMP);

b) User Requirements Document (URD);

c) Systems Requirement Document (SRD) possibly in the form of the contractual SRD agreed between the IPTL and the preferred contractor; the SRD should also include associated acceptance / verification criteria that need to be addressed in the ITEAP; the proposed Combat System Design including Subsystem Specifications, Coherence Specifications, Interface Documentation, Design Policy Papers;

d) Integration, Test Evaluation and Acceptance Plan (ITEAP);

e) Through Life Management Plan (TLMP);


Unclassified 89

f) The contract;

g) Training Needs Analysis;

h) Combat System Safety Case;

i) Models supporting the Combat System definition from a number of complementary perspectives including: functionality, Human Factors (HF), architectural performance, installation, Availability, Reliability and Maintainability (AR&M) and operational effectiveness.

4.3.1.2 The Combat System Manager (CSM) will inherit the documentation from the previous phases (where applied) and this can be used as the basis of the project file for the Demonstration and Manufacture Phases. Arrangements need to be made to share and manage documentation development between MOD, contractors and equipment sub-contractors (and if appropriate partners).

4.3.2 Inputs Associated with Contract Placement

4.3.2.1 The contractual basis for D&M is centred on the result of negotiations between the procurement authority and the Combat System Integrator (CSI) organisation (considered later under the heading ‘Organisation’ (4.5 Organisation)) (Note: This may be part of the prime contractor’s organisation.). Advice should be sought from the relevant contract authority.

4.3.2.2 The actual coverage, form and content of the requirements and acceptance criteria may vary according to the contracting regime and the level of agreement between the procurement authority, the Combat System Integrator and the supplier organisations (described under ‘organisation’).

4.3.2.3 Documentation developed as part of the Assessment Phase or earlier will have to be reviewed in order to align the proposed design with the negotiated requirements and acceptance baseline by both the MoD and the successful contractor. Care must be taken to both ensure commonality between MoD and industry baseline data and identify any variation in requirements, which may impact on system performance. Some consolidation will be necessary to ensure that the negotiated contract baseline for demonstration and manufacture is related back to the requirements definition in terms of any technical changes agreed through the clarification and negotiation process prior to contract award.

4.4 Outputs

4.4.1 There is a range of output products arising from the D&M Phase. The deliverables from the equipment demonstration/acquisition process can be grouped into:

a) Equipment sets for integration and installation on vessels and in shore establishments (and trainers) together with supporting test equipment where necessary;

b) Design, demonstration and manufacture information, including demonstration and manufacture plans, specifications and drawings;

c) Equipment test, trials and acceptance information including test and integration documentation (including ship fitting/services data), trials plans, specifications and schedules, and equipment Acceptance criteria and Results;

d) Equipment operation and support information including Master Record Index (MRI) data, Configuration Management (CM) documentation, technical publications, upkeep documentation, Logistic Support Analysis Record (LSAR) and operator/maintainer training documentation.

4.4.2 Additionally, there are many output products, which are maintained from previous phases and extended as a result of system demonstration. These can be grouped under the following headings:

a) Management plans;

b) Requirements and acceptance data;


Unclassified 90

c) System functional and performance models used for tracking the Combat System design and to be used in support of aspects of system assessment, optimisation and acceptance. These are provided together with analysis reports detailing the results and recommendations of investigations such as change impact, actual performance shortfalls etc.;

d) Integration test plans, schedules and specifications in preparation for the subsequent integration activities;

e) System test, trials and acceptance information including trials plans, specifications and schedules, and system Acceptance Criteria and results;

f) System operation and support information including MRI data, CM documentation, technical publications, upkeep documentation and LSAR.

4.4.3 All items will be produced, maintained and controlled as an inherent element of the system baseline design description under the appropriate CM procedures. This may be controlled and coordinated through a central platform-oriented information infrastructure or at the Combat System level depending on the contract regime.

4.5 Organisation

4.5.1 Organisational Roles

4.5.1.1 The division of responsibilities between MoD and industry organisations is necessarily dependent on the outcome of negotiations for the D&M contract and is commensurate with the level of risk accepted by industry. There is therefore no absolute model, which defines the boundary of responsibilities and illustrates all variations.

4.5.1.2 D&M involves a number of organisations each of which have distinct roles to play in the demonstration of the Combat System as an enterprise, as depicted in Figure 4.3 also shows various enterprise options for allocating responsibilities between MoD and Industry.

4.5.1.3 Though the actual boundary of responsibilities is negotiable within the prevailing procurement regime, the roles are generally well understood. They are as follows:

a) End User, or customer of the system, in this case the Royal Navy with a nominated “Customer 2”, represented through the DCDS (EC), with a nominated “Customer 1 and his requirements manager”, CSA, FOSM, FOSF and Dstl organisations. During the D&M Phase, end-user involvement is managed through the procurement authority;

b) Procurement Authority, in all cases the MOD, and including Combat System (CSM’s core team), whole warship (platform project manager), and equipment (Equipment Project Manager (EPM)) representation;

c) Warship Integrator, responsible for overall management of the whole warship programme in a prime contract setting;

d) Combat System Integrator, responsible for the design, demonstration, manufacture and integration of the Combat System, defining its constituent equipments and overseeing their acquisition, installation, integration and acceptance;

e) Platform Design Authority, responsible for the design, demonstration, manufacture, integration and acceptance of the platform structure and systems. The platform design authority is responsible for installation and integration of Combat System equipments on board the platform;

f) Equipment Supplier, responsible for developing, modifying, producing and supplying equipment and associated design, test and support documentation.


Unclassified 91

Figure 4.3 Demonstration and Manufacture Organisational Relationships

4.5.2 Contractual Arrangements

4.5.2.1 The roles above are defined to set organisational issues in perspective. Inevitably there is much room for negotiation of roles and responsibilities given a particular contractual context and there are a variety of possible arrangements under which the demonstration and manufacture phase can be progressed:

a) Under the auspices of a Prime Contract for a whole warship procurement (e.g. T45, LPH, LPD(R), Astute). In this case, MoD involvement is at the highest level of project management, dealing with an industry prime contractor through a formal contractual relationship. MoD control of SE activities is through ‘hands off’ management of the whole warship/CSI organisations. The main MoD responsibilities under prime contractor arrangements are monitoring and reviewing progress, identifying and tracking risk, promulgating change, and agreeing acceptance mechanism;

b) As a Combat System Integration Authority with Industry assuming the role of CSI (e.g. as originally proposed for the S&T Update Final Phase WSIA). This arrangement is applicable to Mid Life Updates (MLUs) where the Combat System enhancements are significant and platform modifications required are minimal;

c) As a Design Agency, in support of the MOD(DPA) (e.g. S&T Update Initial Phase WSA, Vanguard CSDA). This permits MoD to control the procurement of principal equipments, which contribute to the Combat System capability and devolve the day-to-day SE activities to industry;

d) Other Equipment-Oriented Combat System projects still in existence (e.g. T23, T42, CVSG) have inherited organisational set-ups based on MoD equipment procurement approaches. For these projects, MoD have generally taken full responsibility for procurement and integration of individual equipments;

END-USER ORGANISATION

Development &ProductionContract

EquipmentContracts

Equipment forIntegration

PROCUREMENTAUTHORITY

EQUIPMENTSUPPLIERS

COMBAT SYSTEMINTEGRATOR

CombatSystemPropulsion

Marine Engineering Platform

Whole Warship PROCUREMENT AUTHORITY

MARINE PROPULSION

PLATFORM DESIGN AUTHORITY

WHOLE WARSHIPINTEGRATOR

Equipment Oriented

Design Agency

CS Integ’n

Authority

PrimeContract Partner/Alliance

COMBAT SYSTEM ACQ POLICY

MOD

Industry


Unclassified 92

e) MoD and Industry Partnerships, sometimes branded as an ‘Alliance’, where MoD and Industry work together, provide an alternative approach. Involvement of MoD and industry in trade-offs throughout the life cycle to optimise the application of SE practice although it is not clear how partnership will be effected across contractual boundaries.

4.5.2.2 The actual contractual arrangement, which comes into place, is largely dependent on the procurement strategy invoked and the boundary of responsibilities negotiated between the MoD and industry. In each case, the primary responsibilities are to put in place a controlled, coherent and co coordinated process for managing the range of SE tasks to be undertaken.

4.5.2.3 The main focus of this section from an organisational point of view is the procurement organization where design authority has been vested to industry though an appropriate contract. Note: Organisational considerations are subject to reporting structures, which will vary and are currently under review.

4.5.3 Whole Warship

4.5.3.1 The Warship Project Manager (WPM) is responsible for the warship project as a whole, coordinating marine, platform and weapon system engineering activities and any foreign participation which may be required. The CSM needs to report to the WPM on a regular basis and maintain a close liaison with members of the WPM’s team in order to effectively manage all issues arising from the Combat System and platform interaction.

4.5.4 Combat System

4.5.4.1 Though the primary role of MoD during this phase is as an informed customer applying ‘hands off’ project management and overall direction to the Combat System implementation project, the wide variety of potential high-risk issues demand an appropriate level of SE capability and well directed scope in terms of a CONEMP and URD.

4.5.4.2 The MoD Combat System core team must be capable of supporting the management functions to a level appropriate to the size and complexity of the Combat System project, typified by the following roles:

a) CSM, responsible for the project as a whole;

b) Technical Co-ordinator, responsible for controlling and directing the multi-disciplinary team and monitoring and reporting of the technical quality of the work;

c) Administration Co-ordinator and Planner, responsible for advising the CSM of progress against budget and schedule and associated planning activities;

d) Quality Assurance (QA) Manager, responsible for ensuring that appropriate quality control processes are in place both in the MoD and within the contract organisation;

e) Specialist Subcontractor/Customer ‘friend’ providing technical and administrative support throughout the phase where needed;

f) Contracts and Finance support;

g) Customer 1, 2 and Requirements Manager.

4.5.5 Equipment Supply

4.5.5.1 The equipment supplier will enter into a contractual relationship with the Combat System integrator in order to provide their equipment and associated deliverables according to an agreed programme. The involvement of EPMs, responsible for particular equipment selected for inclusion in the Combat System is an important consideration.

4.5.5.2 Whilst it is important that the CSI can exert influence over the equipment supplier to ensure visibility of the equipment design and provide evidence of progress, there may be limitations. The precise arrangements on the Combat System/equipment supplier contract will depend on:


Unclassified 93

a) Whether the equipment is legacy/Non Demonstration Items (NDI), Commercial Off The Shelf (COTS), new or modified and the extent of demonstration required to meet the equipment/subsystem requirement;

b) The level of commercial agreement achievable between the parties;

c) The underlying requirement which led to the equipment being developed in the first instance. This may be market-led, MOD-driven, or determined from non-UK influences such as EU initiatives or US DoD policy;

d) The significance of the equipment in the Combat System design (i.e. the number and complexity of its interfaces with other equipments and its contribution to overall Combat System functionality, performance and effectiveness);

e) The Combat System programme;

f) Whether or not the item is Government Furnished Equipment (GFE).

4.6 Demonstration and Manufacture Phase Processes

4.6.1 Introduction

4.6.1.1 The division of responsibilities between the CSM and contractor organisations is dependent on the outcome of negotiations for the demonstration and manufacture contract and is commensurate with the level of risk accepted by the contractor. The boundary of responsibility identifying which activities are assigned to the CSM organisation and which are assigned to the contractor organisation must be clearly established in relation to the positions agreed by both parties. This should be documented in the suite of management plans as discussed under the section on SE (Technical) and project management. Clearly there is much scope for variation between contracts and the implementation of the generic process must be tailored to reflect prevailing project circumstances.

4.6.2 Demonstration and Manufacture Phase Activities

4.6.2.1 The primary activities which are to be undertaken during the Demonstration and Manufacture phase are depicted in Figure 4.4 and described in detail in the clauses which follow:


Unclassified 94

Figure 4.4 Demonstration and Manufacture Phase Activities

4.6.2.2 The process model for this phase sets out those activities, which must be conducted. The process model reflects the primary activities introduced earlier and reflects the system/subsystem/equipment hierarchy through which Combat System demonstration is achieved. Manufacture in the system context is largely contained within the subsystem/equipment demonstration and manufacture processes and these topics are addressed in sufficient detail to appreciate system-level aspects.

4.6.3 Systems Engineering (SE) (Technical) and Project Management

4.6.3.1 Objectives

4.6.3.1.1 SE (Technical) and Project Management covers:

a) The goals and activities to be conducted during the D&M Phase including acceptance;

b) The conduct of all technical activities, including their integration to meet the overall technical objectives;

c) The processes through which the phase objectives are met in programme and cost terms;

d) The provision of all deliverables including equipment and supporting documentation required by the contract in preparation for installation, integration, acceptance and in-service support.

LIFE CYCLE PROCESS FACTORS Review of produceability of Subsystem/Equipment designs Definition and acquisition of Shore Integration Facilities Preparation for the progressive integration of subsystems and equipments Platform Integration and Installation Planning and preparation for acceptance Support Infrastructure Consideration of Disposal

DOCUMENTATION



SYSTEM STUDIES


Management Plans System Baseline Documentation Detailed Design Documentation Test, Trials & Acceptance Documentation Operation & Support Documentation Handbooks

Alignment AR&M Communications Cost EMC Human Factors ILS InterfacesOperational Effectiveness Platform Integration Safety Security Shock Software Training

Configuration Definition Configuration Control Configuration Accounting

Maintenance of Prototypes and Models System Change Investigation and Impact StudiesMonitoring and Assessment of Subsystem and Equipment Characteristics (Design Evaluation)

Logistics Support Analysis (LSA) Life Cycle Costing (LCC) Integrated Supply Support Technical DocumentationMaintenance Planning Support & Test Equipment Manpower & HF Training & Training Equipment

SYSTEMS ENGINEERING (TECHNICAL) and PROJECT MANAGEMENT

Contract Management Risk Management Quality Management Project ManagementSystems Engineering Management Acceptance Reviews & Audits

Requirements Engineering

ACCEPTANCE

DESIGN CHANGE MANAGEMENT

Subsystems/ Equipment for

integration

SUBSYSTEM/EQUIPMENTDEVELOPMENT and PRODUCTION

Acquisition of NDIs/COTSAcquisition of New/Modified Equipment

ACCEPTANCE &CERTIFICATION

COMBAT SYSTEMTRIALS

SYSTEM INTEGRATION& TEST

INTEGRATION TEST and TRIALS [Chapter 9]

Planning for Acceptance Evidence Gathering Formal Certification

SystemsAnalysis

SystemDesign

COMBAT SYSTEM DEVELOPMENTPreparation for Integration

Data Management


Unclassified 95

4.6.3.1.2 During earlier phases, the scope of project management and SE activities is contained since they are necessarily oriented towards developing the design as a concept. Though previous phases consider and investigate all manners of management and engineering issues, it is not until the demonstration and manufacture phase that the acquisition process begins in earnest. At this point the scope of the project expands significantly to include demonstration and/or acquisition across a wide and varied set of interacting subsystems and equipments.

4.6.3.1.3 It is important to acknowledge that once D&M begins, all parties across the Combat System enterprise - from MoD at the very top representing the ultimate customer, the RN, through the Combat System integration organisation, to the subsystem and equipment suppliers and in turn to their suppliers - have entered a set of ‘partnering’ relationships.

4.6.3.2 MoD CSM Role

4.6.3.2.1 Though the contractual boundaries delineate the actual responsibilities of each party which will vary widely with the nature of the contractual agreement, accountability for the success or failure of the Combat System to meet project level goals of fitness for purpose, programme and cost ultimately lie with the MOD. It is essential to put in place appropriate management controls which provide clear lines of communication across all levels of the enterprise and which foster a spirit of commitment and co operation.

4.6.3.2.2 The MoD CSM role throughout this phase is one of ‘hands-off/eyes-on’ management, i.e. to ensure that the Combat System enterprise is progressing to cost and programme and that any changes are thoroughly investigated and if agreed, implemented in the most cost-effective manner.

4.6.3.3 Activities during the Demonstration and Manufacture Phase

4.6.3.3.1 SE (Technical) and project management activities required to co-ordinate demonstration and manufacture must be anticipated during previous phases and management plans provide the means to document proposed policy and practice.

4.6.3.3.2 These activities need to be considered in the context of other related project-level management disciplines (contract management, risk management and quality management) as well as whole warship procurement and subsystem/equipment procurement issues across the hierarchy. This is illustrated in Figure 4.5 which emphasises the inter-relationships between related activities at different levels of the hierarchy.

4.6.3.3.3 Many of the activities are common to previous phases covered in this guide.


Unclassified 96

Quality

Systems Engineering Management

Project Management

Subcontract Management

Risk

Contract

Quality


Project Management

Subcontract Management

Risk

Contract

Quality


Project Management

Risk

Contract

Subsystem/Equipment Contractual relationships are detailed through Contract and Subcontract Management Plans. The Contract Management Plan calls up lower level Management Plans to link overall Combat System objectives with plans for Subsystem/Equipment implementation.

Whole Warship/Combat System Management Plans provide the link between overall MOD objectives and Industry plans for implementation. Systems Engineering Management requires specific plans for high risk areas such as Design & Development, Production and Build, Integration, Trials, Acceptance, ILS, AR&M, HFI etc.

MOD Project MOD need Management Plans to define the overall objectives and controls of Combat System Development in context with overall warship procurement and individual equipment projects

Figure 4.5 Hierarchy of Project Management Activities

4.6.3.3.4 The policy of Smart Acquisition means that MoD and Industry often collaborate in the decision-making process during D&M. This means that the MoD must closely monitor the progress of a project as an informed customer.

4.6.3.3.5 The Contract Management Plan (CMP) is the highest level in the management plan hierarchy, and cascades responsibility down the Combat System enterprise through subordinate industry-authored (and MoD approved) CMPs. The CMP sets the context for the balance of responsibilities within MoD and between MoD and industry, and thus bounds the Combat System contract.

4.6.3.3.6 Priority Project Management activities to be conducted during the demonstration and manufacture phase are summarised in Table 4.1.

4.6.3.4 Systems Engineering Management

4.6.3.4.1 Within the scope of Combat System and lower-level SE management activities are the manufacture of detailed plans addressing particular facets of the SE process.


Unclassified 97

Table 4.1 Project Management Activities

PM Discipline Topic(s) Priority Activities During This Phase

Contract Management

Contract Management Plan (CMP) The MoD CMP, which formalises the Combat System Procurement Strategy, will be formally set in place for the duration of the Demonstration and Manufacture phase.

Any contract changes need to be reflected by the MoD CMP and thereafter through the subordinate CMPs where affected.

Risk Management Risk Management Plan (RMP) Risk Register Risk Analysis Risk Reduction Action Plan (RRAP)

Continued application of Risk Analysis methods, update of the Risk Register and raising of RRAPs, as a result of risk promulgation up through the Combat System implementation hierarchy.

Regular liaison with the CSI and CSESs by ongoing ‘visibility’ and formal risk review at programmed Reviews and Audits.

Quality Management

Quality Plan Software Quality Plan Systems Engineering Management Plan (SEMP) [see later]

Periodic auditing of management processes through the Combat System Enterprise to ensure compliance with recognised Quality Standards e.g. BS5750 ISO 9000 series certified.

Auditing of the Systems (and Software) Engineering processes, against a recognised (e.g. Carnegie-Mellon) SE Capability Maturity Model.

Project Management

Project Management Plan (PMP) Review of the PMP to reflect any changes in organisational/reporting arrangements.

Programme Control and Performance

Reviews and Audits

Regular scheduled review of the programme(s) and progress. Liaison with EPMs to review Weapon Equipment Delivery Programmes (WEDPs) and relationship with Combat System programme;

Liaison with CSI; Agreement and visibility of the Project Review Cycle; Action tracking.

Review of methods/tools support and training.

Configuration Management and Change Control Plan

Auditing of the Configuration Management process for controlling the effects of change to the system design and implementation baseline(s).

4.6.3.4.2 The exact coverage of detailed plans is subject to variation amongst the assignment of SE activities within organisational structures. The MoD needs to review and agree all subordinate plans generated by the Combat System enterprise as detailed in Table 4.2.


Unclassified 98

Table 4.2 Systems Engineering Management Activities

SE Discipline Topic(s) Priority Activities During This Phase

Systems Engineering Management (General)

The Systems Engineering Management Plan (SEMP) provides a top level plan for integrating ‘best practice’ SE activities across the project.

Review and agreement of inter-related plans addressing SE issues across the Combat System hierarchy to ensure compatibility with overall MoD SEMP objectives.

Work Breakdown Structure (WBS) Demonstration, refinement and agreement of the Combat System WBS; Maintenance and review of the WBS spend profile and input of information into project LCC analysis; Monitoring of project deliverables; Review of MoD resource profiles.

SE Process Performance Monitoring of key Technical Performance Measures (TPMs) at programmed control points identified by the CSI, to assess actuals versus predicted performance and plans for containment; Monitoring of design baseline control (see CM); Visibility and review of CSI design data.

System Security Combat System Security Policy Periodic review and audit of Combat System project information and handling; Review of integration of security into the Combat System implementation; Management of security risk through physical transmission, espionage and intelligence gathering.

System Safety Safety Management Plan (SMP) Facilitate the achievement of safety objectives stated in the SMP through the demonstration and refinement of the Safety Case through the phase.

Design & Demonstration Management

Software Management Plan Interface Management Plan EMC/Signatures Management Plan

Review to ensure compatibility of lower level plans with MoD SEMP Objectives.

Manufacture & Build Management

Integration Plans GFE/GFI Management Plan Jigs & Tools Management Plan

Integration, Test Evaluation and Acceptance Plan

Modelling, Test & Trials Management Contract Acceptance Schedule

ILS AR&M Management Plan Human Factors Integration Plan Technical Publications Plan

Through Life Management Plan

The MOD’s overarching Planning Document supported by key documents such as the CONEMP, URD, SRD and ITEAP.

Systems Engineering Management Activities (continuation)


Unclassified 99

SE Discipline Topic(s) Priority Activities During This Phase

Disposal Disposal Plan

Safety Analysis Demonstrate ability to support safety case.

Integration strategy and Plan

A subset of the TLMP addressing how integration and interoperability will be achieved.

Definition of system boundaries, interfaces, data structures, security etc, how they support the User IO Requirements and how they are tested.

4.6.3.5 Technical Performance Measures

4.6.3.5.1 Technical Performance Measures (TPMs) should be maintained for all critical performance requirements at both the system and subsystem/equipment levels. TPMs for all configuration items should be monitored throughout the demonstration and manufacture phase for movement outside bounds of acceptable performance. These should be accompanied by a definition of the purpose/scope, reporting period and details of the collection method. Considerations include:

a) Requirements;

1) Number of requirements decomposed & analysis suggested,

2) Number of requirements responded to,

3) Number of requirements agreed;

b) Design;

1) Load budget estimate,

2) Load budget given by supplier,

3) Total Combat System load budget prediction,

4) Interfaces identified;

c) Procurement;

1) Request For Quote (RFQ)/Invitation To Tenders (ITTs) prepared, reviewed, issued,

2) Tender responses received,

3) Assessment criteria produced,

4) Tender responses clarified;

d) Modelling;

1) Scenarios identified/reviewed/endorsed,

2) Measures Of Effectiveness (MOEs) identified/reviewed/endorsed,

3) Measures Of Performance (MOPs) identified/reviewed/endorsed,

4) Effectiveness/capability predictions,

5) Performance prediction,


Unclassified 100

6) System thread analysis;

e) Safety;

1) Hazards identified,

2) Hazards with As Low As Reasonably Practicable (ALARP) assessment,

3) Safety case(s) preparation,

4) Test & integration,

5) Tests specified/issued;

f) In Service Support;

1) Faults raised/accepted/attributed,

2) Solutions proposed/implemented/accepted,

3) General (e.g. risk),

4) Problems addressed/resolved,

5) Risks identified/mitigated/resolved.

4.6.3.5.2 TPMs should be formally reviewed through the project review cycle, which identifies the major control points of the system/subsystem/equipment programmes. The assessment of TPMs at each control point should be conducted in addition to establishing that other technical, budget and programme requirements are being met.

4.6.3.6 Related Information

4.6.3.6.1 Further information on project management aspects of a Combat System project is available in the CDPIs and DPMGs.

4.6.4 Life Cycle Process Factors

4.6.4.1 Objectives

4.6.4.1.1 Life Cycle Process Factors are those aspects of the Combat System life cycle which need to be anticipated well in advance to achieve a design optimised against a range of design issues including through-life costs. Effectively, this means looking ahead to consider what can be done at this phase to ease what needs to be done during subsequent phases.

4.6.4.1.2 At this phase in the life cycle, the majority of through-life aspects will have been previously considered in some depth. The main objective at this phase is to maintain focus and ensure that optimisation of the design is pursued through succeeding levels of the Combat System/subsystem/ equipment hierarchy.

4.6.4.1.3 In particular, preparation is needed for integration test and trials, anticipation of acceptance, support whilst in-service and due consideration of disposal. Each of these areas needs to be addressed in relation to the demonstration intent so that the evolving design can be optimised.


4.6.4.2.1 The role of the MoD CSM includes ensuring that through-life aspects are planned for and considered in sufficient depth throughout the D&M Phase. This needs to be addressed both in the implementation of the specified requirements and specifications, and for any changes which are assessed and applied to the system design baseline.


Unclassified 101

4.6.4.2.2 To this end, the CSM must satisfy himself that he retains sufficient visibility of through-life issues across subsystem/equipment D&M and platform design and build processes. Again, the vehicle for maximising visibility of demonstration is through the project review cycle through which risk issues are formally raised and action identified. Where the procurement boundary of the project system requires integration with other systems, interface mechanisms must be agreed.


4.6.4.3.1 These activities during the D&M Phase are anticipated and supported by the extensive range of through-life modelling activities (i.e. models, simulations and system representations used for change impact/assessment, training etc.) described under systems studies (see later).

4.6.4.4 Integration Strategy

4.6.4.4.1 The contractor will have to provide the tools for Combat System integration in a timely way; be it modifications to an existing Shore Integration Facility (SIF)/Shore Demonstration Facility (SDF), a new SIF/SDF, or a new approach, such as synthetic environments. The tools must be installed, tested, and accepted prior to the commencement of the Combat System integration programme.

4.6.4.4.2 During this phase the integration strategy would be updated in accordance with the equipment delivery dates that emerge from the chosen equipment suppliers. The demonstration of the intended group and overall Combat System tests and trials into detailed tests/trials orders would commence and the supporting scenarios would be developed for input to the scenario generator. The First Of Class (FOC's) installation, test and trials programme would also be developed at this time.


Unclassified 102

Table 4.3 Summary of Life Cycle Process Factors

Life Cycle Aspect Priority Activities at this Phase

Demonstration and Production

Review produceability of subsystem and equipment designs.

Integration Anticipate integration through the refinement and implementation of the integration strategy including:

• Strategy for testing, with due emphasis on maximum system-level testing through FATs;

• Definition of the sequence of integration and programme liaison with Equipment sponsors and manufacturers;

• Planning for extent/re-use of SDF/SIF facilities;

• Definition of stimulation/simulation and scenario generator functionality required in support of the Integration Strategy;

• Appraisal of commonality of system architectures across the fleet;

Definition and development of Test Equipment and rigs to support system-aspect testing;

Definition and development of test and trials recording and analysis equipment;

Instigation of data management practice for all data exchange across all inter-connecting equipments, whether through direct links or via a data highway;

Advance planning for the Shore-based Integration environment (e.g. housing, services and supplies etc);

Integration with the Platform.

Acceptance Refinement of the Acceptance strategy and preparation for acceptance, ensuring that verification and validation activities throughout Subsystem/Equipment Demonstration and Manufacture are adequate and in place. Including:

• Scheduling of progressive acceptance activities in Equipment FAT;

• Acceptance in the shore environment;

• Acceptance on the FOC vessel;

• Acceptance on follow-on vessels;

• Contractual acceptance across the Combat System enterprises;

• Support to Platform Acceptance.

In-Service Support Support infrastructure; Anticipation of obsolescence of equipment components.

Disposal Ensure that identification and review of disposal issues is adopted at all appropriate levels of the system hierarchy.


Unclassified 103

4.6.5 Design Change Management

4.6.5.1 Objectives

4.6.5.1 The Combat System engineering process must recognise that change is inevitable. Change may be driven by external changes to the requirement as a consequence of real world events, but more likely through the nature of the demonstration process. The key to success is to manage change such that any impact is first assessed before being implemented. By judicious use of CM techniques and co-ordination of the implementation process through formal reviews, it should be possible to contain change to a manageable level.


4.6.5.2.1 During these phases, the MoD CSM role is one of identifying, promulgating and assessing the impact of changes to the Combat System to meet a new or modified operational capability. This may result from a change in threat or the definition of a new role, which demands further investigation.

4.6.5.2.2 Alternatively, change may be necessary to meet performance shortfalls of specific equipment(s). In such cases, the impact of shortfalls will require further investigation in terms of what this means to the capability of the warship and the consequent impact on system operational context.

SYSTEMDESIGN

SYSTEMSANALYSIS


PROCESSINPUTS

PROCESSOUTPUTS

SYSTEM STUDIES DOCUMENT PRODUCTION

Document& Report

EvaluateChange

ImplementChange

ProposeChange

AnalyseChange

InformationBase

Figure 4.6 Design Change Management Activities


4.6.5.3.1 The generic SE process illustrated in Figure 4.6 provides a suitable template for co-ordinating design change management activities. These can be considered as:

a) Propose change and analyse change, i.e. the definition of the change within the context of previously conducted requirements engineering and system analysis activities;

b) Evaluate change, to perform impact assessment across pertinent performance aspects such as functionality, architectural impact, operability, AR&M, effectiveness and costs (including consequential costs);

c) Implement change, to incorporate the change in the system design baseline for cascading through any affected subsystem and equipment demonstration and manufacture contracts;

d) Document and report the change impact assessment, proposed modifications etc.

4.6.5.3.2 All changes will demand a controlled regime (i.e. a project review cycle) through which to revisit the system baseline design and ensure that any impact is identified and understood.


Unclassified 104


The first step in assessing the change is in expressing it as new or modified requirements and adding this information to the URD/SRD. The URD/SRD (and its electronic form held in a RDB) must continue as the repository of all configured requirement statements and the vehicle by which progress of the acceptance process will be documented and tracked. The requirements set will be reviewed and changes agreed for incorporation (deletion, modification or addition). Changes to the requirements baseline will need to be controlled through the CM regime as applied to the URD/SRD. (Typically using DOORS and Architectural Views.)

4.6.5.5 Systems Analysis

4.6.5.5.1 The results of previous systems analysis activities will need to be revisited to ensure that the implications of any changes are fully understood, quantified and carried through to the design. Each will be supported by pertinent system studies, described later in this Section.

4.6.5.5.2 Any changes to functional requirements will need to be highlighted and traceability information followed through. Changes will be reviewed and formally agreed through review to confirm that there is a suitable basis for proceeding with system design activities.

4.6.5.6 System Design

4.6.5.6.1 In a similar way, the system design should be revisited to verify compliance with the requirements. Shortfalls need to be investigated and contained wherever possible or changes to the system design proposed for their resolution. This may include revisiting the following:

a) System Design, as defined by a physical model and its variants, to confirm the partitioning and investigate the impact of any changes to the implementation and its interfaces;

b) Design Documentation, in particular ensuring that any changes are reflected in the requirements of the coherence specification and the appropriate subsystem specifications;

c) Design Verification, to ensure that traceability between requirements and analysis and design entities is maintained.

4.6.5.6.2 System studies should continue to be used to ensure that the design is compliant with the requirements and represents a viable solution. In particular, architectural performance modelling and Human Computer Interface (HCI) operability modelling should be conducted to identify and quantify the likely performance of the integrated system. Design review is the formal route through which any major changes to the system design are declared. This is the precursor to formally invoking changes to affected subsystem/equipment contracts.

4.6.6 Combat System Demonstration

4.6.6.1 Objectives

4.6.6.1.1 Combat System demonstration can be considered as a major risk reduction activity - ‘bridging the gap’ between the demonstration and/or acquisition of subsystems/equipments into which it has been partitioned followed by the progressive integration of equipments into a complete whole to provide the required Combat System functionality.

4.6.6.1.2 The main objectives of this activity are to oversee the acquisition process, in particular the acceptance of system components, and prepare for integration both within a shore environment and on the FOC vessel. The primary considerations here are:

a) Interface and data management (that is the co ordination of data flow across equipment interfaces, and enforcement of system coherence rules/standards);

b) Preparation for integration in the shore environment (SIF/SDF), including provision of the facility;

c) Liaison with the platform design and build process;


Unclassified 105

d) Co-ordination of the evolving equipment design and management of Combat System shortfalls.


4.6.6.2.1 The MoD CSM role, as a minimum, is one of ‘informed customer’, monitoring progress of the project against the programme, facilitating communications and the resolution of issues between Combat System, equipment and platform programmes wherever there is potential conflict. Additionally, liaison will be necessary to prepare the way for extension of use of existing shore facilities.

4.6.6.2.2 The CSM should satisfy himself that policy and strategy for integration of the Combat System is being implemented in an effective manner, and those integration risks are identified early and appropriately managed. This also means ensuring that the testing and acceptance of equipments meets the needs of the Combat System and that duplication of testing is minimised.

4.6.6.3 Preparation for Integration

4.6.6.3.1 The ITEAP, prepared and refined during earlier phases will have defined the overall approach to integration, test and Acceptance, the role of modelling, data management, and the implementation of the strategy in terms of equipment test strategies, shore-based integration, and integration on the FOC and follow-on vessels. During the demonstration and manufacture phase, preparation for implementation of the strategy will become a priority task. This includes:

a) Rigorous application of data management practice for all data exchange across all inter connecting equipments, whether through direct links or via a data highway (including ‘implied’ interfaces);

b) Further definition of one-off, repeat tests of functionality, performance, and HCI aspects through stand-alone tests, Combat System test and residual tests;

c) Definition of the sequence of integration and programme liaison with equipment sponsors and manufacturers;

d) Promulgation of the ITEAP strategy, with due emphasis on maximising system level interface compatibility testing through equipment Factory Acceptance Trials (FATs);

e) Demonstration of Combat System integration test documentation;

f) Equipment design proving with particular emphasis on interface design and implementation of data exchange and protocols across interfaces;

g) Planning for extent/re-use of SDF/SIF facilities;

h) Definition and acquisition/development of stimulation/simulation and scenario generation facilities required in support of shore-based integration, test and trials as determined by the integration strategy;

i) Definition and development of test equipment and rigs to support system-aspect testing;

j) Definition and development of test and trials recording and analysis equipment;

k) Advance planning for the shore-based integration environment (e.g. housing, services and supplies etc);

l) Generation of evidence in support of higher level e.g. Platform Acceptance.


Unclassified 106


4.6.6.4.1 Data Management, that is the co-ordination of data exchange across separately procured Combat System equipments, is a fundamental control mechanism of the integration policy. The implementation of data management procedures serves to reduce the risk of interface problems during the integration process and optimise data exchange across the Combat System design solution. This involves:

a) Definition of responsibilities for data management between platform and Combat System projects, and support from other organisations (e.g. MCTA);

b) Interface standardisation covering documentation, network interfacing, data formats, and messages (e.g. Def Stans 21-13, 21-24, 21-26 and 21-28);

c) Interface definition through formal issue cycle of Interface Specification (IFS) and Data Exchange Specification (DES);

d) Documentation of interface protocols and definition of representative scenarios;

e) Definition of data highway/equipment linking tests and trials and reporting;

f) Data exchange proving, through tests and follow-up analyses;

g) CM of all interface documentation and test data.

4.6.6.4.2 The chosen contractor will implement the contracted ‘design for integration’ approach offered in the Assessment Phase, enforcing the standards for the definitions of application data elements, co-ordinate systems, network interfaces, network protocols, and application protocols included in their coherence specifications. They may have to develop the coherence specification as demonstration of the Combat System equipments are likely to raise issues which are best dealt with in that document.

4.6.6.4.3 The contractor will also apply their information and interface management method which should be visible to MOD, policing the demonstration of detailed DES and IFS in the context of their mandated standards, and then comprehensively checking compliance in ‘design proving’ FATs.

4.6.6.4.4 Further information concerning Integration and Data Management Policy for Surface Ships can be found in Def Stan 21- 88.

4.6.6.5 Platform Integration

4.6.6.5.1 The integration of the Combat System into the platform is a major facet of the system demonstration process, which has to be considered throughout all D&M activities. This includes the realisation of the installed Combat System in terms of its physical characteristics (size, shape, weight etc.) and the interactions between the Combat System and the platform, which characterise the platform as a fighting unit.

4.6.6.5.2 The range of platform engineering disciplines involved with the integration of Combat Systems equipments include:

a) Whole platform and upper deck engineering including spatial integration addressing access for operation and maintenance, shock excursion envelopes, removal and maintenance envelopes, HF requirements, and advising on construction;

b) Naval architecture, to monitor and control weight and stowage against assigned budgets and advise on ballast arrangements and requirements;

c) Structural engineering, to produce and verify structural design guidance information (scantlings) and advise on structural analyses including shock, noise and vibration;

d) Marine/mechanical engineering, to develop ship system schematics, mounting arrangements, and manage budgets for equipment services and constraints such as heat load, ventilation, cooling etc;


Unclassified 107

e) Electrical engineering, to develop and test electrical supply and distribution systems and switchboards;

f) Security, Safety, Frequency Allocation and Mutual Interference studies and design constraints.

4.6.6.5.3 The relationship between Combat System and platform engineering issues becomes more complex as each proceeds from design through demonstration and manufacture. Due consideration is required of the life cycle process factors, in particular, installation, tests and trials, acceptance, and support.

4.6.6.5.4 As a consequence, it is essential that the platform and Combat System design, demonstration and manufacture activities proceed in a co-ordinated manner. Naturally this requires extensive liaison to ensure that all potential risks associated with the integration of the Combat System into the platform are identified and effectively managed.

4.6.6.6 Platform Design and Build

4.6.6.6.1 Current initiatives within the platform integration specialism favour an integrated approach using a concurrent engineering environment. This may include Computer Aided Design (CAD) and Electronic Data Management (EDM) facilities wherein all engineering drawing data and associated design documentation are held in electronic format within a configured central repository.

4.6.6.6.2 Tool support to the engineering database provides the opportunity to review 3D virtual representations of compartments, equipment installation arrangements, allocated services, spares holding and equipment removal routes to gain early visibility and agreement of physical platform integration issues and solutions. This can be used in support of the acceptance process, once the relationship between model and real world physical implementation is verified.

4.6.6.6.3 The provision of ship-fitting data to the shipbuilder and SIF/SDF are significant milestones in the equipment demonstration and manufacture programme covering delivery of preliminary, initial and final guidance information packs. The exchange of information in support of installation of the Combat System equipment on board, includes:

a) Compartment general arrangements;

b) Upper deck layouts;

c) Equipment definitions and detailed drawings;

d) Analysis and calculations;

e) Cable runs;

f) Pipe/cable sizing;

g) Piping isometrics;

h) Weights and Centres of Gravity (Cs of G);

i) Seating/mounting arrangements.

4.6.7 Subsystem/Equipment Demonstration and Manufacture

4.6.7.1 Objectives

4.6.7.1.1 Subsystem/Equipment Demonstration and Manufacture comprises equipment implementation up to the point of delivery from the factory for integration either into the SIF/SDF, FOC or follow-on vessel. This includes both the acquisition of NDIs/COTS equipment and the demonstration, initial manufacture and repeat manufacture of new or modified equipments.


Unclassified 108

4.6.7.1.2 At the Combat System level, activities are generally related to the monitoring of subsystem/equipment contracts to ensure that their contribution to the system design is progressed in a disciplined manner. As a consequence, the main priorities are:

a) Visibility of the subsystem/equipment design, demonstration and manufacture processes for the identification and management of equipment-level risks;

b) Acquisition of information in support of on-going system studies including performance parameters;

c) Provision of evidence in support of acceptance, both to satisfy compliance with the Subsystem Specifications, and in support of the Combat System acceptance strategy;

d) Enforcement of CM and change control processes under an appropriate and auditable regime;

e) Acceptance Arrangements.


4.6.7.2.1 The MoD CSM role is concerned with tracking the demonstration of high risk items through the review cycle, gaining early visibility of potential problems, and monitoring progress towards acceptance. This will be managed in the context of Combat System demonstration.


4.6.7.3.1 Prior to demonstration, the system will have identified/defined many components, which may be categorised as:

a) NDIs i.e. legacy/in-service equipments and equipments to be procured without modification as well as ‘off the shelf’ items including products developed for military application and COTS;

b) New equipments and modified variants of equipments in service. In addition, in order to integrate NDIs into the Combat System, protocol/data conversion units may need to be developed.

4.6.7.3.2 The items selected for inclusion in the system need to be acquired and the acquisition managed through a co-ordinated, structured and disciplined process as illustrated in Figure 4.7 and described in the clauses which follow.

4.6.7.4 Acquisition of Non-Development Items

4.6.7.4.1 Unmodified in-service NDIs which are to be acquired and incorporated into the Combat System without modification need to be procured in the most cost-effective manner. Acquisition will need to address the re-qualification of existing equipment against devolved requirements using existing test data wherever possible in support of the formal auditing process described later.

4.6.7.4.2 In-service equipments which have already achieved System Acceptance (SA) and which can be produced and used without modification need to be acquired though the most appropriate procurement route, determined by the CSI. Acquisition may often present problems however and obsolescence is a major concern. There may be valid commercial or practical reasons why suppliers choose not to re-start manufacture of existing designs, e.g. the expense of re-tooling after previous manufacture runs have long ceased, availability of manufacturer’s drawings (through manufacturer negligence, business failure or failure to ensure ESCROW), component obsolescence etc.


Unclassified 109

FOC Platform Design & Build

Follow-onPlatform

Integration

Acquisition of NDIs/COTS Equipment

Development of Pre-Production Prototype forShore Integration/Reference set

Initial Production Unit forFOC Integration Test & Trials

Repeat Production Units forFollow-on Platform Integration Test & Trials

Acquisition of New/Modified Equipment

RE SA SD PD DD Imp Int Test Acc

Requirement Reviews Design Reviews Test Reviews Audits

EquipmentRequirementsEngineering

Equipment-levelSystem Analysis and

Design

EquipmentPreliminary

Design

EquipmentDetailedDesign

EquipmentIntegration

EquipmentTests & Design

Acceptance

EquipmentImplement-

ation

Combat System Development SIF/SDF

Evaluated NDIs/COTS Equipment

Data Management Preparation for Integration

FOC

Figure 4.7 Subsystem/Equipment Demonstration and Manufacture

4.6.7.5 COTS

4.6.7.5.1 COTS technology may be defined as ‘readily available technology developed to exploit the requirements of the market place’. This includes both commercial products and items developed for military application. COTS is not a panacea to demonstration problems, since it brings with it constraints which need to be addressed in relation to the Combat System life cycle. These include:

a) Scope of COTS implementation (i.e. from single COTS component to complex interacting system of many COTS components) directly influences the significance of COTS issues at the Combat System level;

b) Acquisition of COTS products demands a flexible requirements engineering process through which trade-offs between requirements and offered product characteristics can be evaluated;

c) Integration of COTS components (particularly diverse software packages developed by competing organisations) into an operable system is a non-trivial undertaking. Incompatibilities are inevitable, given the diverse features of available products;

d) Testing demands a definition of the requirements to which the subject equipment has been designed. Requirements to which re-used/COTS equipment has been designed are not necessarily available or, if they are, complete;

e) Systems and equipments built from COTS components are dependent on the COTS suppliers for standardisation, documentation and business longevity for through-life support;

f) COTS products have their own unique upgrade paths, influenced by their trading market, which may affect compatibility between interacting COTS components. Maintenance is potentially complicated whether or not interfacing technology is used to contain the impact of one COTS item on another.

4.6.7.5.2 In order to be able to take advantage of the benefits, it is important that the design of the Combat System can in some way contain the rapid evolution of COTS products. This means that the Combat


Unclassified 110

System itself may need to evolve through a controlled regime of continuous engineering to co-ordinate design evolutions which meet the requirements, maximising the use of state-of-the-art products whilst taking action in anticipation of COTS obsolescence. The implementation of CM is therefore a key issue in the control and management of hardware and software, equipment and system build configurations.

4.6.7.6 Acquisition of New/Modified Equipment

4.6.7.6.1 New equipment and modified variants of existing equipment need to be acquired through a more rigorous process through which trade-offs between system and equipment-level issues can be more effectively raised and resolved. Close monitoring of equipment programmes is essential to ensure that they do not stray from the specified functionality as defined by the requirements and subsystem specifications. Any compromises, concessions, or design clarifications must be adequately assessed for their impact upon other subsystems and agreed changes reflected in baselined data, including models.

4.6.7.6.2 New equipments necessary to meet the Combat System requirement will have been identified during previous phases. Often and somewhat inevitably, a Combat System design will include equipments where the complexity and cost of the demonstration has necessitated a separate procurement programme. This is typically the case for the command system and principal sensor e.g. integrated sonar suite or major radar or tracker equipment. In these cases, the degree of independence of the equipment/subsystem is a major risk to the integrity of the Combat System. This must be afforded suitable visibility within the Combat System risk management regime.

4.6.7.6.3 Some equipment may require modification to meet the requirements of the Combat System design. The extent of the modifications will determine the validity of available documentation, models and performance data. Furthermore, the level of acceptance testing required proving compliance will also be an issue of negotiation and agreement on a case-by-case basis.

4.6.7.6.4 Though each equipment project will have its own specific demonstration strategy, the generic process can be described through the following phases:

a) Demonstration of a pre-manufacture standard prototype equipment for design acceptance testing (design proving), environmental qualification and Electromagnetic Compatibility (EMC) testing;

b) Refurbishment of the equipment and its installation, integration test and trials in a shore based facility (or for use as a reference set). The equipment unit used for design acceptance testing, environmental qualification and EMC testing is most likely to be used as a reference set due to the risk of the environmental testing seriously reducing equipment longevity below the bounds required for the platform fit;

c) Initial manufacture of the equipment to be subject to installation, integration test and trials on the FOC platform;

d) Full (repeat) manufacture of the equipment for installation, integration, test and trials on follow-on platforms.

4.6.7.6.5 The underlying approach is to demonstrate compliance with the requirements by means of a complete set of D&M tests. These are applied in full on the first set of equipment to be developed and produced. For subsequent manufacture runs, specific subsets of tests identified from the test hierarchy will be sufficient to demonstrate compliance thereafter.

4.6.7.7 Equipment Design

4.6.7.7.1 Design reviews provide visibility of progress (and opportunities for acceptance) of the design detailed by subsystem/equipment design specifications, drawings and supported by calculations.

4.6.7.7.2 The equipment design needs to be progressed from an established baseline, and early activities in the equipment demonstration process serve to formally determine the basis for subsequent demonstration. The extent to which formal reviews are applied for reviewing the design is an issue for consideration against the contractual milieu and what is considered most appropriate for risk management purposes. This includes:


Unclassified 111

a) Equipment-level requirements engineering culminating in review to ensure that the specific requirements are appropriate and achievable;

b) Equipment analysis and design activities to consolidate the proposed design and define it in terms of configuration items prior to embarking on further design work and committing to equipment assembly subcontracts/purchasing of equipment components.

4.6.7.8 Equipment Implementation and Integration

4.6.7.8.1 Equipment implementation proceeds with the procurement of equipment components and sub assemblies and the demonstration of hardware. Equipment integration involves the progressive build up of components, sub-assemblies, hardware and software into working equipment.

4.6.7.8.2 Test reviews provide a check that the preparations for testing and the associated test documentation is adequate as a basis for proceeding into formal testing. A test review is performed for each configuration item and will include an inspection of test results.

4.6.7.9 Equipment Tests and Design Acceptance

4.6.7.9.1 Equipment Test activities generally use a pre-manufacture prototype as the vehicle for demonstration and acceptance of the design and for subsequent environmental and EMC testing. At the system level, equipment tests will support system-level studies, providing metrics on predicted equipment performance based on low-level performance test data.

4.6.7.9.2 Design acceptance of the equipment will focus on physical performance and verify interfaces and data exchange in a dynamic representative scenario, within the constraints of stand-alone operation. Selected (previously run) tests will be used to confirm and formally agree test results presented for formal acceptance. Provisional acceptance will be sought for those elements of the design, which cannot be fully accepted until the completion of environmental qualification or later through Combat System integration test and trials.

4.6.7.9.3 Environmental qualification testing proves that the design is capable of supporting operation of the equipment in a representative ship or submarine environment in accordance with the equipment environmental test specification.

4.6.7.10 Configuration Audits

4.6.7.10.1 Configuration Audits are formal checks that each configuration item is functionally consistent and physically compliant with the requirements. Configuration audits should be conducted on all equipment with some rigour so that manufacture can proceed without having to repeat all tests on each item for proof of compliance.

4.6.7.10.2 Conducting configuration audits throughout the demonstration and manufacture programme on each configuration item in an incremental fashion provides a progressive build-up of proof of compliance and confidence in the manufacture build standard for the equipment.


Unclassified 112

4.6.7.11 Manufacture Readiness

4.6.7.11.1 Once acceptance of the design is achieved, equipment manufacture can commence. Manufacture readiness review determines the status of all outstanding demonstration-related issues, manufacturing concerns and resources, which must be resolved, before manufacture can be given the go-ahead. Unless a “Service” is being procured, an agreed stage, which may be prior to acceptance off contract, design documentation will come under the formal configuration control of the customer as a basis of contract. This is especially important for non-functional requirements of importance e.g. performance, safety and security. Critical Design Reviews are normally used to mark this transition if prior to acceptance.

4.6.7.12 Equipment Factory Testing

4.6.7.12.1 Equipment Factory Testing repeats a representative set of design acceptance tests to verify that the manufacture equipment meets the design requirements. This involves a series of tests of the installed equipment including mechanical and cable continuity checks and a comprehensive set of function tests to verify interface operation and quantify physical performance.

4.6.7.13 Related Information

4.6.7.13.1 Subsystem/equipment demonstration has been the subject of much standardisation and additional guidance is available within the AMS and Def Stan documents relevant to weapon equipment procurement, and Naval systems Upkeep.

4.6.8 Acceptance

4.6.8.1 Objectives

4.6.8.1.1 The primary vehicle for demonstrating and ensuring compliance with the requirements is the acceptance process. This is a central theme of modern SE practice, which demands due attention at each phase of the life cycle.

4.6.8.1.2 Acceptance is an incremental process through which evidence of compliance with the requirements should be accumulated throughout all project phases. Adoption of such an incremental process leads to a schedule of acceptance, which demonstrates compliance as early in the project as possible, consistent with the need to generate definitive evidence. This approach serves to minimise risk of eventual failure, and hence reduce overall programme costs.

4.6.8.2 Combat System Issues

4.6.8.2.1 As is the case for many aspects of this phase, acceptance of the Combat System has to be placed in context. The basis of acceptance is determined by the requirements. In the case of the Combat System, these are likely to comprise Combat System-level requirements (which define the required capability of the Combat System as a whole), lower level requirements and constraints on component equipment, as well as constraints imposed on and by the platform.

4.6.8.2.2 Acceptance of the Combat System is closely coupled with acceptance of the hierarchy of equipment and subsystem components of which it is constituted, and the platform on which it is fitted. Although many aspects of acceptance cannot be completed until later phases, the planning and conduct of acceptance must be given a high priority, since it is an integral part of the demonstration, manufacture, delivery and integration of component parts. Acceptance risk may be further contained through the adoption of a structured modelling approach as detailed in systems studies (see later).


4.6.8.3.1 The CSM must liaise closely with the MoD acceptance team dedicated to dealing with the acceptance activities which fall within the remit of the MoD organisations involved. This will depend on the contractual arrangements under which the Combat System contract is let, however the main acceptance responsibilities of the CSM, with respect to the Combat System, are:


Unclassified 113

a) Co-ordination of MoD resources involved in the acceptance process including Royal Navy organisations, in particular the Acceptance Authority, and Ships’ Staff Standing By. Relevant organisations should be recorded in the TLMP and be members of the Acceptance Community of Interest;

b) Together with the Requirements Manager, negotiation, agreement and approval of detailed acceptance proposals to demonstrate that contract requirements will be met;

c) Witnessing of acceptance events and presentation of acceptance evidence;

d) Signification that evidence submitted clears defects, deficiencies, and problems;

e) Processing of concessions, waivers, or changes relating to MoD requirements;

f) Conferral or recommendation of formal acceptance.

4.6.8.4 Acceptance Activities during Demonstration and Manufacture

4.6.8.4.1 The strategy for acceptance through the Combat System programme will have been explicitly detailed by the ITEAP generated previous to this phase. The ITEAP serves to define roles and responsibilities both of MoD and industry across the acceptance process, which is illustrated schematically in Figure 4.8.

PD DD Imp

RE

SA

SD Int

Test

Acc

Lower-levelSubsystem/Equipment



Subsystem/Equipment

Deliverables



Formal Acceptance

Combat System Development

RE

SA

SD Int

Test

Acc

Verification

Verification of Equipment Design

Validation of Equipment/Compliance with Requirements

Subsystem/EquipmentDevelopment and Production

Verification

Verification of System Design

Management of the Acceptance ProcessVisibility of progress towards Acceptance/

Progressive Acceptance



Validation of System/Compliance with Requirements

Figure 4.8 Combat System Acceptance Process

4.6.8.4.2 The acceptance process can be thought of as ‘closing the square’, satisfying Combat System design requirements cascaded down the subsystem/equipment hierarchy by the accumulation of acceptance evidence produced from lower level acceptance events.

4.6.8.4.3 At each level, verification of the transformations between requirements, analyses, design, implementation and test activities ensures traceability of requirements to eventual products. This validation mechanism supports certification and acceptance.


Unclassified 114

4.6.8.4.4 Visibility of the progress towards acceptance is an important consideration. Reviews are a major element in the acceptance process, especially in determining whether the acceptance evidence generated satisfies requirements. The acceptance process has therefore to deal with both success and failure of acceptance events and link in to general management processes to initiate and be involved in recovery programmes if defects and deficiencies arise.


4.6.9.1 Objectives

4.6.9.1.1 System Studies are those activities, which are continued throughout the Combat System life cycle providing descriptive, analytical or simulation-based symbolic models of system emergent properties and predictive indications of system performance across a number of key disciplines.

4.6.9.1.2 In support of the demonstration and manufacture phase, the primary objective of modelling is to plan and anticipate integration across equipment interfaces, to predict system performance and to assess the effect of introducing changes to the Combat System baseline design.

4.6.9.2 Inter-related Modelling Activities

4.6.9.2.1 The Combat System can be regarded as one level in the hierarchy of modelling activities conducted to varying degrees during this phase. This is depicted in Figure 4.9 which shows the typical span of inter-related modelling activities.


Unclassified 115

OVERALL CAPABILITYThe capability of the Warship(according to its Roles, Tasks,

Missions and Scenarios) need to beevaluated in the context of other

assets

OA Studies

PlatformStudies

Whole WarshipStudies

EffectivenessModelling

SystemAR&MHCI Prototyping

Operability

Weapon & Sensor ArcCoverage

NavalArchitecture

CompartmentArrangements

Cable Runs/ServiceSchematics Service

Budgets

LSAVulnerability LCC/TLC

Human FactorsSignatures

PhysicalArchitecture

Combat SystemStudies

COTS/NDIsBenchmarking

is necessary to supportevaluation of Non

Development Items

PREDICTIVE INDICATIONCombat System Studies contributeto Warship performance prediction

and provides a basis for acceptanceof operation-dependent design

aspects

PLATFORM INTERFACESThe interaction between Platform

Design & Build and CombatSystem engineering requires

effective co-ordinationEQUIPMENT MODELS

Definitive ‘as-built’ EquipmentPerformance data become availableas the Equipment Design proceeds

through its Life Cycle

REAL DATADefinitive ‘as-built’ Equipment

Performance data becomeavailable as the Equipment Design

proceeds through its Life Cycle

EquipmentModelling

EquipmentEvaluation

EquipmentTest & Trials

SystemFunctionality

PlatformDesign and Build

Figure 4.9 Combat System Studies in Context

4.6.9.2.2 The D&M phase is typified by detailed design and manufacture information becoming available through evaluation and test of developed or acquired equipments. This can be used to refine equipment models, which contribute to Combat System-level modelling activities. Similarly, as the platform design and build process progresses, corresponding platform studies will further define many of the physical constraints which need to be considered for successful installation and integration of the Combat System with the platform.

4.6.9.2.3 Combat System and platform modelling activities need to be co-ordinated with each other where they overlap and where they contribute to the overall ‘whole warship’ characteristics, in particular operational effectiveness and life cycle costs. Ultimately the characteristics of the whole warship in turn contribute to a defined level of capability assessed through Operational Analysis (OA). All the above have to be carried out in the context of agreed concepts of operation.


4.6.9.3.1 The modelling activities conducted across each of these areas need to be managed across the hierarchy. It is ultimately the responsibility of the CSM to ensure that modelling activities conducted in other areas, such as that for equipment programmes, or platform-oriented modelling, which can support the Combat System level objectives are identified, coherent with interfacing systems, and modelling resources made available to the Combat System project.

4.6.9.3.2 The CSM also must ensure that all Combat System-level modelling activities are progressed in a concerted manner, and at the appropriate level commensurate with the purposes of risk reduction. To achieve this, a decision must be made whether to maintain Combat System models (delivered from the


Unclassified 116

previous Assessment Phase) within a MOD/Defence Science and Technology Laboratories (Dstl) controlled reference facility, in order to independently assess and influence the system demonstration process, or devolve responsibility and maintain close visibility of contractor modelling activities.

4.6.9.3.3 In either case, the CSM must ensure maximum compatibility between outputs of the Combat System level modelling activities and inputs required for modelling at project and capability levels.

4.6.9.4 Modelling Activities conducted during previous Phases

4.6.9.4.1 Models assembled during previous phases, and which represent various aspects of the Combat System design baseline, should be carried through demonstration and beyond. This is a continuing theme consistent with the modelling strategy formulated during previous phases and identifies the use of modelling:

a) For analysis to further understanding of system behaviour and to provide confidence in the projected system operation and performance;

b) For specification through representation of particular facets of the design and identify and specify information flow across interfaces;

c) To consider dynamic aspects of the system in operation;

d) In support of the assessment of a proposed design or comparison of options against defined criteria and support trade-off studies;

e) As a formalised description of the Combat System configuration baseline against which changes can be assessed and their impact explored;

f) As required by MoD policy e.g. Views as required by the MOD’s Architectural Framework (MODAF).

4.6.9.4.2 In pursuit of these objectives, modelling outputs from previous phases are likely to include Functional Models (including logical models of the requirements analysis, and physical models of equipments and system design), prototypes of equipment/subsystem Man Machine Interfaces (MMIs), Equipment AR&M analyses, and preliminary effectiveness models using agreed scenarios. Furthermore, platform studies will have generated draft platform services’ budgets published in the Weapon Equipment Schedule (WES) and draft spatial layouts and fitting arrangements which will provide the basis for consideration of platform integration aspects.

4.6.9.5 Modelling Activities during the Demonstration and Manufacture Phase

4.6.9.5.1 The main objective of modelling in this phase is to use and evolve models generated from earlier activities to:

a) Enumerate and aid the understanding of requirements, to assist in requirements analysis and support the manufacture of system and subsystem specifications;

b) Refine model fidelity using data generated from the performance evaluation of real equipment;

c) Assess the impact of any shortfalls or improvements in system design performance;

d) Support the optimisation of the design;

e) Support investigation and problem solving associated with integration across equipment interfaces;

f) Identify under-utilised or dormant capability (in terms of functionality and architectural performance) accompanying the incorporation of NDIs;

g) Support acceptance of design aspects which cannot be definitively or practically demonstrated (e.g. operational effectiveness);

h) Support subsequent analysis of test and trials results e.g. pre-trials predictions and post trials analysis;


Unclassified 117

i) As required to support MODAF.

4.6.9.5.2 Modelling is an important technique for the investigation of system performance aspects. Many of the objectives above require a complementary approach to modelling where the key performance parameters addressed by one model can be used as a basis for further study in another aspect. The subject of framework methodologies for identifying, deriving and quantifying key performance parameters, and their relationship to MOEs, is a developing area currently pursued by both Dstl and industry.

4.6.9.6 Prototyping

4.6.9.6.1 Prototyping particular elements of the Combat System design, such as MMIs and novel signal processing algorithms, for evaluation outside the core design process is encouraged as a risk-reduction measure. The usefulness of prototypes however must be justified through trade-off between the cost of developing representative prototypes to the required fidelity, and the confidence, which can be ascribed to the results of prototypes, especially in the area of performance.

4.6.9.6.2 Prototypes developed during previous phases can be carried through system demonstration and manufacture and maintained alongside the evolving design. Some prototyping tools now offer the facilities to quickly prototype aspects of the system design as a means of exploring design issues. Prototypes generated through these means can then be used to support specification and detailed designs.

4.6.9.7 Functional Modelling

4.6.9.7.1 Functional models provide a descriptive record of the baseline design against which the detailed design can be evolved, monitored and evaluated, as a basis for predictive studies and for investigating potential discontinuities across subsystem/equipment boundaries. A functional (physical) model can be used as the co-ordinating baseline both as a reference of the evolving design and as a basis for further investigations into other performance aspects such as operability, architectural performance and AR&M.

4.6.9.8 Human Factors and Operability

4.6.9.8.1 With appropriate HCI operability models and MMI demonstrators developed ahead of equipment manufacture, potential operator performance, equipment/subsystem inter-operability and proposed drills can be evaluated without the need for manufacture hardware and/or software.

4.6.9.8.2 These can be refined, as design performance data becomes available. Parameters, which reflect equipment and system operability may be entered into effectiveness models to assess their impact on system effectiveness in, agreed operational scenarios.

4.6.9.9 Architectural/Engineering Performance Studies

4.6.9.9.1 Architecture modelling serves to explore the physical characteristics of the Combat System design to produce an objective analysis of:

a) Interface loading identifying bottlenecks and under-utilised capacity;

b) System response to interface over-loading;

c) Likelihood of data loss due to overwriting of data, on both interfaces and buffers;

d) Processor loading;

e) System response times;

f) Interaction with other sub-system components and external systems.

4.6.9.9.2 The Combat System design as defined for demonstration and manufacture provides the most stable, detailed and thereby valid baseline for detailed architectural investigation. As subsystem and equipment design information becomes definitive, an architectural model can be developed to represent the evolving solution and provide a realistic model of system dynamics for evaluation of key performance parameters.


Unclassified 118

4.6.9.10 Availability, Reliability and Maintainability AR&M

4.6.9.10.1 AR&M are vital operational characteristics and between them have a dominant impact upon both operational effectiveness and Life Cycle Costings (LCCs). Initial AR&M assessments in Assessment Phase can only at best reveal indications of potential problem areas. Detailed design information (down to component level) produced during demonstration is necessary before any meaningful AR&M statistics can be generated. As more definitive design information becomes available (including from prototypes and pre-production equipment), detailed AR&M analyses of equipment may be conducted and AR&M prediction may yield results of more confidence. Impact of significance to the TLMP must be recorded.

4.6.9.11 Operational Effectiveness

4.6.9.11.1 Effectiveness may be considered as how well the Combat System performs its allotted missions, roles and tasks in pre-determined scenarios, and is derived from an assessment of the system’s performance expressed in terms of its functionality, architecture, operability and availability.

4.6.9.11.2 To maintain a common basis for performance assessment as a whole, it is desirable that operability, AR&M and effectiveness be assessed against a set of common scenarios, which must be coherent with approved operational concepts and maintained by MOD. This should give an indication of the use of the system in a representative set of conditions, so that meaningful objective comparisons can take place.

4.6.9.11.3 As the detailed design solution is realised and indicative performance is provided through other modelling and evaluations, the effectiveness model developed during earlier life cycle phases can be further refined. An effectiveness prediction can also take into account the compliancy of solutions against functional and performance requirements and assess how effective the system as whole is as a result of the additional capability. Results which have an impact on the requirement must be recorded and may be used to advise trade off decision.

4.6.9.12 Model Validity

4.6.9.12.1 The validity of the modelled system is an important consideration and care should be taken to verify and calibrate model outputs against real-world data wherever possible. Any conclusions made as a result need to be qualified in terms of the limitations of the modelling technique and the fidelity of the model representation. With many models, validity cannot be formally assured and, in practice, confidence must be gained in them in part through their usage. As with other aspects, modelling must be under configuration control.

4.6.9.13 Synthetic Environments

4.6.9.13.1 Synthetic Environments are distributed networks of interacting systems which can be used for system demonstration and evaluation, training, mission rehearsal, strategy and tactics demonstration. Synthetic environments can be created involving a wide variety of training simulators, war game facilities and real equipment in live exercises all integrated through the extensive use of specialised protocols. These techniques are currently emerging and should offer scope for reducing integration risk.

4.6.9.13.2 The concept aims to unite available and emerging technologies to gain an insight into system functionality at the earliest opportunity. It proposes the implementation of a virtual system, using a combination of established equipments, bespoke hardware-based simulators and model-based simulated equipments using modelling tools.

4.6.9.13.3 System functionality can be evaluated using a library of standard acceptance scenarios, which cover examples of the physical and operational environments. Each of the entities which form the system need not be present at any one location but can take part in a system simulation either locally to a scenario controller or via telecommunications.

4.6.9.13.4 This approach enables activities akin to system integration at a system level based upon design concepts rather than implementation. As equipments become available, equipment prototypes and the system re-examined, repeating trials performed with the simulators to investigate system compatibility can replace simulations. The results of these trials can be fed reiteratively into the system demonstration phase to yield a more compliant and effective system. The evolution of the system can continue through the


Unclassified 119

life cycle to provide the basis for a shore facility in which training and other activities requiring a valid and accurate representation of the final system can take place.

4.6.9.14 NDIs and COTS

4.6.9.14.1 For modelling to reveal its potential, detailed design data is a necessary pre-requisite for all aspects of the assessment. The adoption of NDIs (excluding legacy systems for which performance data is available) and widespread usage of COTS technology is likely to provide less visibility of the detailed design since Intellectual Property Rights (IPRs) are fiercely protected. Where NDIs and COTS provide a critical component of the Combat System design, then it will be necessary to provide an independent evaluation of the subject equipment across a predetermined set of performance evaluation criteria.

4.6.10 Integrated Logistic Support (ILS)

4.6.10.1 Combat System-level ILS Issues

4.6.10.1.1 The demonstration and manufacture phase delivers the military capability required and expressed in the URD – this includes the support solution. From the Combat System perspective, this phase is typified by the acquisition of individual equipments and preparation for their integration together and with the platform environment and through life maintenance.

4.6.10.1.2 All ILS activities need to be considered from both system and platform integration viewpoints through extensive liaison with both EPMs and the platform design authority. In support of this objective, the WSM will have access to appropriate ILS expertise within his team or maintain direct lines of communication with a dedicated ILS team. The maintenance of external interfaces is also part of the support solution and business agreements are required to back this up.

4.6.10.2 ILS Activities during the Demonstration and Manufacture Phase

4.6.10.2.1 The main thrust of ILS at this phase is the continued application of ILS and Logistic Support Analysis (LSA) principles throughout the subsystem/equipment acquisition process, and the preparation of optimum arrangements to satisfy the Combat System’s operational and upkeep needs when in-service. This will be achieved through the fully tailored ILS/LSA programme with inputs to the LSAR. The range of ILS activities to be conducted during this phase is illustrated in Figure 4.10 which also provides Def Stan 00 60 task designations where appropriate.


Unclassified 120

Maintenance Planning Supply Support

Support & Test Equipment Manpower and Human Factors

Training and Training Equipment Technical Documentation

Packaging,Handling,Storage & Transportation Facilities

Non-Operational Computer Resources Support Life Cycle Costing

Reliability & Maintainability Safety

ILS/LSA 100 PLANNING &

CONTROL

200 MISSION & SUPPORT SYSTEM DEFINITION

300 PREPARATION & EVALUATION OF SUPPORT SYSTEM ALTERNATIVES

400 LOGISTIC SUPPORT REQUIREMENTS

500 SUPPORTABILITY ASSESSMENT

Review (and update) ILSP and LSAP [102] Programme and Design Reviews [Task 103]

Update Use Study [Task 201] Mission & Support System Standardisation [Task 202]

Comparative Analysis [Task 203] Review Technological Opportunities [Task 204]

Review Supportability/Design Factors [Task 205]

Review Functional Requirements [Task 301] Review Support System Alternatives [Task 302]

Review Trade-off Analysis [Task 303]

Perform Task Analysis [Task 401] Early Field Analysis [Task 402]

Post Production Support Analysis [Task 403]

Supportability Assessment Report [Task 501]

LSAR Reports

LSAR

PLATFORM

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

EQUIPMENT ACQUISITION ILS Monitor system/equipment design process

Ensure equipment efficiently supported Accept Contractor ILS Deliverables

PLATFORM DESIGN & BUILD ILS Ensure CM/Platform Fit Definition system

set up and operated correctly Provide the Platform Authority with evidence of support requirements

compliance

Figure 4.10 ILS Activities Conducted during the Demonstration and Manufacture Phase

4.6.10.2.2 During demonstration, identification of detailed support requirements can be undertaken since the system/equipment design should be reasonably stable. Whilst some design optimisation is still possible it should be limited. During this phase, the support infrastructure can be identified and optimised to ensure:

a) Compliance with the overall platform support concept;

b) Limitation of the support requirements (especially unique to type support);

c) Consistency with the available support infrastructure;

d) Availability when required.

4.6.10.2.3 During manufacture, support items should be procured and delivered. The objective is to provide a cost effective level of support when required. Care must be taken to ensure that any design changes during manufacture are considered and reflected in the support procurement process. Issues relating to support during acceptance and hand-over must also be anticipated.

4.6.10.2.4 Throughout this phase, the CSM needs to maintain overall visibility of ILS activities through close liaison with the ILS manager in order to ensure that the overall strategy is being implemented in a coherent manner. This will ensure that all deliverables, and in particular the ILS-related items (such as the LSAR, handbooks, training and support infrastructure) continue to meet the overall Combat System support requirements at the optimum LCC.

4.6.10.2.5 MoD needs to maintain a close ‘watching brief’ through the scheduled programme of design reviews and audits as determined by the project review cycle. The review cycle must address and review all ILS aspects as an integral part of the demonstration and manufacture process across all levels of the system/subsystem/equipment hierarchy.

4.6.10.2.6 The degree to which scrutiny is necessary depends on the overall ILS strategy appropriate to the Combat System project. The application of risk identification and management techniques from the


Unclassified 121

support perspective across the system hierarchy should serve to identify those areas requiring close monitoring.

4.6.10.2.7 The key activities during this phase are:

a) Develop Support Policy Statement;

b) Review ILS funding and Long Term Costing (LTC) Estimates;

c) Maintain ILS and LSA Strategies;

d) Maintain Use Study;

e) Maintain ILS and LSA Plans;

f) Review of Contractor ILS and LSA Plans;

g) Review LSA Tasks in Progress;

h) Perform Detailed Task Analysis, and Enter Results of Maintenance Task Analysis into LSAR;

i) Produce Strategy for LSAR In-service;

j) Identify Support Resource Requirements;

k) Update and Review LCC Modelling;

l) Monitor Contractor Logistic Demonstration.

4.6.10.3 Liaison with the Platform Design Authority

4.6.10.3.1 ILS activities conducted throughout the demonstration and manufacture of constituent equipments of the Combat System cannot be conducted in isolation from the platform design and build process. The overriding objective here is to synchronise, through the project review cycle, the ILS/LSA programme across design, AR&M, HF and LCC areas. This is to ensure that the Combat System and the platform together conform to the through-life support requirements and that there is commonality of purpose across all ILS activities.

4.6.11 Documentation

4.6.11.1 Objectives

4.6.11.1.1 The D&M Phase marks the beginning of Combat System demonstration as an enterprise across a number of co-operating organisations necessary to realise the design. The suite of documentation generated during previous phases continues as statements of Combat System-level objectives and design intent. These objectives are cascaded through the organisational hierarchy as contractual requirements and give rise to a plethora of documentation of increasing detail, which refine the requirements and provide details of their implementation.

4.6.11.1.2 Within the scope of this guide it is not possible to identify each and every deliverable document across the hierarchy. However, a summary of the key documents is provided together with indications of associated action required during this phase.

4.6.11.1.3 The role of the MoD CSM is to ensure that all the required documentation is produced to the requisite content and quality, faithfully reflecting the emerging design which will satisfy all the overall Combat System objectives.


Unclassified 122

Table 4.4 Summary of Documentation

Activity Documentation (Output Products) Associated Activities

Systems Engineering (Technical) and Project Management

Policy and Strategy Papers Management Plans (see Project Management) System Security Policy Software Management Policy Configuration Management Plan Life Cycle Costings Safety Cases

Updated to keep abreast of demonstrations during the phase. Updated, reviewed and agreed by CSM.

System Requirements

URD, SRD CONEMP/CONUSE ITEAP TLMP

Configured as baseline data, together with acceptance data Changes raised by DEC defined by IPT RMs and investigated by CSM. Changes against the baseline embodied in baseline increments when agreed.

Change investigation and impact analysis performed by CSI. Agreed changes promulgated down the contractual hierarchy as necessary.

The CONUSE is developed from the CONEMP to reflect the actual system solution.

System Design System (Design) Specification Coherence Specification Subsystem Specifications Interface Documentation (SIDs, IFSs, DESs)

Maintained by CSI under direction of CSM. Configured as baseline data. Used to support investigation of change impact as above.

Combat System Demonstration

Data Management Policy Shore Facility Integration Strategy

Developed by the CSI under direction of CSM.

Equipment Demonstration and Manufacture

Equipment FDIP Process Plans Equipment Requirement/Procurement Specifications/SOW/STIR Equipment AR&M Reports Equipment Interface Documentation Equipment Manufacture Drawings Equipment Guidance Information (IGI & FGI) Equipment MRI data Equipment Test Schedules and results.

Developed by EDC under direction of CSI (or EPM dependent on Design Authority arrangements). Reviewed and agreed by CSI and CSM.


Unclassified 123

Summary of Documentation (continuation)


Platform Design Design Baseline Definitions (Naval Architecture, External Structures, Deck Structures, Bulkhead Penetrations, Chilled Water, Ventilation, Hydraulics, Heat Management etc.) Design Disclosure Papers Load Analysis Reports Platform Design Assessment Reports Safety Justification Report General Arrangement (GA) Drawings Signature Management Plan Interface Certificates Interim Guidance Information (IGI) Final Guidance Information (FGI)

Developed by PDC under direction of PPM. Reviewed and agreed by PPM and CSM.

System Studies Operational Scenario Definitions Effectiveness Models Effectiveness Assessment Reports

Conducted by CSI under direction of CSM. Reviewed by DEC and FLEET

Operability Demonstrators System Operability Assessment Reports

Conducted by CSI under direction of CSM. Reviewed by IPT RMs and CinCFleet.

Functional Models Architecture Assessment Reports AR&M Modelling and Reports Trade-off Study Reports

Developed by CSI under direction of CSM. Subsystem/Equipment-level modelling conducted by EDC under direction of CSI/CSM. Reviewed and agreed by CSM.

Integration, Test & Trials

Integration Test Evaluation and Acceptance Plan Shore Integration PFA Strategy Integration Test Requirements/Specifications Test Equipment Specifications Trials Policy, Trials Requirements, Trials Management Plan Subsystem Integration Test Procedures and Reports Combat System Integration Test Procedures and Reports

Developed by CSI under direction of CSM. Reviewed and agreed by CSM.

Equipment II/Trial Procedures and Reports Equipment PFA/STW Procedures and Reports Equipment HAT(E) Procedures and Reports Equipment Integration (PINT/SINT/FINT) Test Procedures and Reports



Unclassified 124

Abbreviations

CSM MoD Combat System Manager ILSM MoD ILS Manager CSI Combat System Integrator CSSC Combat System Support Contractor PPM Platform Project Manager PDC Platform Design Contractor EDC Equipment Design Contractor


4.6.12.1 Objectives

4.6.12.1.1 Specialist Discipline Support is a general term for the many different specialisms, which need to be considered during the design, demonstration and build of the Combat System. The main purpose of grouping these separate specialisms together under a single heading is to make sure that they are individually resourced and collectively addressed.

4.6.12.1.2 The main purpose of specialist discipline support at this phase is to ensure that all design, demonstration and manufacture issues are appropriately addressed from each specialism viewpoint. For each specialism, the objectives are to:

a) Review the Combat System design solution and identify improvements to the design where possible; (Note: Non-functional areas, e.g. Security, Safety and Performance, especially where certification require specialist support);

b) Predict the performance of the design;

c) Devise and agree the tests and trials which will show that the design meets the requirements;

d) Review evidence of compliance with the requirements.


4.6.12.2.1 The MoD CSM role is to provide guidance to the Combat System integrator, to review and approve progressive acceptance evidence in support of the specified requirements. In support of this objective, the CSM will require access to a pool of specialist engineering expertise either within his own team, or temporarily assigned from elsewhere.

4.6.12.3 Concurrent Engineering

4.6.12.3.1 Concurrent engineering is a key theme for the demonstration of integrated systems. The implementation of concurrent engineering practices must serve to apply participating specialist disciplines to ensure that the evolving design can be efficiently produced and supported. In organisational terms, this requires an integrated multi-disciplinary team approach to system demonstration, wherein peer participation of all engineering disciplines is applied throughout each system life cycle activity. Concurrent engineering places significant demands on understanding, processes and procedures to achieve overall system-level technical coherence.


4.6.12.4.1 As with all previous phases of the life cycle, the consideration of the evolving design from a number of engineering discipline perspectives is required to support trade-off analysis to optimise the system design. Specialist support is needed in the areas identified in Table 4.5.


Unclassified 125

Table 4.5 Specialist Discipline Support

Engineering Discipline Priority Activities at this Phase

Air Engineering Liaison with aircraft project(s) to refine Combat System interfaces.

Alignment Review design/implementation documentation to ensure that all equipments (in particular Command System, sensors and effectors) are aligned to a common platform datum.

AR&M Ensure visibility of equipment and system AR&M analyses in support of predictive compliance with AR&M requirements.

Cost Maintain and refine Through Life Costings/LCCs as demonstration progresses. Influence the demonstration process to optimise LCCs.

Documentation Review and approve operating documentation including handbooks, operating instructions, test instructions, maintenance instructions and training material. See also Def Stan 00-60, RN ILS Manual.

EMC, RADHAZ, Tempest, Magnetic and Nuclear Survivability

Identify, plan and manage signature parameters including EMC, Radiation Hazards, Tempest, Magnetic, Electromagnetic Pulse and Electrostatic Discharge. Review predictions of system susceptibility and emissions, and proposals for improvements. See Def Stan 08-124.

Environmental Factors Ensure that equipment is developed, tested and accepted against appropriate and realistic environmental requirements (atmosphere, temperature and humidity, noise, vibration, shock, watertight integrity, fire etc). See Def Stan 08-124.

HF/Operability Raise and promulgate fleet-wide HCI and ergonomics issues and assess impact of evolving design on manpower, tasking and skills requirements. Develop HF Log detail and conduct regular assessment of Combat System HFI. Update and revise HF AQ and ACs. Oversee demonstration of training aids, rigs and simulators. See Sea Technology Group Publications 10 & 11.

ILS Facilitate design for support trade-offs. LSAR population. See ‘ILS’ in this Section, Def Stan 00-60, RN ILS Manual.

Installation and Ship Services Ensure early Guidance Information is made available to the platform projects. See ‘Combat System Demonstration’ in this Section.

Interface Data Management Ensure Interface and data management controls are in place iaw Integration Strategy. See ‘Combat System Demonstration’ in this Section.

Modelling Use of MoD reference sets to verify modelling outputs from Equipment-level and Combat System-level modelling activities in particular functionality of physical design, operability and operational effectiveness. See ‘System Studies’ in this Section.

Specialist Discipline Support (continuation)


Mutual Interference Physical, electronic, shock, electro-magnetic.

Naval Architecture Liaison with platform/whole warship design and build projects for review of


Unclassified 126


Combat System contributions to buoyancy, stability, Cs of G etc.

Operational Performance/Effectiveness

Prediction of MOPs and MOEs in support of capability assessment. Refinement and agreement of operational scenarios used for modelling and acceptance.

Safety Review safety issues through auspices of warship-level Safety Panel in accordance with JSP 430, DEF STAN 00-56 and the Health and Safety at Work Act (HASAWA). Develop Ship Safety Case.

Security Ensure compliance with the System Security Policy for identification and authentication, control of access, security accounting and audit, TEMPEST protection, and generation/update of pertinent Security Operating Procedures. In accordance with JSP 440.

Shock and Vibration Ensure that new equipments are designed to withstand shock parameters as assigned iaw DG Ships Parts 1 and 2. Further information regarding shock is provided in the Shock Manual. See BR 3021 and Def Stan 08-107.

Software Demonstration Ensure that all software is developed according to a consistent process capable of supporting predictive and quantifiable metrics (in accordance with an established Software Capability Maturity Model).

Structural Engineering Review structural analysis using finite element analysis models and tools.

Training Review and agree training needs of operators, maintainers and base support, training equipment, including shore-based and ‘on-job’ facilities. See also RN ILS Manual.

Interoperability IO is a mandated KUR for all projects delivering CIS content, See AMS IO CIS Topic. Requirements to support it will apply to many equipments.

4.6.12.5 AR&M

4.6.12.5.1 Throughout the D&M Phase, the Reliability and Maintainability (R&M) project panel will continue to provide the project manager with advice and monitor the achievement of R&M requirements by the contractor. If it becomes apparent that the R&M requirement will not be met, or achievement may be threatened by financial or timescale constraints, the project panel should review the situation and advise the project manager so that he may decide the best course of remedial action.

4.6.12.5.2 Hence it is important that the AR&M project panel maintains full visibility of R&M matters throughout project demonstration and continue to function until System Acceptance (SA) has been achieved. Evidence that R&M requirements specified in the URD have been met is required before a submission for acceptance can be made.

4.6.13 Configuration Management (CM)

4.6.13.1 Objectives

4.6.13.1.1 CM of the emerging design is a critical activity within the D&M Phase. All information associated with the Combat System/subsystem/equipment hierarchy and its relationship to the platform needs to be managed under a formal regime in accordance with the CM plan.

4.6.13.1.2 The main objectives of this activity are to ensure that the design is progressed through the contractual hierarchy though an auditable set of inter-relating processes, and that change is managed in a controlled manner. CM therefore supports the assessment of change against incremental baselines used to co-ordinate and communicates the evolving Combat System design solution.


Unclassified 127


4.6.13.2.1 The primary role of the CSM is likely to be that of ‘informed customer’ ensuring that the CM function is adequately policed and that CM baselines are co-ordinated across all D&M processes. The MoD activities during this phase should be thought of as the top level of the design and implementation ‘pyramid’ as depicted graphically in Figure 4.11. MoD needs to ensure that the SE process applied throughout the Combat System demonstration and manufacture enterprise is optimised to engineer products which comply with the requirements and support the specified level of capability.

4.6.13.2.2 The level of involvement should be determined chiefly by the contract management policy. Clearly, it is important for the CSM to keep a close watch and actively participate in all Combat System level reviews and audits which should highlight any lower-level activities in the subsystem/ equipment areas, which require closer scrutiny. In parallel, the risk management process should highlight any potential areas of concern well before actual problems arise.

4.6.13.2.3 It is important that MoD activities in support of Combat System demonstration and manufacture activities are co-ordinated against an appropriate project review cycle. This is so that the results of any intra-mural or independently funded system studies (such as those conducted using MoD reference models) can be used to influence demonstration and manufacture at the earliest opportunity.


4.6.13.3.1 CM, though important during previous phases to keep track of requirements, specification and plans, becomes a fundamentally significant issue with the proliferation of requirements, specifications, physical deliverables, test and acceptance data during the demonstration and manufacture phases. CM provides the means for co-ordinating the design and definition of the product and for assessing and progressing design changes. All products and documentation must be configured within the CM regime. The various facets of CM during these phases are illustrated in Figure 4.11, which illustrates the extent of CM of SE activities across the Combat System hierarchy.

4.6.13.3.2 CM must be seen in the context of the overall management objectives documented by management plans throughout the enterprise. Project management documentation must also be configured within the Combat System CM regime.


Unclassified 128

Support Baselines

Acceptance Baselines

Verification Baselines

Physical Baselines

Functional Baselines

Requirements Baselines

Subsystem/ Equipments

Combat System

Whole Warship

The Project Review

Cycle imposes formality across

Development and Production, synchronising

Combat System-level and lower level activities

MOD needs to be confident that all levels of the hierarchy are audited. This is to ensure that the SE process is optimised to engineer products which comply with the requirements and support the specified level of capability.

Ultimately, Configuration Management is a whole-product issue, which needs to be co -ordinated across the Combat System/subsystem/equipment hierarchy. Each level must apply CM to an appropriate degree, supported by automated tools and procedures.

Figure 4.11 Configuration Management Baseline

4.6.13.4 Baselines

4.6.13.4.1 CM provides the mechanisms through which the design can be taken through D&M in a controlled manner. This is achieved through imposition of CM baselines across each of the SE activities. The baselines provide a progressive scheme, which records the status of the design, demonstration, manufacture, integration and test processes with rigour so that the design is supported by an auditable regime.

4.6.13.4.2 The following items need to be incorporated in the configured project baselines with appropriate mechanisms for change to be authorised, controlled, recorded and managed:

a) Management Plans;

b) Requirements hierarchy with associated visibility of constraints and acceptance information;

c) Models;

d) Build Records;

e) Design and Demonstration Documentation;

f) Manufacture information, in particular Installation Guidance Information.

4.6.13.4.3 A number of baselines across SE activities are recommended as follows:

a) Requirements Baselines - identify all operational, functional, physical and performance requirements together with their constraints, source derivations and allocations;


Unclassified 129

b) Functional Baselines - determines the status of functional requirements as explored and expressed through modelling of the design and supported by traceability between requirements and model elements;

c) Physical Baselines - record the evolving design implementation as it is progressed through design, demonstration and manufacture activities;

d) Verification Baselines - to manage the models and analyses used to predict Combat System performance and to refine their predictions, as more detailed performance data becomes available. This should includes data of relevance to both performance and overall capability;

e) Acceptance Baselines - established alongside the requirements baseline to control associated acceptance information including URD, SRD, ITEAP and TLMP, proposed evidence of compliance, procedures, tolerances and pass/fail criteria;

f) Support Baselines - to maintain data in the LSAR alongside the evolving Combat System implementation.

4.6.13.4.4 Each of the baselines should be reviewed at the point in the project review cycle at which they are first instantiated and thereafter audited at each formal review. The project review cycle provides the structure through which project baselines and change control processes are formally audited. The baselines provide a solid foundation for the impact assessment of engineering change requests through which change control can be effected. Each baseline represents a particular aspect of the design evolution and should be synchronised so that the design itself moves forward through the formal process of reviews.

4.6.13.5 Maintenance of Baseline Data

4.6.13.5.1 With the increasing quantity of information to be configured, it is necessary to provide an environment through which all data (including models and engineering information) can be effectively managed. Some form of electronic data management with recourse to hard copy where necessary, is an essential part of modern CM.


Unclassified 130



Unclassified 131

Systems Engineering Guide for Naval Combat Systems – In-Service Phase

5. In-Service Phase

5.1 Introduction



5.2 Purpose of the In-Service Phase

5.2.1 The purpose of the In-Service (IS) phase is to support and maintain the capability to fulfil operational requirements. This includes the upkeep and support of both the operational Combat System and the shore-based development, integration and training facilities. From the Combat System perspective the IS phase is ‘phased in’, with equipments entering the stage separately as they each achieve System Acceptance. System Acceptance of the Combat System as a whole coincides with the acceptance of all constituent subsystems/equipments and ensures that the support infrastructure is in place.

5.2.2 Through previous phases, as the design was implemented, the characteristics of integrated equipments will have determined the actual properties of the overall Combat System. Support considerations will have been addressed in earlier stages through the application of Integrated Logistic Support (ILS). In the IS phase, ILS becomes the dominant discipline as the support principles are put into practice.

COMBAT SYSTEMDESIGN MODIFICATION

IN-SERVICEUPKEEP

IN-SERVICEASSESSMENT

Requirements

Acquire Equipment

Analysis/Design

Acceptance

Integrate/Test

Support Infrastructure

SE Process

DPSS

SDSA

RE

The In-Service Support stage is characterised by the maintenance and repair of the operational Combat System through upkeep support.Data gathered from operational usage of the Combat System provides the basis for continued assessment of system performance. Thegeneric SE process provides a framework for the investigation of system characteristics, shortfalls and potential solutions. This may lead toimprovement or enhancement of the Combat System implementation so that it continues to meet its operational requirement.

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

DATA RECORDING & ANALYSISDEFECTS & DEFICIENCY REPORTSOPERATIONAL DEFICIENCY

REPORTS

SPARESFIXESENHANCEMENTS

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

Modelling, Analysis andAssessment

Figure 5.1 Combat System In-Service Phase

5.2.3 The baseline Combat System capability which has achieved System Acceptance must be not be allowed to degrade whilst IS. To this end, the Combat System must be supported through upkeep, by the provision of spares, fixes and enhancements, as necessary. IS assessment of the warship and Combat


Unclassified 132

System serves to identify shortfalls which may require remedial action, and even modification to the design. These themes are illustrated in Figure 5.1, which identifies the central IS activities as IS upkeep and IS assessment. These give rise to Combat System design modification depicted by the familiar ‘V’ development model of system implementation.

5.2.4 Pre-planned capability upgrade may be effected as part of an Incremental acquisition approach. In this approach, staged upgrades to the combat system are made as budgetary constraints allow, or military demand or threat evolution requires, and/or technology maturity permits. Design and Acceptance of the successive increments should be conducted in a manner which keeps the amount of system re-testing to a minimum.

5.2.5 In-Service Support (ISS) Activities

5.2.5.1 The realisation of ILS which manifests itself through the central ISS activities of IS upkeep, IS assessment and Combat System design modification is shown in the context of other Systems Engineering (SE) related activities, together with the operational IS Combat System in Figure 5.2.

5.2.5.2 Once the Combat System has entered service, the maintenance and assessment of the IS Combat System becomes the driving force for any potential enhancements. This means that the assessment of the implemented system is the reference from which other activities will be proposed and conducted. The basis for this assessment will be the suite of performance models and supporting documentation assembled within a through-life support infrastructure.

DATAACQUISITION

MAINTENANCE/ENHANCEMENT

TRIALS andACCEPTANCE

REMOVAL andDISPOSAL

Disposal by SaleSalvage For Spares

Destruction

IN-SERVICE SUPPORT


DOCUMENTATION




SYSTEM STUDIES

IntegratedCombat Systems

Shore Facilities, TrainersFOC, Follow-on Vessels

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

Through-LifeSupport Infrastructure

Key OutputsCase for future capability

Proposals forMLU/Enhancements

Learning From ExperienceReports

Project Archive

In-Service Combat SystemOPERATION

TrainingFacilities

ShoreFacilities

TestEquipment

Spares

Through-LifeSupport System

PerformanceModels

DesignDocumentation

Through-LifeData

IN-SERVICEUPKEEP

IN-SERVICEASSESSMENT

TRIALS andACCEPTANCE

SYSTEMDESIGN

MODIFICATION

Figure 5.2 In-Service Support Phase, Overall Process

5.2.5.3 Overarching the central ISS activities are the project management and SE (Technical) management functions. In keeping with previous sections, activities which support the central activities throughout the stage are depicted separately. These address the conduct of system-level studies, the production of documentation, specialist engineering discipline support and Configuration Management (CM).

5.2.5.4 The infrastructure and mechanisms required for support will have been considered during earlier stages and will already exist when the ISS stage is reached. It is this fact which is inherent within the ILS strategy which enables analysis of supportability throughout the Combat System’s life.


Unclassified 133

5.2.5.5 A Combat System’s component parts will usually consist of complex equipments, which are the subject of separate procurement programs as well as Commercial Off The Shelf (COTS) items, and items, which are already in service. Supportability of the complete system therefore demands a co-ordinated and structured approach with an infrastructure designed to minimise the cost of ownership of the system and ensure that it continues to meet its design intent.

5.2.6 System Acceptance, Support Handover and Operational Handover

5.2.6.1 There are two aspects of ISS: the initial support of the Combat System and equipments prior to ultimate acceptance of the Combat System; and the support of the Combat System after this point through the operational lifetime of the warship. How these are to be managed should be stated in the TLMP and ITEAP. Combat System Acceptance represents the point of transfer of responsibility of support from the Combat System procurement authority to the Combat System support authority and this can be a phased handover as illustrated in Figure 5.3.

Figure 5.3 Staged Handover of Support Responsibilities

5.2.6.2 In order to promote the application of SE principles throughout these latter stages, it is desirable that the System Acceptance of systems and equipments coincide with the delivery of all DLODs and the handover of the Combat System for operational use (e.g. under a Certificate of Clearance for Use (CCU)). This however is unrealistic for projects other than a new class of platform with predominantly new equipments. Nevertheless the CSM, in conjunction with Customer 2 should produce a capability roll-out plan to first deliver the capability required at ISD and then the remaining capability to achieve Full System Acceptance.

5.2.6.3 A pragmatic view must be taken in the light of the procurement regime and the relative priority of equipment and system programmes. For equipment developed as a fleet-wide fit, there is a balance to be sought between the needs of the Combat System programme and that of the equipment in its wider context.

5.2.6.4 Equipment programmes are often asynchronous with warship programmes and have to meet their own contractual milestones (including System Acceptance) which do not necessarily align with the Combat System Acceptance programme. Equipments may be fitted to different platform classes, the Combat Systems of which have conflicting System Acceptance programmes. Furthermore, Combat Systems are subject to ongoing development even when IS and the new equipment may be supplied by DPA and DLO.

The Combat System will comprise both new/modified (New-to-Service) equipments for which FWA is sought and equipmentswhich have already achieved FWA (In-Service). Combat System FWA can only follow FWA for each of the constituentequipments and handover from the Equipment Design Authority to the Equipment Support Authority therefore will be phasedover a period of time as each equipment achieves FWA.

COMBAT SYSTEM

IN-SERVICE EQUIPMENT

NEW-TO-SERVICE EQUIPMENT

ISS

ISS

ISS

ISSFWA

FWA

FWA

FWA

Combat SystemNWSTs/FWA

Trials

Combat SystemNWHTs/FWA

Trials

Equipment NWSTs/FWA TrialsEquipment NWHTs/FWA Trials



Equipment SATs

Equipment SATs

Equipment SATs

ISS

ISS

ISS

Equipment HATs

Equipment HATs

Equipment HATs

Please note : for FWA, read System Acceptance


Unclassified 134

5.3 Inputs

5.3.1 Inputs from the Previous Phase

5.3.1.1 The installed Combat System equipment is handed over to CINCFLEET at ISD with a CCU defining the limitations of operation, following System Acceptance. Responsibility for the support of the IS Combat System is also normally handed over at this time from the DPA to the DLO along with financial accounting responsibilities.

5.3.1.2 System Acceptance includes acceptance of all the separate DLODs against the URD and SRD in the manner previously established by the CSM and IPTL (in the ITEAP) and agreed with both Customers 1 and 2.

5.3.1.3 Supply support mechanisms developed with the involvement of the support authority during previous stages will be administered by the DLO. CM of Combat System equipment configurations and associated items listed above will be administered within the CM process of the whole warship. Additionally the Logistics Support Analysis Record (LSAR) developed during earlier stages, will be maintained to reflect actual support performance.

5.3.2 Relationship with the In-Service Combat System

5.3.2.1 Documentation raised as a result of the operation of the IS Combat System also serves as input to the activities performed within the IS stage. These include:

a) Defect and deficiency reports identifying deficiencies or failures considered to have resulted from defective design;

b) Operational defect reports identifying operational performance deficiencies which require urgent action.

5.3.2.2 Information exchange between onboard engineering databases and the support infrastructure also needs to be considered. This includes configuration data and statistical equipment performance data gathered on board which needs to be transferred into the UPKEEP system.

5.4 Outputs

5.4.1 Outputs in Support of In-Service Operation of the Combat System

5.4.1.1 Amongst the many output products of the IS stage, the single and most significant of these is the equipment modifications (hardware and/or software) and associated documentation provided to CINCFLEET to maintain the required level of operational capability.

5.4.2 IS Phase Outputs - to Removal and Disposal

5.4.2.1 Removal and Disposal will apply to all Combat System equipment and related items which have reached the end of their IS life whether due to declared obsolescence (including equipment removed from platforms undergoing refit) or through sale to another nation.

5.4.2.2 Depending upon whether the Combat System remains in use in other assets, disposal may also apply to hardware within the IS infrastructure particularly training facilities, SDF, test equipment and spares.

5.4.3 Key Outputs to Future Programmes

5.4.3.1 Additionally, there are the overall life cycle outputs, which will feed forward into future programmes. These include:

a) Case for future capability/proposals for Mid-Life Update (MLU)/enhancements, based on the ongoing evaluation of IS Combat System performance and effectiveness;


Unclassified 135

b) Learning from experience reports, identifying business areas and aspects of current engineering practice, which may benefit from future improvement.

5.4.3.2 As an invaluable source of design, implementation and operational data for future projects, the project archive and through-life support infrastructure needed to manage support data (i.e. Information Technology (IT) system equipment) must also be considered for retention/archiving (or disposal). Potential areas for consideration include:

a) Through-Life Support System(s) including LSAR, spares management systems, handbook information, electronic documentation;

b) Performance Models/analysis data and design documentation created during the design activities and maintained/augmented through the operational life of the Combat System in support of Post Design Support/Services (PDS) activities;

c) Through-Life Test and Trials Data defining the operating characteristics of the IS Combat System.

5.5 Organisation


The IS phase involves a number of organisations, which collectively fulfil the roles depicted in Figure 5.4. As before, the precise set of responsibilities and boundaries within the organisational structure depends on the prevailing interface between the DPA and DLO, and the level of responsibility devolved to industry.

5.5.2 Contractor Logistic Support

5.5.2.1 Contractor Logistic Support (CLS) is the concept of divesting responsibility for support of equipments to industry. This will provide incentive to the contractor to concentrate on supportability during the design stage and to achieve lower spares costs to the MoD by delaying the initial spares provision until a usage rate has been established. Some contractors may not have the experience to fully understand all the elements of supportability required by the RN and therefore care should be exercised when a CLS contract is being considered to ensure that the contractor is completely aware of its implications and of his responsibilities.

5.5.2.2 CLS has many implications for the organisational boundaries between DLO, (DPA) and industry during the IS stage. This accompanies a shift of MoD emphasis from direct involvement in IS activities (‘support provider’) towards a pro-active management role (‘support decider’). As a result, the activities conducted by the support organisation will move into the industry subcontract domain.

5.5.2.3 Nevertheless, the underlying process through which the IS phase is conducted remains largely unaffected such that boundaries of responsibility can be overlaid onto the process detailed in Part B.

5.6 In-Service Phase Process

5.6.1 Introduction

5.6.1.1 The ISS phase comprises a set of activities the prime objective of which is to support and maintain the system’s operational effectiveness. The process demonstrates that the Combat System continues to meet its design requirements through IS system assessment and monitoring in terms of material and operational performance; maintenance and defect repair activities; and identifies and plans the implementation of agreed changes.

5.6.2 In-Service Process

5.6.2.1 The activities undertaken during the IS Phase are depicted in Figure 5.4 which shows the core IS activities introduced in Part A in the context of related SE activities. Each of these areas is explored in detail in the clauses, which follow.


Unclassified 136

Figure 5.4 In-Service Support (ISS) Phase Activities

5.6.2.2 Feedback from IS operation to the support process is a vital factor in ensuring that any shortfalls identified from monitoring Combat System performance are fully addressed. System studies provide the modelling and analysis capability to support this aim.


5.6.3.1 MoD Role

5.6.3.1.1 Prior to embarking on IS activities, the Combat System Manager (CSM) should ensure that, for the duration of the IS Phase, there will be:

a) A clear statement of the division of responsibilities between the different MoD authorities involved in IS (e.g. CSM, CINCFLEET (PM and CL), Equipment Project Managers (EPMs) (DPA and DLO), MCTA, Acceptance Authority et al);

b) A clear statement of where contractor support is to be employed, their responsibilities, the rules under which the contractors will communicate with other parties involved (both MoD and other contractors).

5.6.3.2 Management Plans

5.6.3.2.1 There is a need to define the organisational interfaces both within MoD (DPA/DLO) and between MoD and industry where IS activities are contracted out. The management plan hierarchy described in Annex B provides the necessary controls. The adoption of suitable plans provides the required visibility of the management process throughout the organisational structure and need to accompany the project (i.e. transferred from DPA to DLO at an agreed point). The plans listed in Table 5.1 must reflect the fact that the scope of activities within the IS stage are significantly less-development oriented.

COMBAT SYSTEMDESIGN MODIFICATION

MODIFICATIONREQUIREMENTS

IMPACTANALYSIS

MODIFICATIONDEFINITION

IN-SERVICE SYSTEMASSESSMENT

Data Recording and Analysis

Defect and Deficiency Reporting

Performance Assessment

AR&M Monitoring

EMC

Signatures

Safety

IN-SERVICE UPKEEP

Test Equipment

Shore Facilities

Spares Provisioning

Maintenance and Repair

Training Co-ordination

Supply Support

Waterfront Support

PROJECT MANAGEMENT and Contract Management Risk Management Quality Management Project Management Systems Engineering Management Acceptance Reviews & Audits

TRIALS and ACCEPTANCE

ACCEPTANCE/ CERTIFICATION

RE-INTEGRATION AND TEST

COMBAT SYSTEM TRIALS

SYSTEM STUDIES Maintenance of Prototypes and Models System Change Investigation and Impact Studies Emergent Requirement Evaluation System AR&M Analysis System Operational Performance

DOCUMENTATION


A&A Installation Guidance Information Design Change Safety Case Handbooks Engineering Databases and Information Systems Records Archive and Library

AR&M EMC Operational Effectiveness Safety Security Signatures

CONFIGURATION MANAGEMENT Configuration Definition Configuration Control Configuration Accounting


MODIFICATION/A&AIMPLEMENTATION


Unclassified 137

Table 5.1 Project Through Life Management Plan (TLMP)

PM Discipline Topic(s) Priority Activities During the IS Phase

Contract Management

Contract Management Plan (CMP) Task Specification

A CMP should be produced and maintained by the CSM. This may call up a Task Specification listing Task Areas and deliverables. The CMP and Task Specification should be reviewed annually and amended if appropriate.

Risk Management Risk Management Plan (RMP), Risk Register, Risk Analysis, Risk Reduction Action Plan (RRAP)

Continued application of Risk Analysis methods, update of the Risk Register and raising of RRAPs.

Quality Management

Quality Plan Software Quality Plan

The quality plans address the requirements of Def Stan 05-91 and Def Stan 05-95 and First Level requirements of Def Stan 08-207.

Project Management

Through Life Management Plan (TLMP)

The TLMP should extend over the duration of the IS period and address all those issue in the SSE and relating to all aspects of IS Support.

Assurance Programme Control and Performance Reviews and Audits Project Review and Assurance

Development of interfaces with other planning systems to exchange planning information as necessary to ensure consistency between the plans.

Regular scheduled review of the programme(s) and progress.

Agreement and visibility of the Project Review Cycle; Action tracking.

Review of methods/tools support and training.

Configuration Management and Change Control Plan

Auditing of the Configuration Management process for controlling the effects of change to the system design and implementation baseline(s).

5.6.3.2.2 All MoD-specific plans will have been developed during previous stages and need to be revisited and amended where necessary to reflect the IS stage policy and constraints. Where major elements of the IS programme is contracted out to industry, appropriate management plans will need to be developed by the successful contractor.


Unclassified 138

5.6.3.3 Systems Engineering (SE) Management

5.6.3.3.1 Although the IS stage accompanies a significant reduction in the level and complexity of tasks associated with development and integration, there remains a need to co-ordinate and maintain ‘through life’ SE activities. The SEMP inherited from the previous stage should provide an adequate basis for co ordination of the SE process, tailored to reflect the reduced scope of activities to be conducted during the IS stage. The SEMP for the IS stage must address (either within itself or through lower-level plans as deemed appropriate):

a) AR&M analysis;

b) CM;

c) COTS support and upgrade strategy;

d) Defect analysis;

e) EMC;

f) ILS/LSA and continued population of the LSAR (subject to the ILS Plan);

g) Installation engineering;

h) Performance assessment;

i) Safety management (subject to a separate plan);

j) Signature management (subject to a separate plan co-ordinated through the platform project);

k) System modelling;

l) System modifications and documentation (including A&As);

m) Use and maintenance of shore-based facilities.

5.6.4 Integrated Logistic Support

5.6.4.1 Objectives of ILS during the IS Stage

5.6.4.1.1 ILS activity during the IS stage will be aimed at ensuring that:

a) The supportability recommendations are translated into support products, i.e. spares, manuals, training etc.;

b) The recommended support system is adequate to support the operational system, and is available in a timely manner;

c) The support system is optimised by using feedback from the field to fine tune the support infrastructure;

d) The identification of any deficiencies to allow the initiation of corrective action.

5.6.4.2 ILS/LSA Acceptance

5.6.4.2.1 Acceptance and validation of support aspects should be an integral part of the overall ship acceptance procedures. Evidence will be required to show that the support package is in place and to provide confidence that the Combat System and component equipments can be supported through life. Final acceptance may require demonstration of the effectiveness of the support measures.


Unclassified 139

5.6.4.3 LSAR Management

5.6.4.3.1 The policy for the management and maintenance of the LSAR will depend on the complexity of the project and the probability of changes in the future. The LSAR will be transferred to DLO as part of the transfer of responsibility for the Combat System equipment/platform. This will form part of the formal hand-over and validation process between DPA and DLO.

5.6.4.3.2 LSAR management also includes development of LSAR interfaces with other organisations and IT systems, with the collective objective of tracking WLCs.

5.6.4.4 In-service Failure Reporting and Analysis

5.6.4.4.1 A strategy for the level of IS failure reporting and analysis to be carried out will have been developed which take into consideration:

a) Complexity and/or novelty of the system;

b) Operational significance;

c) Safety significance;

d) Experience with other similar systems;

e) The need to prove reliability or other predictions.

5.6.4.4.2 Defect reporting supports trend analysis used to evaluate the success of the original upkeep policy, and any corrective measures taken. Upkeep policy should be reviewed regularly to ensure that the assumptions are still valid.

5.6.4.5 Availability, Reliability and Maintainability (AR&M)

5.6.4.5.1 The AR&M of equipment are a major consideration of the use of support resources and hence IS costs. The AR&M activities during the IS stage of the Combat Systems life cycle consist of monitoring the AR&M achieved in real field usage, taking corrective actions for problems identified and considering the effects of design or usage modifications on AR&M. Data reporting from the field usage can be used for:

a) Feedback to designers to identify and cure AR&M problems;

b) Provide data for IS AR&M trials or for the contractors warranty clause and for System Acceptance;

c) Provide feedback to, or, for support decisions (e.g. spares holdings) and for setting targets for future equipments.

5.6.4.6 Obsolescence and Disposal

5.6.4.6.1 The Combat System, equipments and platform will have been procured against a predicted, or required, IS life. This may be extended, or shortened, as overall defence policy dictates. Obsolescence must be anticipated and activities initiated in response to find a suitable replacement, or extend the life of equipments and component items.

5.6.5 In-service Upkeep

5.6.5.1 Upkeep Policy and Plans

5.6.5.1.1 Upkeep is the set of related activities, instigated by the support authority, necessary to sustain an IS system or equipment in a specified material state and/or at a specified level of operational performance. In the Combat System context, upkeep is defined by Combat System Upkeep Policy and Plans, which co-ordinate upkeep policy for component subsystems/equipments, by defining common system-level objectives as summarised in Table 5.2:


Unclassified 140

Table 5.2 Upkeep Policy

Policy Area Topics

Overall Upkeep Policy Equipment fault diagnosis including Built-In Test Equipment (BITE). Overall repair policy: Upkeep by Exchange versus Upkeep by Repair. Onboard repair. Onboard maintenance conducted through Planned Maintenance Schedules (PMS). Base repair/overhaul and requirements for removal whilst de-perming. Refitting dockyard support for repair, refurbishment, system proving and STW. Support of shore facilities. Support of shore-based training equipment including Command Team Trainer (CTT) and Maintainer trainers.

Upkeep Periods Definition of requirements for upkeep cycles of maintenance periods:

• Self-Maintenance Periods (SMPs) and Maintenance Periods 1, • Docking Assisted Maintenance Periods (DAMPS), Assisted Maintenance Periods (AMPs) and Maintenance Periods 2, • Docking Periods, Capability Update Periods (CUPS) and Refits and Maintenance Periods 3/4.

Test equipment and tools Common Range Electrical Test Equipment (CRETE). Calibration. Test Equipment for shore facilities.

Defects Defect/deficiency reporting and corrective action.

Configuration management Configuration management of IS Combat System configurations and modifications.

Software and firmware Software and firmware issue and control.

Logistic Support Spares provision, built-in spares, initial replenishment stocks and stowages. Packaging, handling and transportation.

5.6.5.1.2 Upkeep policy is developed and defined during the early stages of the life cycle to the point at which specific upkeep requirements supporting the policy can be mandated through contractual arrangements invoked for the development and production stage. Upkeep plans should be maintained during the IS stage, as a baseline of upkeep and support.

5.6.5.2 MoD Responsibilities

5.6.5.2.1 The DPA and DLO operate many joint working procedures to ensure the seamless transfer of projects from one organisation to the other and back again when required. These measures include the ‘dual-hatting’ of IPTs. The DLO usually has prime responsibility for ensuring the upkeep and support of platforms and their systems. Fleet operating patterns directly impact on the maintenance and repair periods available to Ships and Submarines. Under CSAs between the DLO and CINCFLEET the DLO, tailors its upkeep and support practices to the needs of the warships.

5.6.5.2.2 Combat System IS matters should be resolved by an upkeep support working party.

5.6.5.3 Upkeep Activities

5.6.5.3.1 The primary objective of upkeep is the planning and provision of manpower, material and facilities to support the Combat System through its life. This is administered through:

a) Combat System Upkeep Policy and Plans (as above);

b) Combat System Ship Weapons Upkeep Planning Information (SWUPI);


Unclassified 141

c) Test Equipment, Hand Tools and Stowages;

d) Spares ranging and scaling, evaluating the implications of introducing changes to the spares allowance and stock holdings for the Combat System identified from shortfalls in existing provisioning and allowances;

e) Maintenance Schedules, ensuring their suitability to maintain the declared Upkeep Policy;

f) Training Co-ordination;

g) Handbooks;

h) Master Record Index (MRI);

i) Establishment Lists reviewed and updated to ensure that they reflect all Combat System changes and contain the necessary information to support maintenance and defect rectification;

j) Defect Management (see also paragraphs on IS assessment);

k) Modification Control (see also paragraphs of Combat System design change management);

l) Ship Fit Configuration definition and control;

m) Software status control;

n) A&A sponsorship and support;

o) Design Authority Refit Requirements (DARR);

p) Weapon Equipment Delivery Programme (WEDP);

q) Post Maintenance Trials Planning;

r) Shore facilities supported both at system-level and for any equipment, which is unique to each facility including design support, so that each facility can be kept up-to-date and fully operational.

5.6.5.4 Supply Support

5.6.5.4.1 Integrated Supply Support Procedures (ISSP) are an integral part of the ILS process and are covered by Def Stan 00-60 Part 20-25. They cover the procedures for Initial Provisions (IP), Codification, Procurement Planning (PP), Order Administration (OA) and Invoicing, based upon the functional standard of AECMA 2000M and harmonised to integrate with the LSA and electronic documentation standards.

5.6.5.4.2 Current and future supply support will make use of UPKEEP, a major IT system which when complete will replace many of the current systems such as OASIS, CRISP, PROFILE 77 etc. The purpose of UPKEEP is to provide IT supports to the DLO with respect to upkeep and stores supply to the Royal Navy. It allows data to be shared, which means that information is recorded once only on UPKEEP. This information is then available throughout the MoD.

5.6.5.4.3 Configuration data provides the basis for most activities within the UPKEEP system. It will be used in maintenance, supply, provisioning and planning activities associated with building, refitting, maintaining and repairing vessels and weapons systems. In the configuration data control area, UPKEEP will hold configuration data for surface warships, submarines, Royal Fleet Auxiliaries (RFAs) and other support vessels, and for systems and equipment fitted at engineering and shore training establishments.

5.6.5.4.4 The provision of accurate configuration data, which is readily available in the DLO, enables maintenance work to be planned more accurately. This results in the more effective and efficient conduct of maintenance and upkeep. This may entail configuration of shore facilities to reflect the planned Combat System configuration and the proving of the update before being effected onboard.


Unclassified 142

5.6.5.5 Commercial Off The Shelf (COTS) Support

5.6.5.5.1 The major impact of COTS on the through-life support philosophy is the relatively short life expectancy of COTS products (between 1 and 5 years maximum), compared to the 25+ years IS life expectancy of the warship. The MoD must implement a shorter replacement cycle, which will significantly impact equipment turnover, training of personnel and the provision of spares.

5.6.5.5.2 Research in the US on the implications of introducing COTS hardware and software into military systems is being monitored by UK MoD through information exchange programmes such as MIEA 5929 (UK-RN-N-5929). Furthermore, a NATO/industry advisory group has already produced guidance as ANEP 54 ‘Guidance for COTS Insertion’ and which includes implications for life cycle support.

5.6.5.5.3 In the future, it is forecast that engineering change will be caused more by supportability issues excluding AR&M (i.e. items becoming obsolete), than by changing requirements. With commercial components:

a) Inter-operability is a particular problem - there is no guarantee that the replacement part or parts supplied from different vendors will work together;

b) Change is likely to occur without notice or notification. Upgrades are driven by commercial factors alone;

c) Manufacturers performance claims are not always accurate.

5.6.5.6 Waterfront Support

5.6.5.6.1 Support to ships and submarines at the waterfront by a competent, experienced and broadly based engineer is a key feature of improving availability. The provision of waterfront support is primarily intended to ensure that ships and submarines achieve and maintain the required degree of operational readiness and availability.

5.6.5.6.2 Of particular importance is the timely resolution of engineering problems encountered onboard. These problems can be either maintenance or design-related. The technical response to any such problems must include a method of ensuring that all of the class is informed of the problems. This will give maximum exposure within the operating community of problems arising that might affect other vessels, either currently or at some point in the future.

5.6.5.7 Further Information

5.6.5.7.1 Compatibility with the existing publications for upkeep should be maintained although the impact of ILS and accompanying tools may require tailoring of existing methods. Guidance should be sought from the AMS and any other sources of expertise within or outside the MoD.

5.6.6 In-service Assessment

5.6.6.1 Objectives

5.6.6.1.1 IS Assessment is the continuous assessment of the IS capability, performance, effectiveness and AR&M of the Combat System against the URD and SRD. This assessment process applies to both the warship(s) and shore facilities and may identify the need for improvements to the Combat System design in order to maintain or enhance the system design.

5.6.6.1.2 The baseline design of the Combat System will have previously been demonstrated during System Acceptance. Further assessment is necessary to evaluate performance through life and detect any degradation.

5.6.6.1.3 IS Assessment requires a close liaison with many MoD, RN and contractor IS support authorities/agencies with whom it exchanges the necessary data, and whose facilities and expertise are used for analysis and recording of data, including:

a) The ship or submarines for Combat System performance and effectiveness data;


Unclassified 143

b) CINCFLEET for equipment defect reports and data relating to availability, reliability and maintainability of Combat System equipments;

c) DPA whose facilities are used for the custody and amendment of system interface documentation, and for the custody, maintenance and operation of specialist Combat System models;

d) Acceptance and Accreditation Authorities and Advisors.

5.6.7 In-service Assessment –

5.6.7.1 MoD Role

5.6.7.1.1 The Combat System design authority should, via the Combat System management group, develop and maintain a register of Combat System problem areas which should contain:

a) A brief description of the problem;

b) Combat System implication statement in terms of consequent impact on other equipments and/or operational capability;

c) Actions taken to date including previous review and next review dates;

d) Possible further risk reduction actions;

e) Possible fall back options.

5.6.7.1.2 IS performance monitoring should continuously analyse equipment and system performance, defect and AR&M reports and other capability and performance data from sea, and reports and analysis from other facilities. This analysis should enable achieved performance to be established, compared with agreed performance characteristics and recommendations to be made to rectify under performance.

5.6.7.1.3 The Combat System design authority should keep under review developments in defect and performance reporting to ensure compatibility with the Combat System philosophy and requirements for support with the aim of minimising demands on ship’s staff for gathering performance data.

5.6.7.2 Defect and Deficiency Reporting

5.6.7.2.1 Defect Analysis is required to review Combat System and equipment defect reports and identifies remedial action. This demands liaison with all relevant authorities to ensure that work is progressed satisfactorily or, when system-level issues arise, conduct the management activities leading to the initiation of remedial action and the clearance of the defect.

5.6.7.2.2 A Defect and Deficiency Report (Form ERC/S2022) may be raised at any time during the life of a platform by parties operating, maintaining or supporting the Combat System to report a deficiency or failure of the Combat System equipment that is considered to have resulted from defective design.

5.6.7.2.3 When action in response to a Defect Report has implications or remedy beyond an equipment boundary and affects Combat System capability, or raises system level issues the remedial activity may include the introduction of an authorised design change by modification or As&As.

5.6.7.3 Operational Defect Reports (OPDEFS)

5.6.7.3.1 OPDEFS are defect reports of operational performance deficiencies. OPDEFs require an urgent response and take priority over S2022s and other technical queries. In due course, an S2022 to support the OPDEF will be required to be raised by the originator of the OPDEF. This will often occur after the OPDEF has received a response.

5.6.7.4 Performance Assessment

5.6.7.4.1 Performance Assessment establishes the baseline performance of the Combat System, building on the evaluations leading to System Acceptance. Data from operational ships, submarines, shore facilities and


Unclassified 144

operational/tactical analysis facilities, should be analysed to assess Combat System performance against the designed capability. The purpose of this assessment is to identify:

a) Shortcomings in achievement due to the design, upkeep or support;

b) Limitations of the design;

c) Changes in the operational requirement requiring changes to the designed capability.

5.6.7.4.2 Combat System IS performance assessment should be reported at regular intervals, drawing together evidence supporting the case for change. This should be provided against an identified shortfall and include practical proposals for design change.

5.6.7.4.3 Combat System performance data useful for operational performance analysis includes:

a) OPDEFs;

b) ERC/S2022s;

c) Fleet Trial reports;

d) Minor Trial reports;

e) MCTA reports;

f) Index reports;

g) Acoustic Signature reports;

h) Magnetic Signature reports;

i) Analysis of SAT results;

j) Certification and Re-certification trials;

k) Patrol Analysis reports;

l) Infra-red signature reports.

5.6.7.5 Availability, Reliability and Maintainability Monitoring

5.6.7.5.1 During the IS phase there will be a requirement to form an IS AR&M panel to quantify IS reliability, identify factors inhibiting reliability and recommend appropriate measures to improve AR&M.

5.6.7.6 Electromagnetic Environmental Effects (E3)

5.6.7.6.1 There is a through-life requirement to identify all EMC problems (including the potential effects of Electromagnetic Pulse (EMP)/Ionising Nuclear Radiation (INR), and TEMPEST) which adversely affect the capability of the Combat System. This involves management of the Combat System electromagnetic environment, investigation of reported deficiencies in design or practice as they affect Combat System performance, and the proposal and implementation of change where justified. Further advice is available from the MoD Technical Authority for all E3 issues: DCSA Test & Reference Branch, Blandford, Dorset.

5.6.7.7 Signature Management

5.6.7.7.1 Liaison is required with the operational authority in the co-ordination of all aspects of the management of the warship signature. This will include:

a) Ensuring that changes do not adversely affect the warship signature;


Unclassified 145

b) Use of data from signature ranging reports and patrol reports for signature reduction or removal of defects;

c) Set up of a magnetic safety clearance register for the Combat System to support a (submarine) de-perm when requested;

d) Management of approved signature measurement or ranging activities, utilising existing MoD sponsored facilities and authorities wherever possible.

5.6.7.8 Safety Management

5.6.7.8.1 All safety issues affecting the IS Combat System, must be addressed through the MoD Ship Safety Management System (SSMS) through the following activities:

a) Review and update the Safety Management Handbook (SMH);

b) Maintain the Combat System safety live file and hazard log, and provide updates to the baseline safety case, and platform safety certificate;

c) Assess the safety aspects of all proposed A&As, and equipment modifications for identification of potential hazards;

d) Assess the safety status of each platform by monitoring S2022s, OPDEFs etc including proposals for rectifying identified safety shortcomings.

5.6.7.8.2 Any limitations necessary to ensure safe operations of the Combat Systems must be specified and promulgated in Certificates of Clearance for Use (CCU), and trials or operations which could be outside the design standard must be identified and appropriate formal safety advice provided.

5.6.7.9 Obsolescence

5.6.7.9.1 Obsolescence, be it that of an engineering component or an equipment is a through life issue which the Combat System design authority is responsible for addressing. The equipment sponsor (i.e. EPM) will be responsible for overcoming the obsolescence of a component part of his equipment responsibility. This will usually be a function of a support contract with the equipment supplier.

5.6.7.9.2 Often it will be possible to effect component replacement which will not affect the function, form or fit of an item. In some cases, these problems may be overcome by ‘life-time buys’ of components. Some equipment will be an amalgam of COTS and MoD-developed items where the intellectual property rights are owned jointly and sometimes where an item is wholly MoD developed. In each case the design authority will be in the forefront of addressing any obsolescence aspects.

5.6.7.10 Combat System Design Modification - Objectives

5.6.7.10.1 There will be many cases where, as a result of defects or deficiencies identified, a change to the Combat System equipment is required. These may result in modifications necessary to address design shortfalls or assuage the effects of obsolescence. Modifications may also be required to apply improvements to designs instigated by the equipment supplier (e.g. COTS upgrades). Modifications must not be used to be upgrade or modify capability required by the URD without DEC approval.

5.6.7.10.2 Proposed modifications to an individual Combat System equipment or sub-system will need to be reviewed in the context of the Combat System requirement to assess feasibility, impact and justify the benefits. This will result in the proposal being managed as a Combat System modification or A&A. Other circumstances, where mission-specific (e.g. submarine special fits) and area-specific equipment are required, should also be addressed in a comparable manner.


Unclassified 146

SDF/SIF HATs SATs

MODIFICATIONREQUIREMENTS

MODIFICATION/A&AIMPLEMENTATION

IMPACTANALYSIS

MODIFICATIONDEFINITION

COMBAT SYSTEMRE-TEST & TRIALS

EQUIPMENTRE-TEST & TRIALS

SYSTEMRE-INTEGRATION

SE Process

SS DP

SDSA

RE

CCB Approval

Existing EquipmentDesign

In-ServiceAssessments

Figure 5.5 Combat System Design Modification

5.6.7.10.3 Modification to the Combat System baseline should be co-ordinated using a structured approach, as illustrated in Figure 5.5, addressing pertinent issues through the following sequence:

a) Definition of the proposed modification including requirement, design impact supported by analysis/investigation, proposed implementation, risk and funding issues;

b) Review and approval to proceed through the Change Control Board (CCB);

c) Initiation of procurement action for any hardware, software and firmware required to implement a system modification once it has received approval;

d) Arrangements for the implementation, integration, trials and acceptance of Combat System modifications and A&As;

e) Co-ordination of modification and A&A guidance information packages.

5.6.7.10.4 Modifications and A&As will be tracked by means of a register of all proposed changes to the Combat System to be reviewed and published at regular intervals. This should be supported by a plan identifying the progress of Combat System modifications and A&As.


Unclassified 147

5.6.7.11 Combat System Design Modification Activities

5.6.7.11.1 Combat System modification typified by component equipment changes need to be managed by the design authority through the Combat System Management Group according to a disciplined systems engineering process as illustrated in Figure 5.5 and described in detail in previous paragraphs. The focus of activities during the IS Phase is to define the modification, assess its impact and once agreed, oversee its implementation, integration, testing and trials. This is achieved through the following activities:

a) Analysis and design activities defining Modification Requirements, performing Impact Analysis and Modification Definition to investigate the effects of fitting and integrating modified equipments, the interaction between them and the effect on the overall Combat System. This will involve investigation of AR&M, operability, performance, vulnerability, environmental factors, mutual interference, RADHAZ, and TEMPEST/EMC. Overall Combat System alignment, blind arcs, personnel safety, and firing arcs will also be addressed. The activities include identifying: the total cost of the changes to all affected equipments; the cost of their integration; and a programme to ensure the orderly introduction of the modification to the fleet;

b) Following CCB approval for Modification Implementation, the modification will be implemented by the Equipment Supplier, under the direction of the EPM;

c) Integration, test and trials will follow through Equipment Re-test and Trials, System Re-integration and Combat System Re-test and Trials to confirm correct operation in Shore facilities (SDF/SIF) and onboard the warship, under the direction of the CSM.

5.6.7.12 Support Services

5.6.7.12.1 Combat System and equipment modification is achieved through support contracts placed on industry, typically the System and Equipment Suppliers. Support contracts should cover the activities necessary to implement modifications needed to rectify design shortfalls, following equipment System Acceptance. Support activities serve to maintain the Combat System design in support of the overall operational requirement and are not intended to aid the development of future enhancements.

5.6.7.12.2 The provision of Post Design Services (PDS) services must cover all aspects of weapon system installation engineering. For platforms that are having new equipments fitted, the PDS contractor (i.e. equipment supplier) will assume responsibility for the following:

a) Attendance at line-out inspections;

b) Resolution of discrepancies between Level 3 guidance information and ‘as fitted’ state of warship with subsequent approval of deviations from design intent;

c) Attendance at Installations Inspections (II);

d) Resolution of MCTA Part ‘B’ II pick-up items;

e) Liaison with EPMs to resolve any shortfall/problems with Government Furnished Equipment (GFE).

5.6.7.12.3 For platforms in service, the PDS contractor will assume the following responsibilities:

a) Response to defects raised through S2022s requiring platform attention;

b) Maintenance of the warship/Combat System configuration including equipment fit/datum pack, drawings and updates to handbooks;

c) Response to non-programmed works necessitating the production of Statements Of Work (SOW) or Level 4 drawings.


Unclassified 148

5.6.7.13 Guidance Information

5.6.7.13.1 Guidance information accompanying the modification must address the following aspects:

a) Interdependencies on other equipments being fitted;

b) Equipment availability against planned fitting opportunity;

c) Provision of ship-fitting aspects of the Installation Guidance Package (IGP);

d) Availability of ships services;

e) Mutual interference aspects;

f) EMC and EMP;

g) TEMPEST;

h) Siting of equipment to provide optimum possible firing or transmission/reception arcs, shock clearance, maintenance access space taking into account other existing and any other known new equipment being fitted concurrently or in the future;

i) Safety requirements of JSP 430 covering design and installation.

5.6.7.14 Planning and Financial Aspects

a) In parallel with the above technical aspects of PDS, the following planning and financial aspects also need to be addressed to ensure that the equipments can be fitted during update periods:

b) Timescales for installation, Setting To Work (STW), linking of respective equipments, HAT(E), SAT(E) for equipments;

c) Estimated costs of installation together with contractor supplied items;

d) Approval of A&As (i.a.w. BR 8593(17));

e) Programme of Level 3 guidance information must be agreed with the platform upkeep manager to enable:

1) The refit specification to be produced,

2) Issue of ITT for the production of Level 4 drawings (where required),

3) Delivery to selected contractor for the production of Level 4, full installation drawings,

4) Procurement of any long lead items of equipment to be addressed.

5.6.7.15 Handbooks

5.6.7.15.1 System, User and Maintainer Handbooks must be updated on completion of a refit.

5.6.7.15.2 Trials and Acceptance.


Unclassified 149

5.6.7.16 System Design Change Validation

5.6.7.16.1 Following implementation of a design change, validation will comprise integration, tests, trials, and acceptance certification. Test and trials provide the means to establish the capability of the Combat System and the analysis of trial results will provide evidence for acceptance/certification as well as indications of potential shortfalls requiring attention.

5.6.7.16.2 The conduct and presentation of system trials is co-ordinated by the Combat System Management.

5.6.7.17 Trials Co-ordination and Management

5.6.7.17.1 Maintenance of HAT and SAT policy may be assigned to the Combat System Manager who will co-ordinate working groups to review policy and co-ordinate technical support. Trials-related tasks include:

a) Management and supervision of system integration tasks and trials, acceptance trials, operability assessment trials;

b) Co-ordination of fleet, minor or other trials, and examination of their content for impact on the Combat System;

c) Preparation and distribution of trials schedules;

d) Pre-trial briefings to ships staff;

e) Provision of trials equipment;

f) Analysis and issue of trials reports;

g) Preparation and presentation of acceptance information;

h) Compilation of a trials register identifying proposed trials, trials in progress and those awaiting a final report;

i) Assessment of Combat System trials reports to determine requirements for equipment or system improvements;

j) Initiation of the improvement action as a result of identified shortfalls.

5.6.7.18 Documentation

5.6.7.18.1 Table 5.3 details the Combat System design and upkeep documentation required for IS. Design and upkeep documentation needs to be maintained throughout the stage, reflecting any changes made to the baseline Combat Systems IS and their support arrangements.


Unclassified 150



Design TLMP and SEMP System Master Record Index Test Evaluation and Acceptance Criteria Design Platform Contents List Design System Interface Diagram Design Interface Reference List System Integration Specification System Trials Specification Equipment List Acceptance Policy Paper Acceptance Management Plan System Integration & Trials RequirementsPlatform Contents List System Interface Diagram Interface Reference List System Link Diagrams

Developed by Combat System Management Group (with support from Equipment Suppliers)

Upkeep Upkeep Planning Information Combat System Handbook Weapon Equipment Schedule Weapon Equipment Delivery Programme Weapon Equipment Schedule SDF/SIF Weapon Upkeep Plan SDF/SIF Operating Procedures

Maintained by Combat System Management Group


5.6.8.1 Specialist Discipline Support necessary for the IS phase centres upon the major areas of activity:

a) Support-specific activities relating to upkeep. This is traditionally the ‘core business’ of DLO and the necessary skills should be readily available;

b) IS assessment activities require additional expertise in support of AR&M, effectiveness/operational performance assessment, EMC, signatures and safety;

c) Modification activities necessitating recourse to analysis/design activities. These should be conducted to the scope determined appropriate to the project needs;

d) Integration, Test and Trials activities in support of verification and validation. These should be conducted to the scope determined appropriate to the project needs.


5.6.9.1 Objectives

5.6.9.1.1 System Studies continue throughout the Combat System life cycle refining models and validating their predictions by comparison with real operational data. This is an essential aspect of IS Combat Systems engineering conducted by the Combat System design authority, with responsibility for specific study tasks devolved to the Combat System management group.

5.6.9.1.2 During the IS stage, it is important that the through-life models are consulted to investigate impact of potential change and maintained thereafter to reflect modifications to the baseline Combat System. The main objectives of modelling in the IS Phase is to use and evolve models generated from earlier activities to:

a) Refine model fidelity using data generated from operational usage;


Unclassified 151

b) Assess the impact of any shortfalls or improvements in system design performance;

c) Support investigation and problem solving to remedy defects and deficiencies;

d) Support analysis of test and trials results;

e) Investigate the impact of emergent requirements on the Combat System implementation.

5.6.9.2 Model Maintenance and Validation

Voluminous data recorded during operational use needs to be subject to data reduction and analysis (as described in Section 7) to evaluate key performance parameters i.e. Measures Of Performance (MOPs) and Measures Of Effectiveness (MOEs). Data should be compared with modelling predictions to validate or improve the fidelity of models for through-life use.

5.6.9.3 Change Impact Investigation

5.6.9.3.1 Where specific design modifications options are required, technical investigations using modelling techniques should be used to examine the impact on the baseline Combat System. The Combat System design authority should determine scope of investigations to assess the impact of the change on:

a) AR&M;

b) Combat System architecture;

c) Functional compatibility;

d) Effectiveness;

e) Human Factors (HF)/operability;

f) Performance;

g) Platform installation;

h) Support;

i) WLCs.

5.6.9.3.2 The studies will assist in specification of the change and in the production of any A&A guidance information that is required as a result.

5.6.9.3.3 The CSM should ensure that existing MoD modelling facilities should be kept under review and developed or added to as necessary to satisfy the Combat System requirement.

5.6.9.4 Emergent Requirements

5.6.9.4.1 During the IS lifetime of a Combat System, new or additional requirements may emerge. The respective DECs must put in place the tasking arrangements to ensure that the impact of their new capabilities requirements are fully evaluated and costed for all the potentially affected projects. In this event, the Combat System manager and/or his IS contractor should consult with EPMs to provide assurance that an equipment or subsystem may be added without degradation to the Combat System. The procedures laid down in SSP 38 (being re-issued as a ‘Maritime Acquisition Publication) (and Def Stan 21-88 for Surface Ships) should be consulted.

5.6.9.5 Combat System Trials Support

The data derived from Combat System trials can enable the Combat System design authority to establish or improve the designed capability of the system. Combat System trials schedules should be monitored against emergent requirements and Combat System modifications to ensure that the system trial remains a valid demonstration of Combat System suitability for use at sea, and adequately tests any new functions or


Unclassified 152

modifications. The results from trials including HATs and SATs should be assessed against design statements and the results obtained from other platforms and the SDF to confirm system performance.


5.6.10.1 Objectives

5.6.10.1.1 CM is the process through which the design standard of each system, sub-system and equipment is defined, recorded, and related to the Combat System and ship fit definition/build standard of each platform or facility. CM ensures that, throughout the life of a system or equipment, all aspects of design change are monitored, recorded, and controlled. The primary objectives are as follows:

a) Define the baseline configuration of the system and its constituent equipments;

b) Apply technical and administrative direction to the control of changes to the configuration of the system throughout its life;

c) Maintain the configuration definition and undertake periodic audits to verify its accuracy.

5.6.10.1.2 Combat System CM is a pan-organisation operation involving both the DPA and DLO and, increasingly, Industry. Responsibility for implementing and conducting CM falls to the Combat System Management Group.

5.6.10.1.3 The process includes the auditing of the Combat System configuration throughout its operational life. This supports tracking of the latest status of Combat System design changes from initial identification through approval to their embodiment on the warship.


5.6.11.1 Configuration Management Activities

5.6.11.1.1 During the IS stage, CM activities include:

a) Maintenance of a register of all Combat System modifications (and waivers);

b) Control of the ship fit definition of the Combat System as installed in each platform and shore facility reflecting all authorised changes to equipment hardware, software or firmware;

c) Maintenance of Combat System design and upkeep documentation;

d) Maintenance of Combat System link, interface and test documentation;

e) Demonstration through audits that the Combat System CM system is operating satisfactorily to maintain stringent control of the configuration of the Combat System in each platform or shore facility.


Unclassified 153

Systems Engineering Guide for Naval Combat Systems - Termination or Disposal Phase

6 Termination Or Disposal Phase

6.1 Introduction



6.2 Purpose of the Termination or Disposal Phase

6.2.1 Overall Objectives

6.2.1.1 The AMS states that the objective of this project phase is to “dispose of the capability cost effectively, safely and in accordance with the best practicable environmental option”. This Def Stan considers this final stage of the Combat System life cycle i.e. Combat System’s removal from RN service and subsequent disposal. The overall process is illustrated in Figure 6.1 which shows the possible options: sale to another nation (whole warship disposal by sale) or ‘paying off’ (i.e. decommissioning and de-storing) followed by the dismantling and disposal of salvaged items. Both the operational platform and the support infrastructure maintained during its in-service lifetime will be subject to either or both of these processes.

Item Recycling/Destruction

Item Salvagefor Spares

Item Disposalby Sale

Operation ByForeign Navy


Training Facilities,Shore Facilities, TestEquipment, Spares,

Through-LifeSupport System,

Capability/PerformanceModels, Design

Documentation, Through-Life Data


FOC and FON W arships

Key OutputsLearning From Experience ReportsProject Archive

WHOLE WARSHIPDISPOSAL by SALE

PAYING-OFF, DISMANTLING,RE-USE and DISPOSAL

SYSTEMS ENG INEERING (TECHNICAL)and PROJECT MANAGEMENT

COMBATSYSTEM

DEFINIT IONTRIALS and

ACCEPTANCE

REMOVAL &EVALUATION

FINALASSESSMENT

FAST TRACKSYSTEMS

ENGINEERING

AND/OR

COMBATSYSTEM

RE-ENGINEERING



MAINTAIN and ARCHIVE RECORDS

Foreign SupportOrganisation

SYSTEMS ENG INEERING (TECHNICAL)and PROJECT MANAGEMENT

Figure 6.1 Removal and Disposal, Overall Process

6.2.1.2 The major objectives of this final stage in the Combat System life cycle are to:


Unclassified 154

a) Remove the warship from service in a cost-effective manner which preserves all relevant information for use in other and future Combat System projects;

b) Dispose of physical assets at whole warship, major system or component level through sale, salvaging (component re-use following dismantling) or scrapping (for re-cycling where practical, and by destruction or discarding otherwise) as economically as possible whilst meeting safety, security and environmental requirements.

6.2.1.3 Whole Warship Disposal by Sale

6.2.1.3.1 The drive to realise the export potential of decommissioned RN vessels, means that whole warship disposal by sale is the most common route for redeployment of Combat System equipment. It is likely that this route will bring with it many additional requirements (e.g. for improved customer-specified equipment fits, removal/sanitisation of sensitive data/equipment etc.) and these must be fully defined, the consequent modifications implemented, trialled and accepted before transfer of ownership can occur.

6.2.1.3.2 It is usually the preferred option with major warship sales, to achieve a government-to-government ‘hot’ sale with uniform-to-uniform handover. However, a contractor may be used by MoD to assist in the preparation of warships for overseas sales. Given the typically condensed time-scales which sales of this nature generally demand, the re-engineering process must be accomplished through a set of ‘fast track’ System Engineering (SE) activities as agreed by the customer. The re-engineering task is seldom trivial and the adoption of a formal SE process is strongly advised in order to maintain and optimise the performance of the modified Combat System and its support infrastructure. The process, which should be explicitly defined by the project Systems Engineering Management Plan (SEMP), must strike an appropriate balance of Combat Systems engineering activities (as defined in previous sections) within the constraints of the project.

6.2.1.4 Removal and Re-use or Disposal

6.2.1.4.1 Items removed through re-engineering as a result of whole warship disposal by sale will be subject to re-use and/or disposal. Disposal also follows the platform or installed equipment reaching the end of its operational life to be dismantled into components for sale, salvage for spares or destruction. It is not unusual for warship and Combat System disposals to take place whilst the Combat System remains in active service elsewhere in the fleet. Other notable circumstances which result in disposal are:

a) Major upgrade of capability (i.e. refit) as typified by a Mid Life Update (MLU) in response to changes in perceived threat;

b) Replacement of the Combat System;

c) Modifications to reduce or improve capability as a result of political decisions;

d) Commercial Off The Shelf (COTS) hardware updates;

e) Modifications initiated to address component obsolescence;

f) Removal from service due to high cost of ownership.

6.2.1.4.2 Where the cost of refit or refurbishment is prohibitive, a platform will be subjected to ‘paying off’ which involves decommissioning and de-storing. De-storing is where all stores, victuals, expendable weapons and other consumables are removed from the warship and returned to naval stores (as is also the case with major refits).

6.2.1.4.3 This is followed by the removal of all equipment and associated fixtures and fittings and the evaluation of items for sale, salvage for spares or destruction. Evaluation is conducted through the parameters of environmental issues, safety and security which together influence the practical route for disposal. This provides a mechanism for characterising and certifying all items and ensuring that disposal is conducted in an appropriate and accountable manner.


Unclassified 155

6.2.1.5 In-service Records

6.2.1.5.1 All relevant in-service data, historical log data and final system assessment data (relating to the system capability at service exit) should be recorded, analysed and archived. This data will provide a baseline against which characteristics of future systems can be compared. Known system strengths and weaknesses together with summaries of key data should be recorded to provide a mechanism for feeding back service experience to current and future Combat System projects.

6.3 Inputs

6.3.1 Removal and disposal applies to all Combat System equipment and related items which have reached the end of their in-service life whether due to declared obsolescence (including equipment removed from platforms undergoing refit) or through sale to another nation.

6.3.2 Disposal may also apply, depending on prevailing circumstances (accounting for whether or not the Combat System equipment remains in use in other assets), to hardware within the through-life support infrastructure particularly training facilities, Shore Development/Shore Integration Facilities (SDF/SIF), reference equipment sets, test equipment and spares. Supporting information and the support infrastructure needed to manage support data (e.g. Information Technology (IT) system equipment) must also be considered for retention/archiving. Potential areas for consideration include:

a) Support system(s) including Logistic Support Analysis Record (LSAR), spares management systems, handbook information, electronic documentation;

b) Performance models/analysis data and design documentation created during the design activities and maintained/augmented through the operational life of the Combat System in support of in-services support activities;

c) Through-life test and trials data defining the operating characteristics of the in-service Combat System.

6.4 Outputs

6.4.1 Outputs from the removal and disposal stage comprise the items for re-use or destruction and associated certification:

a) Final system assessments including Availability, Reliability and Maintainability (AR&M) evaluation, final LSAR, final Life Cycle Costing (WLC) and concluding operational performance studies and safety case;

b) General, electrical and mechanical equipment items and associated operating/maintenance documentation where available (i.e. salvaged items);

c) Software;

d) Consumable items such as magnetic media and paper;

e) Ammunition and explosives;

f) Certificates for re-use, salvage as spares or destruction.

6.4.2 Additionally, there are the overall life cycle outputs collected together in the project archive which will feed forward into future programmes. These include:

a) Learning from experience reports, identifying business areas and aspects of current engineering practice which may benefit from future improvement;


c) Design specifications and performance analysis models and data;


Unclassified 156

d) Test and trials data defining the operating characteristics of the in-service Combat System.

6.5 Organisation

6.5.1 In cases where the decision for whole warship disposal by sale is taken, the platform will be formally handed over by the operating authority to the Defence Export Sales Organisation (DESO) and their agents for disposal. The actual arrangements for disposal will depend on the precise circumstances and scope of the disposal activities. The general roles for this and other cases, where equipments are to be removed and disposed of, are illustrated in Figure 6.2.

CUSTOMERORGANISATION

DISPOSAL SALESORGANISATION


DESIGNAUTHORITY

SUPPORTAUTHORITY

OPERATINGAUTHORITY

EQUIPMENTSUPPLIER

COMBAT SYSTEM RE-ENGINEERING

DISMANTLING, RE-USEand DISPOSAL

SALVAGECONTRACTOR

Figure 6.2 Removal and Disposal Organisation Relationships

6.5.2 The following organisations will generally be involved:

a) Operating Authority (CINCFLEET) responsible for defined state of repair and co-ordinating the handover of the platform to DESO;

b) Support Authority, responsible for co-ordinating preparatory work (excluding normal maintenance and upkeep) in support of the sale. This includes defining the content of the package, exclusions and removals. Post their final non-operational date, warships are considered to be DLO assets and management responsibility rests with DLO. Additionally, the DLO will control handover of in-service support information and systems;

c) Design Authority, usually the Equipment supplier engaged by the DPA during procurement phases. The removal activities will be conducted by an industrial salvage contractor. Modification will generally be handled through a ‘prime contract’ arrangement, with the lead contractor liasing with subcontractors tasked with provision of items for integration, test and trials. DPA will also be responsible for certification of items for disposal for sale, salvage for spares or destruction. Furthermore DPA will be responsible for the collation and maintenance of appropriate life cycle records of the Combat System;

d) Combat System Integrator, typically industry responsible for managing the risk associated with modifications acquired via Equipment Suppliers, and bringing together equipment components into a working Combat System through integration, test and trials activities;

e) Disposal Sales Organisation, that is the DESO, Disposal Sales Agency (DSA) and its UK industry contractors for co-ordinating the disposal and sale of naval spares. For whole warship disposal, DESO will act on behalf of the UK government, interfacing with the foreign government Customer Organisation;


Unclassified 157

f) Salvage Contractor for disassembling equipment and identifying disposal routes (elements of this activity may be performed by naval bases/dockyards).

6.5.3 There may be exceptional cases where the intended transfer of responsibilities and information i.e. between industry, DPA and DLO has not been completed prior to the decision to initiate whole warship disposal by sale. In such cases, co-ordination of the maintenance and care tasks should be retained within DPA under the responsibility of the designated warship project manager.

6.5.4 Whole Warship Disposal by Sale - Background

6.5.4.1 Decommissioning and Disposal By Sale

6.5.4.1.1 The decision to dispose of a whole warship as a working entity ultimately rests with Secretary of State for Defence in consultation with the Cabinet. Options to refit, refurbish or sell on will be presented by Consultative Defence Committees as part of the wider UK Defence Policy. Naturally there are restrictions on the export of defence materiel. Sales are limited to NATO allies and other ‘friendly’ countries dependent on the nature of the equipment, its classification, the end-use nation intended usage, political situation/background and industrial factors.

6.5.4.1.2 There may be cases where a formal decision on the sale of a warship is deferred. Such cases would result in decommissioning of the warship and placing in a reserve/standby (‘moth-balled’) state from which it may be brought back to an operational state at some future date. Return to an operational state for RN use would have to be managed in the same way as preparing a warship for sale. This would demand a formal process to determine the material state of the system/equipment and identify any repair/replacement work which may be required. Subsequent re-engineering of the Combat System would culminate in formal acceptance trials.

6.5.4.2 Custody Care and Maintenance

6.5.4.2.1 In exceptional cases (e.g. Upholder class submarines), whole warship disposal by sale may require custody, care and maintenance contracts placed on industry to maintain the platform and support infrastructure (in particular the shore facilities) whilst DESO follows up sales opportunities. It is likely that such contracts will be administered through the DPA who will be responsible for ensuring that the complete package is maintained in a defined and configured state.


6.5.5.1 The nature and scope of the sales agreement negotiated by DESO will determine the extent of the project and thereby the level to which management plans are necessary. Each area of the management plan hierarchy should be addressed to a level appropriate to the project scope:

a) Contract;

b) Risk;

c) Quality;

d) Project Management;

e) SE Management and lower-level plan topics where deemed necessary;

f) Subcontract Management.

6.5.6 Combat System Definition

6.5.6.1 Sanitisation

6.5.6.1.1 Sanitisation of the Combat System is inevitably a major issue. All items of a sensitive nature (whether due to UK policy or policy covering items acquired from other nations) must be identified and appraised against regulations constraining their sale to potential customers. In cases where regulated items


Unclassified 158

make a significant contribution to the capability of the platform (e.g. primary sensor equipment), replacement items will need to be incorporated and/or modifications applied to existing designs.

6.5.6.2 Specification

6.5.6.2.1 The sales agreement negotiated by DESO must call up a suite of documents defining requirements, functionality and performance of the Combat System as a basis for acceptance. These should include as a minimum:

a) Combat System Requirements baseline and acceptance criteria;

b) Combat System Specification;

c) Subsystem Specifications;

d) Coherence Specification;

e) Interface Documentation;

f) Integration, Test and Trials Documentation;

g) Safety Case.

6.5.7 Combat Systems Re-Engineering

6.5.7.1 Modifications

6.5.7.1.1 Where modified, replacement or new equipment is required to be fitted, it is suggested that a formal SE process, documented by a SEMP, is followed to manage what may be regarded as a further iteration of the system development and integration processes. As a minimum, the SEMP must define the overall SE process to be used to specify and co-ordinate the following:

a) Combat System functional and performance requirements;

b) System Design Modification(s) including equipments, interfaces and installation aspects;

c) Integration, test and trials coverage;

d) Operational documentation;

e) Support infrastructure.

6.5.7.1.2 The level to which formal development is invoked must be at the discretion of the project manager in order to realise the overall requirement and ensure that the impact of changing equipment items (both in terms of the Combat System and the platform) is fully appreciated by the customer. Further information on system development can be found in Demonstration and Manufacture and Integration, Test. Evaluation and Acceptance.

6.5.7.1.3 The suite of information accompanying through-life system studies (evolved models, prototypes, data and analyses) should be used as a basis for the ‘fast track’ SE activities, in support of impact analysis, design specification and implementation. Consideration should be given for the inclusion of through-life system study data with the sale.

6.5.7.2 Support Infrastructure

6.4.7.1 It is also emphasised that the In-Service Support (ISS) infrastructure must be considered as an integral part of the whole warship disposal for sale process. Provision of spares is a particularly important issue which should not be overlooked. For further information on the disposal by sale of whole warships, refer to JSP 336 Defence Supply Manual Part 9 - Disposal By Sale.


Unclassified 159

6.5.8 Trials and Acceptance

6.5.8.1 The resulting ‘export variant’ must be subject to trials culminating in acceptance. The extent to which trials are conducted is again a subject for negotiation between the design authority, disposal sales organisation and the customer. See Section 7 for further information on trials.

6.5.9 Paying Off, Dismantling, Re-Use And Disposal

6.5.9.1 With the increasing emphasis placed on through-life issues due partly to recent Integrated Logistic Support (ILS) initiatives, disposal has become a topic of consideration within wider environmental, safety and security issues during early system activities.

6.5.9.2 By identifying removal and disposal as a formal Combat System life cycle stage, a framework of assessment and review activities can be set out. This will encourage early action to ensure that this stage is adequately planned for and appropriate measures taken to ensure that disposal of the Combat System is conducted in an environmentally responsible, safe manner without prejudicing security. Envisaging eventual system disposal is a feature of design approaches such as ‘design for disposal’ and the adoption of measures such as ‘Green Passports’.

6.5.10 Systems Engineering (SE) (Technical) and Project Management

6.5.10.1 The role of SE (Technical) and Project Management during the final stage of the life cycle is to ensure that the Combat System project is completed in a controlled and co-ordinated manner. In particular, from a SE perspective, the project manager must be satisfied that all pertinent information is archived to support analysis for future studies.

6.5.10.2 The central vehicle for promulgating disposal activities is the removal and disposal plan (a subordinate plan to the SEMP). This will have been generated as a draft during earlier stages to identify policy and practice for labelling equipment and components during their fabrication with material composition data to facilitate recycling. Towards the end of in-service life, the removal and disposal plan should be developed to provide the focus to:

a) Establish the goals and relevant activities necessary to perform removal and disposal;

b) Manage the conduct of final system assessments, removal and evaluation through appropriate subcontract arrangements;

c) Manage the disposal, salvage and destruction activities through appropriate subcontract arrangements;

d) Identify the processes through which all environmental, safety and security aspects will be addressed;

e) Ensure that the disposal objectives are met according to programme and cost;

f) Ensure that all relevant information is suitably archived and made available to future programmes.

6.5.11 Final System Assessments

6.5.11.1 An important aspect of disposal is to complete a series of final system assessments and provide information as an archive of the operational life of the Combat System. This will serve as a unique and useful information base for future Combat System studies.

6.5.11.2 The purpose of the final system assessments is to complete and/or terminate ongoing studies which have accompanied the Combat System implementation during its operational life. These should include:

a) Availability, Reliability and Maintainability (AR&M) evaluation, concluding with final gathering of non-obsolescent equipment and system availability and reliability data from on-board records and data collection systems;


Unclassified 160

b) LSAR to bring the Logistic Support Analysis (LSA) up-to-date and perform comparisons on predicted and actual support performance in accordance with Def Stan 00-60;

c) Final life cycle costing to compare estimates and validate models used to analyse WLCs in tandem with the LSAR. This should include the recording and analysis of all costs incurred during removal and disposal;

d) Operational performance studies assessing the continuing capability/performance of the Combat System over its operational life time (recognising any potential changes to the operational context e.g. threat and environment);

e) Safety case report, used to identify aspects of the design and its operational usage which impact on safety. This will have been updated throughout the operational life of the Combat System to address any newly identified hazards;

f) Security evaluation of classified equipment, documents and data for sanitisation.

6.5.11.3 Each will be formally closed over a period of time during which requested information is made available by other parties. Closure of these final system assessments is not a pre-condition to the disposal, salvaging or destruction activities.

6.5.12 Maintain and Archive Records

6.5.12.1 From a SE perspective, it is essential that records are kept so that future applications have comparative performance metrics on which to gauge meaningful estimates. Current innovations in large scale data archiving such as data warehousing provide a practical means for managing the archive of Combat System information generated throughout the life cycle. This information should be made available to new projects through an appropriate archiving facility provided within the IT infrastructure. Access to the archive must be assured so that the data is made available to future programmes for re-use, improved management, contract definition, costing and performance.

6.5.12.2 A core set of information from the Combat System life cycle would include:

a) TLMP including the SEMP;

b) Requirements/Acceptance Database holding the URD and SRD;

c) Design Documentation including System Specification, Coherence Specification, Subsystem Specifications and Interface Documentation;

d) Test and Trials Documentation including Trials Schedules and Test Forms;

e) Combat System Safety Case;

f) LSAR and WLC providing the appropriate ILS Documentation;

g) System Studies including capability/performance models and supporting data;

h) In-service data, historical log data and final system assessment data (relating to the system capability at service exit) identifying system strengths and weaknesses.

6.5.13 Removal and Evaluation

6.5.13.1 Removal

6.5.13.1.1 Items removed which have the potential for sale or salvage as spares are evaluated during removal. Many items will have a resale value which can be exploited through surplus sale outlets. Particular items may be worth salvaging as spares where the associated in-service equipment or variants thereof is still operational and existing spares are limited.

6.5.13.1.2 Removed items may include the following:


Unclassified 161

a) Ammunition and explosives including torpedoes, mines, pyrotechnics and countermeasures;

b) Cabling;

c) Computer hardware and peripherals;

d) Consumable items including paper and other media (e.g. tapes and disks etc);

e) Documentation;

f) Electrical equipment and electronics including Printed Electronic Circuits (PECs) and Power Supply Units (PSUs);

g) Fixtures and fittings;

h) Mechanical assemblies such as racks, shelving etc;

i) Outboard fittings;

j) Software (resident in non-volatile memory) and firmware;

k) Test equipment (e.g. oscilloscopes, multimeters, cable test equipment etc).

6.5.13.1.3 All items must be evaluated against a number of criteria as listed below to ensure the appropriate disposal route is followed and that any certification necessary to support disposal is generated and maintained.

6.5.13.2 Environmental Considerations

6.5.13.2.1 All items considered for disposal should be reviewed through a BS EN ISO 14001-accredited Environmental Management System. This provides planning, control, monitoring, auditing and review mechanisms to ensure compliance with prevailing environmental policy reflecting UK Health, Safety and Environment legislation.

6.5.13.2.2 Relevant Acts of Parliament and related regulations include:

a) Control Of Pollution Act (COPA) 1974 - emissions to air and discharge to water, solid waste and noise pollution;

b) Environmental Protection Act (EPA) 1990 - regulations with respect to solid waste, control emissions to air;

c) Montreal Protocol - restrictions on CFC and related compounds;

d) UNECE (United Nations Economic Commission for Europe) Protocol - to secure a reduction in annual emissions of Volatile Organic Compounds.

6.5.13.2.3 Hazardous items must be identified and disposal conducted in accordance with BS EN ISO 14001. Where items suitable for resale do not comply with current legislation due to age or condition, appropriate notices should be attached.

6.5.13.3 Security

6.5.13.3.1 Evaluation of security risks associated with the handling of computer hardware and software (application software and data), and any electronic, electrical, mechanical or physical items or arrangements accorded sensitive status is a pre-requisite to disposal.

6.5.13.3.2 The removal of all information, both in electronic and hardcopy format, which has been stored or processed by the system must be in accordance with the specified security procedures as invoked by the removal and disposal plan. Computer media with a classification lower than confidential will need to be


Unclassified 162

subjected to downgrade and sanitisation procedures. Other items accorded sensitive status will be catalogued, certified and destroyed.

6.5.13.4 Safety

6.5.13.4.1 Safety issues, including safety of personnel, boat and decommissioning site, nuclear safety in relation to hull integrity and containment, hazardous materials and safe removal of weapons and pyrotechnics, need to be considered within the safety management framework mandated by JSP 430.

6.5.13.4.2 Any certification of safety critical software applications within the system is void once the software is removed from the target hardware. Re-use of such software would require the re-application of Def Stan 00-55 requirements. It is the responsibility of the project manager to ensure that suitable disclaimers are provided and retained with the application.

6.5.13.5 Software

6.5.13.5.1 Removal of computer software shall be conducted in accordance with Def Stan 02-620, covering safety and security considerations (as above), future responsibilities and conditions for re-use.

6.5.14 Equipment/Component Disposal By Sale

6.5.14.1 Items identified for disposal by sale need to be certified for re-use through certification as detailed above. All items shall be administered and catalogued by DESO, DSA or their resale agents in industry.

6.5.15 Salvage For Spares

6.5.15.1 Items identified as potential salvage for spares need to be recovered and passed onto the spares provisioning authority. It is unlikely that many items will be in a condition suitable for use as spares. Exceptionally, for items in short supply, refurbishment may be considered.

6.5.16 Recycling/Destruction

6.5.16.1 Items which cannot be re-used, and do not present a security or safety risk, should be considered for recycling for recovery and re-use of the raw material.

6.5.16.2 Items which cannot be practically re-used or recycled due to security considerations should be destroyed. Certification of destruction in accordance with current regulations and prevailing legislation provides a check on disposal through this route.


Unclassified 163

6.5.16.3 Certificates for destruction will be raised by the project manager for items the destruction of which needs to be witnessed. These will include:

a) Computer hard disks and media;

b) Documentation;

c) Weapons and explosives.


Unclassified 164



Unclassified 165

Systems Engineering Guide for Naval Combat Systems - Integration, Test, Evaluation and Acceptance Process

7 Integration, Test, Evaluation and Acceptance Process

7.1 Introduction

7.1.1 The integration process must ensure that the Combat System is brought together under a controlled and configured regime, through which it can be demonstrated, by means of tests and trials, that the developed system meets its requirements, will therefore deliver the military capability required, and can be accepted into service.

7.2 Purpose of the Integration, Test, Evaluation and Acceptance Process

7.2.1 The primary purpose of the integration, test and trials process is to bring the Combat System together from its constituent stand-alone equipments and integrate it as a functional system within its platform environment. Integration, Test, Evaluation and Acceptance is not always recognised as a formal stage in its own right since it may be considered as the set of system-level development and manufacture activities through which the Combat System is brought together as a complete system. The scope of activities, which need to be, considered however merits a separate section.

DESIGN FOR INTEGRATION/

MODEL-BASED INTEGRATION Design for Integration is a

central theme of earlier stages dealing with System Partitioning.

Models and Synthetic Environments provide

a means to identify andinvestigate integration issues.

PROGRESSIVE PHYSICAL INTEGRATION

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

Requirements

Acquire Equipment

Analysis/Design

Acceptance

Integrate/Test

SE Process

DP SS

SDSA

RE

Integration can be thought of as a matrix of ‘bottom-up’ activities, from equipment installation, testing equipment

working together in groups, through to the fully integrated system. These activities are conducted for the shore-

based (SDF/SIF) proving of high risk items, followed by integration on the First Of Class (FOC) and subsequent

Follow-On vessels.

SYSTEM TEST& TRIALS

SYSTEMINTEGRATION

EQUIPMENT TEST& TRIALS

INSTALLATIONAND STW

SDF/SIF FOLLOW-ON FOC

Figure 7.1 Combat System Integration, Test, Evaluation and Acceptance

7.2.2 Integration will have been considered from the onset of design activities, from partitioning of the system onwards. Risk analysis activities conducted during previous stages will have shaped the integration strategy. This will determine the required mix of design-oriented and model-based activities, and approaches to physical integration through shore-based and in-situ Integration, Test, Evaluation and Acceptance as illustrated in Figure 7.1. The width of the arrows depict the relative extent of testing in each situation, with the shore-based proving and First Of Class (FOC) trials providing the greater coverage of testing required.


Unclassified 166

7.2.3 Design For Integration

7.2.3.1 The concept of design for integration is a continuing thread of the strategy and is explored in some depth in previous sections. It is the far-sightedness, considering integration as a central ‘life cycle process factor’ during previous design activities, which seeks to assist the physical integration of procured equipment, avoiding major incompatibilities and improving inter-operability.

7.2.3.2 In support of this goal, modelling techniques may be used to identify potential integration problems and issues well ahead of actual physical integration at a point where costs associated with subsystem/equipment change can be more effectively contained. The continuation of modelling during this stage, augmented by real data acquired from equipment testing, serves to verify and refine early indications of overall performance. The development of this approach may be witnessed by the ascendancy of Synthetic Environments (briefly discussed in 1.5.7.2, 3.5.4 and 4.6.9.13 Synthetic Environments) where prototypes and models of design components may be brought together for a dynamic simulation of aspects of system operation in a representative scenario.

7.2.4 Progressive Physical Integration

7.2.4.1 The main focus of this section is the progressive physical integration of the Combat System, from the installation and setting to work of individual equipments to the fully functional Combat System. Whilst there are variations in the precise nomenclature adopted by submarine and surface ship Combat System implementation, the underlying approach to integration is comparable.

7.2.4.2 The accepted practice is to first prove high-risk items associated with the integration of novel items such as new technologies, new-to service equipments, and new interfaces in a shore-based environment (e.g. Shore Development Facility (SDF)/Shore Integration Facility (SIF)). This provides the opportunity to extensively exercise subject equipment integrated as a partial system and to perform tests and trials, which cannot be practically conducted on the platform. With the advent of fast WAN connections these facilities may be geographically distributed but conceptually form a single facility.

7.2.4.3 The shore-based environment is a cornerstone of the Combat System integration strategy as an essential risk mitigation exercise. The extent of the equipment fit may vary with the complexity and novelty of the Combat System, however the underlying principles remain. The aim is to prove system design concepts and identify/resolve equipment integration problems in a benign and controllable environment ashore before the equipment is integrated onboard and to provide this facility through life to support the In Service phase.

7.2.4.4 Upon satisfactory completion of tests and trials in the shore-based environment, a similar set of progressive integration activities is conducted for the FOC to prove integration with the platform. These are conducted first through harbour trials, which are followed by sea trials. Subsequent integration activities for follow-on platforms follow the same progressive pattern, but the depth of testing is much reduced.

7.2.5 System Change Management

7.2.5.1 System Change Management during integration, test, evaluation and acceptance is applied in a comparable way to change management activities conducted during demonstration and manufacture, as identified in Section 4. The main thrust here is to first investigate the potential impact of change and co-ordinate the implementation of the change through the analysis and design process.

7.2.5.2 Approved changes will need to be implemented and integrated through the progressive sequence of activities as before. Any shortfalls in performance of the overall Combat System against the requirement determined from impact study and as a result of verifying models against real test data should be ‘fed forward’ into subsequent Combat System programmes.

7.2.6 Integration, Test, Evaluation and Acceptance

7.2.6.1 The above themes are expanded in Figure 7.2 which identifies the major activities conducted during this stage. The main emphasis of this section is the progressive physical integration of the Combat System from the point of equipment acquisition through to the fully trialled Combat System, up to and including System Acceptance (SA). This is achieved through a staged regime of integration, test and trials activities


Unclassified 167

conducted in the shore environment (SDF/SIF) and, through harbour and sea trials on the FOC platform (and to a lesser extent, follow-on platforms).

7.2.6.2 This progressive, configuration managed regime provides a basis for demonstrating Combat System physical installation, functionality, performance, operability, Availability, Reliability and Maintainability (AR&M) and operational effectiveness though an incremental sequence of repeatable tests, each followed by formal acceptance. This is co-ordinated through Systems Engineering (SE) and project management (technical) activities which run throughout all life cycle stages.


Shore Facilities, TrainersFOC, Follow-on Vessels

Equipment for IntegrationFacilities, FOC and Follow-onVessels, Training Facilities,Supporting Documentation

Key DocumentsManagement Plans

Requirements Database Design Documentation


Models & SimulationsCost, Effectiveness,

Functionality, Architecture,HCI, AR&M

Combat SystemINTEGRATION TEST and TRIALS


SDF/SIF HATs SATs

DOCUMENTATION



SYSTEM STUDIES


ACCEPTANCE



SYSTEMINTEGRATION

EQUIPMENTS

COMBAT SYSTEM

SUBSYSTEMS

Support Infrastructure

Key DocumentsManagement Plans

Requirements Database Design Documentation


DOCUMENTATION

SPECIALIST SUPPORT

CM

SYSTEM STUDIES

ILS

ACCEPTANCE

Figure 7.2 Integration, Test, Evaluation and Acceptance, overall Process

7.2.6.3 Both shore-based and platform-based test and trials provide evidence for acceptance of the Combat System through the generation and presentation of acceptance evidence to demonstrate compliance with the requirements. In the interest of improving the cost-effectiveness of the shore-based environment and containing risk, it is desirable to demonstrate compliance with as many requirements as possible ashore. Through the use of scenario generators, simulators and stimulators, it is possible to provide a representative environment for the dynamic test and trials of major aspects of the Combat System.

7.2.6.4 Activities, which support the central integration, test and trials activities throughout the stage, are depicted separately. These address the conduct of system-level studies, Integrated Logistic Support (ILS), the production of documentation, specialist engineering discipline support and Configuration Management (CM). CM is particularly important at this stage as the vehicle for co-ordinating the wide diversity of system/subsystem/equipment configurations and accompanying test documentation.

7.3 Inputs

7.3.1 Integration, Test and Trials commence with the delivery of Combat System equipments, following satisfactory completion of equipment Factory Acceptance Tests (FATs). There is a plethora of output products arising from the previous development and production stage. These are the deliverables from the equipment development process which can be grouped into:

a) Equipment sets for integration and installation in shore establishments (and trainers) and on vessels, together with supporting test equipment where necessary;

b) Design, development and production information, including development and production plans, specifications and drawings;


Unclassified 168

c) Equipment test, trials and acceptance information including test and integration documentation, trials plans, specifications and schedules, SRDs, ITEAPs and TLMPs.);

d) Equipment operation and support information including Master Record Index (MRI) data, configuration management documentation, technical publications, upkeep documentation, Logistic Support Analysis Record (LSAR) data and operator/maintainer training documentation.

7.3.2 Additionally, there are many output products, which are maintained from previous stages and extended as a result of Combat System development. These can be grouped under the following headings:

a) Management plans;


c) System functional and performance models used for tracking the Combat System design and to be used in support of aspects of system optimisation and acceptance;

d) Integration test plans, schedules and specifications in preparation for the subsequent integration activities;

e) Combat System test, trials and acceptance information including trials plans, specifications and schedules, and Combat System SRD, ITEAP and TLMP;

f) Combat System operation and support information.

7.3.3 All items will be produced, maintained and controlled as an inherent element of the system baseline design description under the appropriate CM procedures. This may be controlled and coordinated through a central platform-oriented information infrastructure or at the Combat System level depending on the contract regime.

7.4 Outputs

7.4.1 Handover includes the physical delivery of the Combat System and the transfer of the Combat System Design Authority to DLO (if appropriate). Handover of design authority may not necessarily correspond with acceptance into service. Note that DLO may support platforms before they take on design responsibility. The following items are subject to a transfer of responsibility and ownership:

a) Combat System subsystems and equipments - SA provides the point of handover of the responsibility for individual equipments (and associated test equipment, tools and supporting documentation) to CINCFLEET;

b) Combat System software-intensive subsystems and equipments - maturing of software for handover to DLO needs to be considered against the equipment procurement strategy. Certain items where a continuous development strategy is being employed (e.g. for Command Systems or systems with significant Commercial Off The Shelf (COTS)-components) may be retained by MoD(DPA). Transfer to DLO will occur as direct by prevailing policy;

c) SDF/SIF - these facilities are essential for investigating and replicating problems, defects and deficiencies and for assessing and proving system/equipment modifications. The commitment to supporting and maintaining the shore facilities needs to be considered within the prevailing contractual and procurement policy environment. Handover may follow SA depending on the procurement and acceptance policies;

d) Training facilities – if equipment trainers are procured through equipment contracts they will be subject to the same procurement policies and contractual arrangement as for the parent equipment but may have a separate ITEAP. Composite training systems (e.g. Command Team Trainer (CTTs)) procured under the Combat System URD) will be subject to an equivalent acceptance route to the Combat System. Handover to DLO will therefore follow formal acceptance as described in the ITEAP;

e) Engineering data - includes system design documentation provided as design drawings, specifications and associated information in the datum pack consistent with the MRI system as well


Unclassified 169

as system integration, test and trials documentation retained by DPA (including test form documentation). Engineering data will accompany the Combat System at SA;

f) Models, prototypes and system study information - models and prototypes for evaluating and assessing system functionality, performance, operability, AR&M, effectiveness and Through Life Costs (TLCs) will become an increasingly important route for assessing emergent requirements and performing change impact analysis on key aspects of the design. Models and prototypes will be held in a central reference facility and made available to support both formal acceptance and maintenance through life, the mechanism for which will be described in the ITEAP and TLMP.

7.4.2 Supply support mechanisms developed with the involvement of the DLO during previous stages will be administered through a whole warship support infrastructure. CM of Combat System equipment configurations and associated items listed above will likewise be administered within the CM process of the whole warship. Additionally, the LSAR developed during earlier stages, will be brought forward and maintained to reflect actual support performance.

7.5 Organisation

7.5.1 Contract Regime

7.5.1.1 The organisation of the integration and test teams for naval projects follows similar lines across a wide range of projects, since all of them, inevitably, have to bring together a set of components of a system and test them as an overall assembly. The major feature affecting the organisation of the work is the contract regime, which will be consistent with that set up for the demonstration and manufacturing phases, described in Section 4.

7.5.1.2 Regardless of who is running the project, or what size it is, the important feature is that the implementation of strong project management disciplines are essential for success. The integration and testing of a Naval Combat System involves many different organisations in industry, MoD and the RN and, increasingly, some of the players may be based overseas. This means that the need for communications, planning, and clearly defined responsibilities come to the fore. All of the parties involved will have to work together at some time in the confines of the shore facilities or the vessels.


7.5.2.1 Integration Test and Trials involves a number of organisations each of which has distinct roles to play as depicted in Figure 7.3. Though the actual boundary of responsibilities is negotiable to some extent within the prevailing procurement regime, the roles are generally well defined. They are as follows:

a) End-User organisation, the RN represented by the nominated Customer 2;

b) Support Authority, the DLO, charged with the In-Service Support (ISS) of the Combat System;

c) Equipment Supplier organisations provide the products and service which has been procured as subsystems and equipments and supporting services. It is important that these organisations are available to support integration, test and trials activities, assist in resolving integration problems and correct deficiencies up to and including full acceptance;

d) Conducting Authority, a role associated with Maritime Commissioning, Trials and Assessment (MCTA), the unit

e) responsible for the conduct of tests and trials and recommendation to the Capability Acceptance Working Group (CAWG) who are responsible for recommending acceptance of the system against the URD. Acceptance off-contract the is a DPA responsibility whilst overall acceptance against the URD is a Customer 1 responsibility as advised by the CWG(A), on completion of SA carried out on behalf of CINCFLEET;

f) Combat System Integrator, responsible for bringing all of the equipment components together into a working Combat System. Within this organisation is the Test and Trials Organisation, typically provided by the platform design authority, who is dedicated to organising and conducting tests and trials and producing the test documentation.


Unclassified 170

SUPPORTAUTHORITY

END-USERORGANISATION

Support Productsand Information

Accepted Productsand Services


TEST and TRIALSORGANISATION ACCEPTANCE

AUTHORITY

EQUIPMENTSUPPLIERS

CONDUCTINGAUTHORITY

TestProgramme/

TestDocumentation

AcceptanceEvidence

Products andServices

WEAPON/SYSTEMACCEPTANCECOMMITTEE

Figure 7.3 Integration Test and Trials Organisational Relationships

7.5.2.2 The interaction of these organisations is formalised through the formation of the CWG(A), which serves to influence all integration test and trials activities which lead to acceptance, and for the rectification of identified defects and deficiencies.

7.6 Integration, Test, Evaluation and Acceptance Process

7.6.1 The scope of the Integration, Test, Evaluation and Acceptance process depends upon the nature of the prevailing procurement policy and contract. The extent of a prime contract to design, build, introduce into service, and support a new class of vessel is different in scope to that for dealing with a new variant of an in-service weapon or command system introduced as part of a refit package. However the underlying principles and building blocks of the process are essentially the same.

7.6.2 Responsibility for integration, test and trials activities will be identified in the ITEAP. Increasingly the traditional role of the MoD project team is devolved to industry in the prime contract setting. In other cases, the MoD may retain the integration authority but contract the integration role to a contractor who will act as an ‘agency’ rather than an ‘authority’.

7.6.3 The main focus of integration, test and trials may be considered as a matrix of ‘bottom up’ activities, from installation of equipments through to the fully integrated system as illustrated in Figure 7.4.


Unclassified 171

Figure 7.4 Stages of Combat System Integration

7.6.4 This is the system integration dimension, which is considered in detail in this section dealing with the integration, test and trials process.

7.6.5 In practice, integration must be conducted through a sequence of tests and trials, which provide the assurance that the Combat System is operationally acceptable prior to entering service. To limit exposure to risk associated with integration shortfalls or failure, the accepted practice is to anticipate integration through modelling, and then prove the system in a benign environment ashore before committing to installation and integration on the target platform. The sequence is illustrated in Figure 7.5 and described in detail in para 7.7 Practical Integration, Test, Evaluation and Acceptance

OVERALL INTEGRATION

GROUP INTEGRATION

ONE-TO-ONE EQUIPMENT INTEGRATION

STAND-ALONE EQUIPMENT TESTINGFollowing extensive factory testing includingInterface and Data Exchange Proving, equipments are delivered to the integrationsite.Equipments are installed and subject to repeattests to confirm correct operation during STW.These culminate and contribute to the acceptance of the equipment through HAT(E).

Group Integration comprises further functionaland performance tests of functional warfareareas involving more than two equipments.

It tests the resilience of the group to concurrentoperation, high loads, changes in configuration,and system ‘threads’ through the equipmentsintegrated in the group.

Overall Integration covers those functional andperformance tests that involve more than onewarfare area, together with the concurrentoperation of Group Integration tests to check forany degrading mutual interaction. These culminate in tests of the whole CombatSystem operating under a high load and overan extended period.

One-to-One/Member Integration comprisesInterface, data exchange, functional andperformance trials between an equipment and(separately) each of the other equipments withwhich it interfaces. A reduced set of these trials may contribute toan Interface NWHT.

A B C

D E F

AB

D

AB

A

A

B

C D

E

F

D

B A C

E F

B C A E

A B

B C

DA B

D

Combat System Trials

System Trials

Interface Trials

FII HAT(E)

II FAT


Unclassified 172

Figure 7.5 Practical Sequence of Integration, Test and Trials

7.6.6 The primary activities undertaken during integration, test and trials are depicted in Figure 7.5 which shows the underlying process against the backdrop of the practical modelling/shore/harbour/sea integration sequence typical of Combat System integration projects. The process is described in detail in the remainder of this part, whilst practical aspects of integration are addressed at para 7.7 Practical Integration, Test, Evaluation and Acceptance.

7.6.7 The process elaborated here details the activities, which need to be collectively, addressed by the Combat System organisations as an enterprise. Clearly there is much scope for variation between contracts and the implementation of the generic process, as depicted in Figure 7.5 must be tailored to reflect prevailing project circumstances.

SHIP-BASED TEST & TRIALS SHORE-BASED TESTINGMODEL-BASED INTEGRATION

Sim Sim

ScenarioGenerator

EmSim Sim Stim Stim

Sim Rec

A B C FC E

Effect

Effect

Sens

Sens

Rec A B C F C E D

Upon satisfactory completion of tests and trials in the shore-based environment, a similar set of progressive integration activities is conducted for the First Of Class (FOC) to prove integration with the platform. These are conducted first through Harbour Trials in which the simulated elements (such as external sensors) are replaced by actual equipment fit. peripherals and tests repeated.

SEA TRIALS ( SATs)

HARBOUR TRIALS ( HATs)SHORE FACILITIES (SDF/SIF)MODELLING and SIMULATION Modelling techniques may be used to identify potential integration problems and issues well ahead of actual physical integration at a point where costs associated with subsystem/equipment change can be more effectively contained. The continuation of modelling during this stage, augmented by real data acquired from equipment testing, serves to verify and refine early indications of overall performance. Modelling and Simulation is addressed in Clauses 2 through 3. Harbour Trials are followed by full Sea Trials,

which seek to confirm correct Combat System operation in the operational context. Subsequent integration activities for follow-on platforms follow the same pattern. Sea trials form the basis for acquiring data in support of longer-term goals such as AR&M performance, data for which will be gathered through-life.

The shore-based environment (e.g. SDF/SIF) isa cornerstone of the Combat System integrationstrategy as an essential risk mitigation exercise.This provides the opportunity to exercisesubject equipment (through simulation,stimulation and emulation within a dynamicscenario representative of the operationalenvironment) to perform tests and trials whichcannot be practically conducted on the platform.The aim is to prove system design conceptsand identify/resolve equipment integrationproblems in a benign and controllableenvironment ashore before the equipment isintegrated onboard.

A B

C D

E

F

HATs SATs Modelling SDF/SIF


Unclassified 173

SHORE-BASED TESTING SHIP-BASED TESTING

HARBOUR TRIALS SEA TRIALS


Risk Management System Change Management Acceptance Management AuditsProgress Monitoring and Reporting Cost Control QA

Combat System SAT(F)AR&M Trials

Operational EffectivenessTrials Operability TrialsWEMIT Ranging Trials

RADHAZ

COMBATSYSTEMTRIALS

SHORE FACILITIES


ACCEPTANCE

Contract AcceptanceCombat System FWA

EQUIPMENTACQUISITION

Equipment FATEquipment FATInterface Proving

Data Exchange Proving

COMBATSYSTEMINTEGRATION

EquipmentNWHT/SAT(E)

EquipmentNWHT/HAT(E)

EquipmentNWHT/HAT(E)

Final InstallationInspection

Setting To WorkInstallation Inspection

Final InstallationInspection

Setting To WorkInstallation Inspection

EQUIPMENTACCEPTANCE

Equipment FWAEquipment SAT(F)Equipment HAT(F)

Combat System HAT(F)Combat System HAT(F)

System NWST/SATSystem NWHT/HATSystem NWHT/HAT

Equipment HAT(F)

Interface ProvingData Exchange Proving

DOCUMENTATION

SYSTEM STUDIES

INTEGRATEDLOGISTIC SUPPORT

Test and Trials Documentation Trouble Reports Analysis ReportsEngineering Databases and Information Systems Records Archive and Library

SPECIALIST DISCIPLINE SUPPORT Alignment Communications EMC Procurement Handbooks AR&M Human Factors ILSOperational Effectiveness Platform Integration Safety Security Training

CONFIGURATION MANAGEMENT Configuration Definition Configuration Control Configuration Accounting

Validation and Refinement of Prototypes and Models Data Recording and Analysis CompliancePrediction System Performance Prediction System Change Investigation and Impact Studies

Logistics Support Analysis (LSA) Life Cycle Costing (LCC) Integrated Supply Support Technical DocumentationMaintenance Planning Support & Test Equipment Manpower & HF Training & Training Equipment

DEFECT MANAGEMENT

Full System Integration/Overall IntegrationStaged System Integration (SYSINT)/Group Integration

Data Exchange Proving [e.g. Secondary Integration (SINT) and Functional Integration (FINT)]and Interface Proving [e.g. Primary Integration (PINT)]/One-to-One/Member Integration

Figure 7.6 Integration, Test, Evaluation and Acceptance Activities


7.6.8.1 Objectives

7.6.8.1.1 Managing the integration task requires a wide range of co-ordinated activities to ensure that the Combat System is efficiently brought together from its constituent interfacing equipment and platform components. Project management activities during this stage serve to continue the regime (i.e. the management of contracts, risk, quality, programme, Work Breakdown Structure (WBS), organisation, progress, change etc.) put in place at the beginning of the previous development and production stage.

Note : For FWA read SA


Unclassified 174

7.6.8.1.2 SE activities during this stage follow on from those initiated earlier in the life cycle, with the emphasis on supporting the integration process. SE activities are therefore primarily concerned with:

a) Scheduled demonstration of compliance with functional, non-functional and performance requirements, ultimately determining ‘fitness for purpose’;

b) Identifying and resolving problems arising from installation, physical integration, testing and trials;

c) Co-ordination and collation of information to support the Combat System in-service.


7.6.9.1 MoD Weapon System Manager (CSM) Role

7.6.9.1.1 The MoD CSM role is one of overseeing the Combat System enterprise, ensuring that Combat System integration, tests and trials proceed to cost and programme. NOTE: If the Combat system is procured by a Platform Prime Contractor, the MoD still requires to monitor and have visibility of activities in order to support Whole Ship Acceptance. The activities to be overseen include:

a) Management of system requirements, particularly with regard to change and the assessment of change impact;

b) Maintenance of the overall design through refining the fidelity of through-life models to represent achieved performance;

c) Assessment of achieved performance against baseline definition for identification of potential change for current or future Combat System implementations;

d) Addressing Combat System-wide (coherence) issues, updating the coherence, system, subsystem and Interface Specifications (IFS) when necessary;

e) Managing integration in the SIF or FOC, monitoring the progress of test and trials;

f) Monitoring conformance of the Combat System equipments with platform policies;

g) Monitoring the equipment development programmes and ensure that SDF/SIF plans can accommodate changes in equipment delivery dates;

h) Witnessing a representative set of equipment, integration and Combat System-level acceptance events to confer acceptance of contract articles/deliverables;

i) Assessing during the Integration Test and Trials integration against project metrics and Technical Performance Measures (TPMs).

7.6.9.2 Activities during the Integration Test and Trials Stage

7.6.9.2.1 SE management and project management activities required to co-ordinate integration, test and trials must be planned for and arranged during previous stages and management plans (chiefly the ITEAP which must reach the maturity required in time) and provide the means to document proposed policy and practice. Many of the activities are common to previous stages covered in this guide and are addressed in detail in para 1.6.3.2 Systems Engineering (SE) (Technical) and Project Management. Priority life cycle-related activities to be conducted during the integration, test and trials stage are as summarised there.


Unclassified 175

7.6.9.3 Control Points and Reviews

7.6.9.3.1 The major control points in the integration, test and trials phases that come from the structured approach to the conduct of inspections, tests and trials, are enforced by the adoption of a BR 4050 compatible approach. Entry into each step is controlled by ‘readiness’ reviews and by ‘pre-trial’ meetings. Exit is controlled by post-trial ‘wash-ups’.

7.6.9.3.2 The occurrence of these ‘control points’ in the overall integration, test and trials programme is further structured by the integration strategy and the process which emerges from that. There may be some significant ‘ready for integration reviews’ where the status of major development items and their readiness for a major trial is to be assured.

7.6.9.4 Programme Management

7.6.9.4.1 The programme management of integration, test and trials activities, including the development and use of a SIF should be done as a normal part of the overall project programme management. The formal trials process (from installation inspections through to System Sea Acceptance Trials (SAT(S))) will provide the basic logic for a draft programme. The realities of equipment deliveries, the shipbuilding programme, and the availability of trials ranges, areas, and assets will require very careful planning and a measure of flexibility.

7.6.9.4.2 Once the integration programme has commenced, a pragmatic approach must be taken with regular re-planning. This is necessary to take account of the impact of equipments missing their delivery dates and the temporary loss of equipments that cannot participate in integration until corrective fixes have been implemented and tested.

7.6.9.4.3 For reporting purposes, a sound set of ‘process metrics’ is recommended as best practice. These have to be meaningful, sensitive, and useful for all parties involved in the programme. Metrics must reflect the impact of known delays and the existence of problems being faced (since these may well bring about further delays). In recent times, programme-related metrics tends to be identified by Risk Management techniques. Project management teams will normally agree the metrics to be used as part of the project or programme management plan. Typical examples of these process metrics may include:

a) Project cost, variance, time to completion, cost to completion;

b) Milestones achieved, days variance on milestone;

c) TPM achievement, variance;

d) Requirements compliance (especially User Requirement satisfied), overall programme coverage.

7.6.9.5 System Readiness Levels

7.6.9.5.1 SRLs are indicators of the maturity of equipment and system design, which are selected from actual, measurable performance parameters. These are monitored throughout the design, development, production, build, integration, test, and trials process as a management aid to identify potential performance problems in key areas of the design.

7.6.9.5.2 Modelling provides the initial predictions of SRL value. Later on, more detailed modelling may provide a SRL based on calculations. Development testing may provide the first physical measurements of SRLs but, finally, it is integration, test and trials activities that provide the ultimate data used to assess whether the required SRL growth has occurred, along with the required increase in the SRL confidence level.

7.6.10 Design Change Management

7.6.10.1 Objectives

7.6.10.1.1 The Combat System engineering process must recognise that change is an inevitable consequence of the integration activities. The key to success is to manage change such that any impact is first assessed before being implemented and re-tested.


Unclassified 176


7.6.10.2.1 During this stage, the MoD CSM role is one of identifying, promulgating and assessing the impact of changes to the Combat System as a result of performance shortfalls of specific equipment(s). In such cases, the impact of shortfalls will require further investigation with respect to the capability of the warship.

7.6.10.3 Activities during the Integration, Test, Evaluation and Acceptance Stage

7.6.10.3.1 The need to assess the impact of change remains throughout this stage, to provide feedback to subsequent programmes and to confirm the functionality and performance of the baseline Combat System implementation. During this stage, change is likely to arise from the sequence:

a) Identification of integration problem;

b) Proposals for a corrective change;

c) Assessment of best course of action;

d) Implementation and re-test.

7.6.10.3.2 Design Change Management activities must follow the generic process invoking a controlled regime (i.e. a project review cycle) as described in para 1.6 and 4.6. This provides a template for revisiting the requirements engineering, systems analysis and system design activities to ensure that any impact is identified, reviewed and approved.

7.6.10.3.3 System studies should again be used to ensure that the design remains compliant with the requirements and represents a viable solution. Approved changes need to be introduced following the integration, test and trials process outlined here through the controlled environment of the SIF/SDF before installation and testing onboard.

7.6.11 Equipment Acquisition

7.6.11.1 Objectives

7.6.11.1.1 Although it is not specifically a part of Combat System integration, successful completion of the equipment/subsystem Factory Acceptance Test (FAT) is a pre-requisite of equipment delivery to the SIF, FOC and Follow-on (FON) vessels. It is advisable to ensure that equipment/subsystem FAT specifications include tests which de-risk Combat System integration, whilst the equipment remains under the direct control of the equipment supplier.

7.6.11.1.2 No equipment should be allowed to participate in the SDF or FOC integration until it has successfully completed its FAT. Equipment contracts should ensure that the equipment is subject to design proving at the factory within the constraints of the stand-alone environment.


7.6.11.2.1 Acceptance of equipment and agreement and demonstration of conformance of interfaces is an important activity, which, in the case of high profile interfaces (e.g. command system/principal sensor), may demand MoD CSM representation. Where individual equipments are procured through the MOD, the Equipment Project Manager (EPM) will retain responsibility for satisfactory completion of the FAT.

7.6.11.3 Factory Acceptance Test (FAT)

7.6.11.3.1 The test results from previous testing in particular prototype testing will be presented at the FAT for formal acceptance of the requirements and initial confirmation of Equipment Specification and that the ITEAP is being adhered to. This will include the demonstration of selected tests to confirm and formally agree test results. Provisional acceptance will also be sought for those elements of the design, which can not be fully accepted until the completion of environmental testing or later, in the SIF or through HATs/Sea Acceptance Trials (SATs).


Unclassified 177

7.6.11.3.2 Acceptance will address the functional, physical and performance characteristics of the equipment and the correct functioning of its interfaces within the constraints of stand-alone operation. The FAT will include a review of the evidence presented and formal signing off. Any failures will need to be reworked and re-tested.

7.6.11.4 Interface and Data Exchange Proving

7.6.11.4.1 It is particularly important to test and verify the physical and logical operation of external interfaces to ensure that they conform to a recognised networking / interface standard (as defined by the IFS e.g. Def Stans 00-19, 21-24, 21-26 and 21-28 for data transmission) and comply with their Data Exchange Specifications (DES). For equipments which communicate across a data highway, the integration strategy must determine not only the optimum sequence of integration but also the strategy for validating physical and logical operation of stand-alone equipments prior to connection. Test equipment may be required to demonstrate:

a) Compliance with the relevant interface standard;

b) Interface design proving through data highway simulation;

c) Emulated equipment-to-equipment data exchange in a limited system configuration.

7.6.11.4.2 The use of test equipment is encouraged as a risk reduction measure, though it is acknowledged that the authenticity and fidelity of highway simulation potentially limits proof of the data exchange and associated test scenarios.

7.6.12 Equipment Test and Trials

7.6.12.1 Objectives

7.6.12.1.1 The objective of equipment test and trials (alternatively known as equipment Preparation for Acceptance (PFA)) is to demonstrate that the subject equipment meets its requirements in stand-alone operation. This is to contribute to the assessment of ‘fitness for purpose’ and to determine that, when installed, the equipment will function reliably, safely and to specification at sea in the military environment. Equipment Test and Trials, as summarised in Table 7.1 are the necessary pre-cursor to the staged integration of the Combat System and the test results obtained may contribute to overall acceptance. Further details are contained in BR 4050.


Unclassified 178

Table 7.1 Equipment Test and Trials Activities

Activity Applicability Purpose

Installation SIF/FOC/FON Installation of the equipment in the SIF, or on the FOC or FON vessel.

Installation Inspection (II)

SIF/FOC/FON Carried out after completion of installation work to give clearance to begin Setting To Work (STW) of the equipment. II may be completed subject to defects and deficiencies, which will need to be satisfied before FII. The II serves to confirm that:

• Equipment has been fitted in accordance with approved installation specifications and drawings. Any departure or conflict with the contracted or approved installation specifications or other relevant fitting out information must be reported; • All other installation work, connection to services (e.g. power, chilled water etc), tests and checks required as pre-requisites for the commencement of STW have been satisfactorily completed; • Accompanying test equipment, special tools, documentation (Installation Specification, STW procedures, Maintenance Schedules, Equipment Handbook/Operating Instructions, Build State Records) and services have been provided.

7.6.12.1.2 There is a considerable difference in the level and intention of testing of SA i.e. HAT(F) and SAT(F) and follow-on acceptance. SA is extensive to probe the design and evidence for acceptance; follow-on HAT and SAT is less extensive, sufficient to demonstrate that the equipment is functioning correctly.


7.6.12.2.1 The MoD CSM needs to keep a close watch of progress towards integration and will benefit from visibility of the equipment test and trials programme including regular liaison with the nominated EPM for new and modified equipments, which fall within the remit of MOD.

7.6.12.3 Subsystem/Equipment Implications

7.6.12.3.1 The integration strategy must determine the optimum route for integration of equipments/ subsystems which best addresses risk to the development of the Combat System itself. There is therefore a need to test new/modified equipment fully and rigorously as stand-alone items prior to commencing integration in the SIF. In-service items, which have previously achieved SA and are required in an un-modified form, need to be subject to a HAT to confirm their correct operation. The following table summarises the implications of development status on the integration process.


Unclassified 179

Table 7.2 Subsystem/Equipment Integration

Category Integration, Test, Evaluation and Acceptance Implications

New Equipment - solely developed, integrated and accepted under the Combat System requirement

Modelling Functional and architectural performance models of new/modified equipments can be used to assess functional compatibility and verify data exchange across equipment interfaces. Models should be validated by test data generated in support of FAT.

Modified Equipment - defined by an independent requirement but functionally or physically changed to support integration into the Combat System

SIF Each new/modified equipment must pass its FAT before delivery to the SIF. The SIF equipment set must be identical to the FOC & FON sets. Where the Equipment Development and SIF programmes run in parallel, interim equipments may be delivered. In these cases, rigorous testing is required in the SIF.

New/Modified Interfaces - separate development of interfaces required to integrate NDIs (see below)

FOC All tests from Installation Inspection to System HAT need to be conducted. New/modified equipment installed on the FOC is to be subjected to Equipment HAT(F)/SAT(F) trials in support of SA.

FON FON equipment sets will require only a subset of the tests performed on the FOC to confirm correct installation and operation. As a minimum, FON equipment requires IIs and HAT(E), followed by sufficient integration testing to de-risk the Combat System HAT as determined by the Integration Strategy/Acceptance Management Plan.

NDI In-Service Equipment - typically equipment which has achieved SA to be integrated as is without modification

SIF/FOC/FON

Auditing/re-validation of test data generated from previous SIF and FOC testing for major equipments (central to Combat System core functions).

Integration and repeat test of FOC equipment as determined by the Integration Strategy/Acceptance Management Plan.

NDI Non-COTS - bespoke equipment in-service with a foreign navy which has never been subject to UK MoD acceptance

SIF/FOC/FON

Independent evaluation needed to determine functionality and performance.

NDI COTS - commercial products and items developed for military application to be integrated without modification

Modelling/SIF/FOC/FON

Independent evaluation needed to determine functionality and performance.

Integration of COTS components (particularly diverse software packages developed by competing organisations) into an operable system is a non-trivial undertaking. Incompatibilities are inevitable, given the diverse features of available products.

Testing demands a definition of the requirements to which the subject equipment has been designed. Requirements to which re-used/COTS equipment has been designed are not necessarily available or, if they are, complete.


Unclassified 180

7.6.13 Combat System Integration

7.6.13.1 Objectives

7.6.13.1.1 The integration of a Combat System is a complex ‘bottom-up’ process, bringing together component equipments in an ordered sequence of repeatable steps to realise a fully operational system. There are no absolute rules for the sequence of integration and each case must be planned according to its needs. The vehicle for defining the optimum integration process is the previously developed integration plan, which must establish the sequence, which represents the least risk. However, there are acknowledged activities, which although defined as distinct steps, may in practice be combined according to the needs of the project.


7.6.13.2.1 The MoD CSM role is primarily one of ‘informed customer’, monitoring progress of the project against the integration programme, facilitating communications and the resolution of issues between Combat System, equipment and platform programmes wherever there is potential conflict. NOTE: If the Combat system is procured by a Platform Prime Contractor, the MoD still requires to monitor and have visibility of activities in order to support Whole Ship Acceptance. Additionally, liaison will be necessary to ensure the smooth transition and clear responsibility for liability, from factory to shore facilities where these are under MoD ownership (e.g. Land Based Test Sites (LBTS)/SIF at Portsdown).

7.6.13.2.2 The CSM should satisfy himself that policy and strategy for integration of the Combat System is being implemented in an effective manner, and those integration risks are identified early and appropriately managed. This also means ensuring that testing of equipments meets the needs of the Combat System and that duplication of testing is minimised.

7.6.13.3 Combat System Integration Activities

7.6.13.3.1 Although the principles of Combat System integration are well founded there is some difference in nomenclature between submarine and surface ship application. The following table summarises the integration activities, which are to be conducted.

Table 7.3 Combat System Integration Activities

Submarine Nomenclature Surface Ship Nomenclature

Primary Integration (PINT)

Testing of the electrical interface between two equipments as defined by the associated and approved IFS. This tests the physical and electrical characteristics of the interface together with its basic communications protocol, which supports the data exchange.

Member/One-To-One Integration

Interface, data exchange, functional and performance trials between equipment and (separately) each of the other equipments with which it interfaces. Surface ship policy is to exhaustively check data content and range values in the stand-alone roving of equipments where possible.

A reduced set of these trials may contribute to an Interface NWHT.

Secondary Integration (SINT)

Testing of the data exchange between two equipments for data content and the range of values as defined in the DES. This involves tests of message transfer in isolation. Secondary Integration may be combined with PINT.

Combat System Integration Activities (continuation)


Unclassified 181

Submarine Nomenclature Surface Ship Nomenclature

Functional Integration Test (FINT)

Testing of the data exchange between two equipments to prove the functionality as defined in the DES. Typically, this will involve a sequence of message exchanges between subsystems in line with expected behaviour in the context of a scenario. This is also to ensure that the separate functions of equipment are not impaired through their interfacing.

(Staged) System Integration Test (SYSINT)

Staged series of individual tests to prove the functional interaction of logical groups of equipment to show that equipments can successfully operate in an integrated manner. A gradual build-up of groups of equipment will culminate in full Combat System tests.

Group Integration

Integration of communicating equipments which support one or more functions covered by a functional warfare area (e.g. Above Water Picture Compilation or Anti-Submarine Warfare). Group Integration covers further functional and performance tests where the function involves more than two equipments. It tests the resilience of the group to concurrent operation, high loads, changes in configuration, and system ‘threads’ through the equipments integrated in the Group.

Overall Integration

Overall Integration covers those functional and performance tests that involve more than one warfare area, together with the concurrent operation of Group Integration tests to check for any degrading mutual interaction. These culminate in tests of the whole Combat System operating under a high load and over an extended period. Agreed scenarios are used to create heavy though realistic track loadings for the scenarios used for high load and volume tests.

7.6.13.4 Regression Testing

7.6.13.4.1 Regression testing serves to prove that the addition of new functionality does not detract from the previously proven operation of an existing capability. Regression testing repeats earlier testing under a wider variety of carefully controlled conditions to isolate and correct latent problems. Particular attention needs to be paid to build states including software issue status in order to maintain traceability and control between integration configurations.

7.6.13.5 Further Information

Further information can be found in:

a) BR 4050 Instructions for the Conduct of Naval Weapons Inspections and Trials;

b) The AMS, which has copious advice under ‘Acceptance’;

c) Def Stan 21- 88 Policies and Procedures for Combat System Integration in Surface Ships.


Unclassified 182

7.6.14 Combat System Trials

7.6.14.1 Objectives

7.6.14.1.1 By the time the Combat System equipments are installed in the FOC platform, a comprehensive programme of integration testing in the SIF will have been completed. Although the SIF is a cornerstone of the integration and acceptance programme, it has the limitation that the Combat System environment is simulated and the Combat System may not be fully representative of the ship fit.

7.6.14.1.2 Integration on the FOC and HATs seek to confirm the system behaviour, established in the SIF, on-board the FOC. Subsequent SATs extend the test/trials programme to exercise functionality and performance which cannot be adequately demonstrated in a SIF or on the warship when alongside.

7.6.14.1.3 Combat System Trials are intended to demonstrate that the integrated Combat System meets the SRD as amplified by the ITEAP and TLMP and support the MoD CSM in presenting the Combat System for SA.


7.6.14.2.1 Combat System trials need to be co-ordinated as an integral part of whole warship trials as identified in the whole warship trials plan. NOTE: If the Combat system is procured by a Platform Prime Contractor, the MoD still requires to monitor and have visibility of activities in order to support Whole Ship Acceptance. The plan should be consistent with BR 4050, the needs of individual equipments and subsystems, and the requirements of the Combat System acceptance plan. It is the responsibility of the Combat System Integrator (CSI) organisation, in coordination with the CSM to:

a) Identify appropriate trials and ensure that all relevant trials are included;

b) Determine the conduct of trials;

c) Ensure that all trials results are adequately analysed, recorded and reported;

d) Check the completeness of the trials documentation supplied;

e) Ensure that any additional trials are reported, and agree the appropriate actions required conducting any additional trials.

7.6.14.2.2 The conduct of individual trials is the responsibility of:

a) Individual EPMs for equipments/sub-system specific trials;

b) CSI for installation, operational spaces, safety, and alignment aspects of the whole warship and for Combat System integration and performance aspects.

7.6.14.2.3 The CSM is responsible for ensuring that all parties are aware of their individual responsibilities and that trials produced reflect the current agreed status of the Combat System design.

7.6.14.3 Harbour Integration

7.6.14.3.1 Integration on-board the warship follows the general pattern defined in BR 4050 for inspection and trials. Equipment is installed in accordance with the Installation Specification and then subjected to an Installation Inspection (II). The equipment suppliers are then responsible for equipment Setting to Work (STW).

7.6.14.3.2 Integration in the FOC may be developmental: all new/modified equipments must have reached Naval Weapon Harbour Trial (NWHT) standard before consideration can be given to them being connected to a ship-wide Combat System data transmission system. Experimental connection to other equipments via the highway must be approved by the WSM.

7.6.14.3.3 Where physically separate items of equipment together form a subsystem, which is the responsibility of a single EPM, these equipments will be linked prior to seeking harbour acceptance for the


Unclassified 183

equipments/subsystem through the HAT(E). Equipment acceptance is the responsibility of the EPM, although the CSI may provide assistance if required.

7.6.14.3.4 Following Contractor Sea Trials (CSTs), subsystems are linked on a group by group basis as for the previously conducted SIF integration. Stimulation of most sensors is still required at this point and hence trials support facilities (e.g. scenario generator and recording/analysis) will be required on-board.

7.6.14.3.5 Integration testing onboard should follow the same sequence as for the SIF, with the emphasis on confidence building. If confirmatory results are obtained from key tests, then it is not necessary to repeat the full range of testing, in particular to the extensive range of performance testing, carried out in the SIF. Shortcomings or anomalies require reporting, leading to regression testing or impact analysis studies. Completion of harbour testing will enable the Combat System HAT to proceed.

7.6.14.4 Combat System HAT

7.6.14.4.1 The Combat System HAT serves to prove the functional interaction of the full Combat System with equipment being stimulated to simulate operation in a complex test scenario. Individual and combined equipment functions are checked as for previous integration activities.

7.6.14.4.2 The Combat System HAT is conducted following completion of integration activities to confirm, as far as possible in harbour, that the Combat System is free from defects, working in accordance with approved schedules and will be supportable in-service.

7.6.14.5 Sea Trials

7.6.14.5.1 Following the harbour testing, sea trials allow the Combat System to be tested in the operational environment. The sea trials phase includes both Combat System integration proving as well as performance demonstration of those aspects of the design that cannot be conducted in the SIF or alongside. Where the SIF/SDF has been used to prove both trials orders and the equipments in order to de-risk the sea trial, previously run shore trials may be repeated. Equipment stimulators and simulations are removed to make way for connection to actual sensors at the appropriate point in the integration/trials programme.

7.6.14.5.2 It is important that trials include a range of tests designed to exercise and prove those aspects of system operation only demonstrable in at-sea conditions. They are intended to establish benchmarks for comparison purposes but not necessarily to duplicate previous SIF or harbour tests.

7.6.14.6 Sea Trials Programming

7.6.14.6.1 The extent of sea trials needs to be kept to the absolute minimum to contain the cost associated with safely operating resources, ranges and assets in the execution of specific trials. A schedule of trials best representing the phased demonstration of the Combat System must be produced which seeks to achieve the optimum balance, taking into account:

a) Safety aspects, including collision avoidance, navigation and communications, will always take priority;

b) Individual EPMs seeking to achieve SATs & SA for their equipments where these are new to service;

c) Availability of assets and ranges together with clearances for firings where necessary;

d) Common use of scenario conditions modelled during earlier integration activities (e.g. effectiveness modelling, scenario generation in the SIF);

e) Availability of operational skills through use of the ships staff in the conduct of SATs and effective control of associated assets;

f) Operator familiarisation with the facilities;

g) Opportunities for ship staff training, maintenance, and leave;

h) Grouping of similar resources to be used for a range of trials;


Unclassified 184

i) Contingency to allow for weather, asset non-availability e.g. defective aircraft.

7.6.14.6.2 Control of the sea trials programme should be exercised through regular (daily if required) meeting of all the decision makers involved, the authority, the contractor, the commanding officer, and the inspection staff, or their respective representatives empowered to make decisions.

7.6.14.7 Trials Data Recording

7.6.14.7.1 Recording and analysis support facilities are required for both for ‘quick-look’ (i.e. on-board) purposes and subsequent, more detailed, shore analysis. Individual trial orders will clearly identify the data to be recorded as proof of individual system attributes. ‘Quick-look’ analysis of complex trials recordings should be used to confirm the validity of recorded data before the ship departs the ranges or releases assets. This will serve to reduce the risk of needing to repeat trials due to faulty recording, conduct or procedures. A guide to appropriate data types is provided in Def Stan 21-43. A DR&A ‘Needs Analysis is required by MCTA Policy. Further advice should be sought from MCTA-W1.

7.6.14.8 Repeat Trials

7.6.14.8.1 Results required either, as supporting integration proving or acceptance will be formally presented to the acceptance authority at the earliest opportunity after completion of the trial. Trials assessed as standards ‘not-achieved’ will be carefully reviewed and repeat arrangements agreed between the authority, contractor, the ship’s operational control (OPCON) authority for the warship, range authorities, and the other asset controllers as required.

7.6.14.9 RADHAZ

7.6.14.9.1 Radiation Hazard (RADHAZ) trials are normally conducted during the later stages of vessel construction, after the commissioning of all of the radio frequency emitters, particularly the communications and radar equipment. These trial results are a significant input to the safety case for the vessel. There are two basic sets of RADHAZ trials relating to personnel safety and weapon fuse safety. The objectives are:

a) To provide evidence that the safe areas defined by calculation during design are actually safe on the vessels;

b) To demonstrate that the electro-magnetic (EM) field levels affecting weapon embarkation, stowage or transit routes will not cause inadvertent fuse activity.

7.6.14.9.2 Further advice is available from DCSA EI&IS Test & Reference Branch, Blandford who have specialists in all areas of Electromagnetic Environmental Effects (E3).

7.6.14.10 Ranging Trials

7.6.14.10.1 Ranging trials are undertaken after the construction and commissioning of systems on the vessels, where ‘open water’ conditions are needed to characterise various performance parameters of the installed Combat System and other systems. In particular, characterisation of the vessel’s signatures is undertaken to provide information vital to the survival of the ship in wartime operations. The types of ranging trials performed are summarised in Table 7.4.


Unclassified 185

Table 7.4 Ranging Trials

Trial Performance Parameters or Signatures

Noise Ranging Acoustic Signatures

Magnetic Ranging Magnetic Signatures Degaussing System Performance

Radar/Radio Frequency Ranging

Radar Cross Section Radar (Glint) Signatures EW Calibration Aerial ‘Polar Diagrams’ and ‘Wooding’ assessments to determine sensor and comms receiver/transmitter blind arcs and coverage EW Signature

7.6.14.10.2 The objective of these trials is two-fold:

a) To characterise the signatures of the platform to enable the vulnerability to threat weapon systems to be assessed;

b) To gather information for use in threat evaluation libraries.

7.6.14.11 Weapons Electrical Mutual Interference Trial (WEMIT)

7.6.14.11.1 WEMIT is an assessment of the weapons equipment operation on the performance of the communications systems - both internal and external - and vice-versa. The trial comprises a methodical progression through each subsystem/equipment to assess the effects of operating weapon equipment whilst the communications system is operated in a ‘typical’ operational mode. As a consequence, WEMIT demands a high level of manning to monitor the communications system performance and to detect interference in any of the external or internal communications ‘circuits’ established.

7.6.14.11.2 The objective of the trial is to ensure those interference characteristics, if they exist, are known and that any interference detected is operationally acceptable. A significant feature of this trial is that it should only occur after all equipments have achieved SAT standards. Part WEMITs can be conducted at earlier points to reduce risk in programmes with a high equipment or system development content.

7.6.14.12 Operability Trials

7.6.14.12.1 Operability trials are conducted to ensure that the expected operator performance can be achieved. This work forms part of the Human Factors (HF) proving exercises. Operability trials can be used at any point in the development or integration/acceptance flow. They effectively form the demonstration point for showing that the ‘man’ forms an effective part of the system.

7.6.14.12.2 Operability trials can cover many parts of the HF spectrum, from proving the ergonomics of a console - field of view, ability to reach controls - through to assessment of operator workload in high stress/high load conditions. These trials are a significant feature of the overall trials programme and are used to demonstrate the successful implementation of the HF (management) plan and the achievement of HF requirements.

7.6.14.13 Operational Effectiveness

7.6.14.13.1 Operational effectiveness is a function of ‘performance’ and ‘availability’ of the vessel and its weapon systems in a specific scenario. Test and trials data will be collected during the early phases of in service life when the relevant operational and tactical development authorities control the vessels. This will provide initial evidence in support of the validation of the modelled operational effectiveness performance, allowing any advantages or shortfalls in performance to be characterised.


Unclassified 186

7.6.14.14 AR&M Trials

7.6.14.14.1 In 1981, AR&M performance was given equal significance with operational performance, and AR&M trials have become increasingly important. This is especially so when the responsibility for achieving system or ship-level AR&M performance is allocated to a system or warship project manager or even to a prime contractor.

7.6.14.14.2 Obviously, AR&M trials are long term activities during which AR&M data is gathered to allow comparison of predicted and actual AR&M performance in support of contractual acceptance. This performance depends upon many factors, including:

a) Reliability of the delivered products;

b) Maintainability of the delivered products;

c) Adequacy of logistic (stores/spares provisions and the turnaround time (logistic services));

d) Adequacy of training services to provide competent maintainers.

7.6.14.14.3 These trials provide information to show that ILS requirements have been satisfied or point to areas where rectification effort needs to be applied.

7.6.15 Acceptance


7.6.15.1.1 Integration and acceptance of the Combat System must be planned in concert through the co-ordination of the integration and acceptance strategy developed during earlier stages. The integration process must achieve a fully functioning Combat System, whilst the acceptance process is an ongoing activity to gather evidence to gain contractual acceptance and support SA. The tests necessary for integration purposes will generally provide evidence for acceptance. This is illustrated in Figure 7.7.


SIF


Lower-levelSubsystem/Equipmet




Formal Acceptance

FOC

Combat System

Subsystem/Equipment

Design

Analysis

Reqs

Design

Analysis

Validation of System/Compliancewith Requirements


Follow On

Test

Integrate

Acceptance Trials

Reqs Validation of Equipment /Compliancewith Requirements

Test

Integrate

Manufacture

Acquire

Acceptance Trials

Subsystem/EquipmentDeliverables

Figure 7.7 Combat System Acceptance


Unclassified 187

7.6.15.1.2 The traditional approach to SA for weapon equipment uses Acceptance Criteria (ACs) as the definition of what is to be delivered, and an AQ to ensure that the ‘end user’ is getting what the ACs defined. This has been extended to the Combat System level progressively since 1980 and provides a keystone in the naval project management process. In Smart Acquisition the ACs and AQs are replaced by a combination of the SRD and associated verification criteria traced to the URD and the ITEAP.

7.6.15.3.3 The scope of the contracts being placed at present has significantly extended the scope of supply from industry and the methods of specifying requirements has become more structured. Modern contract regimes demand, at the contract outset, detailed definition of how the contract deliverables will be accepted by the customer and the end user. NOTE: The URD reflects the Military Capability required and therefore requires demonstration that the Interoperability and other Defence Lines of Development (DLoDs) can support the Military Capability required. This cannot be solely within the contractor’s and IPT’s remit but nevertheless must be arranged and included in the ITEAP.

7.6.15.3.4 A major feature of acceptance is that it enables the progressive gathering of acceptance evidence, from all phases of the project. Although acceptance evidence is collected throughout the project, and thereby reduces the risks to achievement of final acceptance, it should be noted that final acceptance is targeted at acceptance of the system installed in the vessel after demonstration in-service, at sea. The steps along the way to introducing a vessel into service with the RN provide a means for defining a staged acceptance scheme, in which Integration, Test, Evaluation and Acceptance plays a significant part.

7.6.15.4 Acceptance Points - Transfer of Ownership and Responsibility

7.6.15.4.1 For contracts that involve the large scale integration of complex equipment into a complex system as part of an even more complex product, whether ship, submarine, tank or aircraft, one aspect which needs to be considered is: the definition of who owns what at any one time, and who is responsible for what.

7.6.15.4.2 From a contractual viewpoint, ownership (transfer of title) should be transferred at the point of acceptance. Ideally, this should be the point of delivery – but rarely is. So, in a contract where items come from a number of different sources, deliverables may be provided by a number of different supplier organisations, and are integrated progressively into the Combat System – it becomes obvious that there is a need for great care in defining:

a) Who is to supply what;

b) Who is to provide what support services, to whom, and for how long;

c) Who is responsible for care, maintenance, and safe operation of delivered items;

d) When the point of delivery occurs – or how it is split in to phased deliveries;

e) When the point of acceptance occurs;

f) When the transfer of title takes place and who assumes ownership;

g) What the responsibilities of the new owner are.

7.6.15.4.3 There are the traditional approaches, used in MoD contracts where MoD are the prime contractor, where the ship’s staff standing by take over ownership of equipment and systems as they progressively reach a HAT ‘standard’. Where Industry is acting as the prime contractor, the rules are different with ownership and responsibility being maintained on industry, rather than transferring to MOD, until some defined acceptance point in the programme. There are also differences between surface ship and submarine programmes, where staged acceptance may be invoked or where the MoD takes over a surface vessel and its combat system for Part IV trials.

7.6.15.4.4 Progressive Acceptance is becoming an increasing feature of Smart Acquisition – although this is still being formulated within official policy.


Unclassified 188

7.6.15.5 Equipment Acceptance

7.6.15.5.1 Equipment Acceptance is a significant feature of the progress towards Combat System acceptance. Typically equipment is developed to support use across the fleet or the submarine flotilla as part of several Combat Systems. The fitness for acceptance into RN service is nowadays assessed at the equipment level and the system level.

7.6.15.5.2 SA cannot be granted until equipment functionality and performance has been satisfactorily demonstrated as part of the system trials. Similarly, it must be demonstrated that all equipment-related Line of Development can support the needs of the Combat System.

7.6.15.5.3 The need for separate equipment acceptance depends on many factors of which the most important are when:

a) An equipment is procured against a stand-alone URD rather than as part of a URD;

b) An in-service equipment requires modifications which affect the URD/SRD;

c) An equipment contains a significant proportion of COTS items and needs to be introduced into service. Note that the acceptance of COTS is likely to raise many new issues, particularly with regard to acceptance.

7.6.15.5.4 Equipment Acceptance is the responsibility of the EPM and the MOD(DPA) directorate holding responsibility for discharging the URD/SRD. The EPM may only be able to achieve partial or provisional acceptance until SA is achieved. (BR 4050).

7.6.15.6 Combat System SAT(F)/Combat System SA

7.6.15.6.1 Currently, with the adoption of a progressive approach to acceptance, the trials used to achieve SA are more likely to be the Combat System SAT on the second (or a later) vessel in the class. Effectively these trials make a point at which SA can be granted. SA depends on much more than trials and achievement of standards at trials. Particularly, ILS provisions and AR&M performance are a significant part of SA, which is signified by the IPT RM supported by the CAWG completing the Acceptance Questionnaire (see BR4050) rather than a single trial.

7.6.15.6.2 SA and SAT(F) trials make the point at which the Combat System’s introduction into RN service is completed and, operational capability is available. It also represents the acceptance by the naval staff that the URDs have been met and the system is ‘delivered’ to the RN’s user branches - operators/maintainers.

7.6.15.7 Contract Acceptance

7.6.15.7.1 All trials contribute evidence to demonstrate that contract requirements have been satisfied. Traditionally, a ‘Final Acceptance Trial’ is conducted before MCTA and the shipbuilders or prime contractor representative in the wardroom sign off the final acceptance certificates. Contract acceptance however is the formal review of the programmed deliverables and the Defect and Deficiency Database (D3B).


7.6.16.1 System Studies initiated during development and production continue during integration, test and trials with the emphasis on using and evolving models to:

a) Refine model fidelity using data generated from the performance evaluation of real equipment;

b) Assess the impact of any shortfalls or improvements in system design performance;

c) Support investigation and problem solving of integration across equipment interfaces;

d) Support acceptance of design aspects which cannot be definitively or practically demonstrated (e.g. operational effectiveness);


Unclassified 189

e) Support subsequent analysis of test and trials results e.g. pre-trials predictions and post trials analysis.

7.6.17 Defect and Deficiency Management

7.6.17.1 Objectives

7.6.17.1.1 Throughout the life span of a project, the Quality Assurance (QA) system has to ensure that there is an adequate method for dealing with problems of non-conformances and customer complaints. Defects and deficiencies are the terms traditionally used for problems experienced during ship-based integration, tests, and trials phases.

7.6.17.1.2 Defects and deficiencies used to be recorded and tracked using either MOD-sponsored or Shipbuilder-sponsored mainframe databases. Currently, the vogue is to strive for a ‘problem/action reporting and tracking process’ which is used throughout the full project term. Essentially, problems are identified and resolutions actioned which need to be appropriately programmed and resourced.


7.6.17.2.1 The test and trials organisation (test groups) will be primarily responsible for managing the resolution of defects and deficiencies during the programme up to vessel acceptance. The resolution of defects and deficiencies is a team activity-requiring liaison between all parties.

7.6.17.2.2 The project team needs to understand whether it will be possible to solve problems within the stated timescale. They need to be prepared for use of more resources or programme slippage. The project (management) team to ensure it is in control should regularly review problem-solving progress.

7.6.17.3 D3B Process

7.6.17.3.1 The D3B process replaces the D448 activities for vessel acceptance.

7.6.17.3.2 The purpose of the D3B process is to agree the defect and deficiency list and, more importantly, who is responsible for fixing the defect or remedying the deficiency and when it will be done. Actions are allocated to the shipbuilder, MoD project organisations, ships staff, or MoD support organisations, to be completed pre- or post-handover of the vessel to the RN.

7.6.17.3.3 With the formalisation of shore-based system integration, tests, and trials as part of the Combat System and ship system acceptance, there has been a move to record and manage defects and deficiencies which are not related to ship-fitted equipment. A variety of approaches exist across different projects, all of which have some basis in the procedure used for ships during build or for equipment and systems in service (e.g. the S2022 procedures). The recording and resolution of defects and deficiencies supports the cycle of formal reviews of requirements, design, product tests readiness, configuration definition, and requirement compliance.

7.6.17.4 Problem/Action Tracking Process

7.6.17.4.1 The essential features of a problem and action tracking process and toolsets used to support it are:

a) Application of a rigorous process and dedication to managing the process is the main point. The tool used (e.g. D3B) to support the process is only a tool. There is no easy alternative to the painstaking exercise of vigorously addressing each and every problem and recording it with the tool in such a way that there is no danger that the data can be misinterpreted;

b) Support of data analysis to give meaningful data, from which conclusions can be drawn and progress assessed, including identification of bottlenecks;

c) Only collect metrics that give meaningful pointers and use the information;

d) Ensure that every problem has an owner and a solution date;


Unclassified 190

e) Follow detail to the level where there is sufficient visibility to manage the project by making the right decisions;

f) Focus attention on areas that will result in the biggest real impact on progress; identify the show-stoppers and remedy them;

g) Use problem recording as a means of identifying risks - those problems that have no apparent straightforward solution should be transferred to the risk register. It should be recognised that one person’s problem can initiate another person’s risk.

7.6.18 Integrated Logistic Support


7.6.18.1.1 The introduction of the practices, which make up the ILS discipline, took place over a period of several years. Support is now viewed as a ‘full life cycle’ activity, rather than ‘what happens after introduction into service and SA’. The level of planning which now goes in to setting up and populating the LSAR, along with the concept and practice of ‘progressive upkeep’ means that the early stage of equipment and system life cycle and their usage in the integration, test and trials activities now becomes an integrated part of the logistic support life cycle.


7.6.18.2.1 There is still the split in responsibilities across the equipment and system life cycle, between the DPA and DLO, who fund ILS activities separately in the periods before and after transfer of responsibility from the DPA to DLO. Handover to the DLO occurs when equipments and systems are declared ‘mature’. Maturity needs to be demonstrated by all those things which have always been associated with SA, particularly:

a) Compliance with the documented operational, functional, and performance requirements;

b) Demonstration of AR&M performance in service, in the hands of the RN operators and maintainers, backed up by the logistic support organisations in MoD and in industry;

c) Evidence of a ‘stable product design’, through the lack of changes needed to make the hardware and software of the system components work as required to meet the specified functionality and performance;

d) The satisfaction of the ‘end user’ with the installed systems from the operator and maintainer viewpoint, once they are considered ‘in service’;

e) The provision of all of the support facilities and services needed to train RN staff; to provide stores facilities and services, to provide maintenance and repair facilities; and so on.

7.6.18.3 ILS Activities during Integration, Test, Evaluation and Acceptance

7.6.18.3.1 The Integration, Test, Evaluation and Acceptance activities play a significant part in the demonstration that the planned ILS aspects meet the requirements. The value of the Integration, Test, Evaluation and Acceptance phase of the programme to the ILS community depends on it being treated as an integral part of the ‘product life cycle’ and a source of valuable, early, real-life information on the implementation of products designed to optimise AR&M performance and achieve specified AR&M levels.

7.6.18.3.2 Over recent years, the move towards Combat System projects, with the improving level of SE has helped with rationalising ILS aspects. The current vogue for concurrent engineering, often implemented by ‘integrated’ design, build, and support teams, must be used as a significant opportunity to further improve ILS planning, implementation, and assessment across the whole of the life cycle.

7.6.18.3.3 The MoD and industry Integration, Test, Evaluation and Acceptance teams need to ensure that there is good interaction with the ILS teams. Each needs to involve themselves in the generation and approval of the integration, test and trials management plan and the ILS management plan - and the ensuing


Unclassified 191

processes throughout the project programme - to make sure that best value is derived from this part of the programme.

7.6.18.3.4 From the time that equipment is first developed, representative AR&M data can be gathered. Even though the information gathered at this stage needs to be used judiciously, it begins to provide evidence that the required or predicted AR&M performance is being achieved (or not, in unfortunate cases). For developing equipment, this will be the first ‘real life’ information for other equipments that it will add to the existing information.

7.6.18.3.5 The maintenance regime can also be brought into play after HAT(E) is achieved and all that is needed to support this can begin to be used and assessed. It is important that these activities are identified as evaluations, in their own right, in the acceptance management plan and the integration, test and trials management plan. The evaluation methods and assessment criteria need to be spelled out here too, just as they would for the functional and performance tests and trials.

7.6.18.3.6 In these early stages, especially with developing equipment, there can be a significant level of ‘disturbance’ as items are accessed for inspection or connection to test equipment. Problems occurring because of the level of disturbance in this phase detract from the expected AR&M growth. This is particularly true of the shore integration facilities and the FOC. It is also important in these early phases to recognize that “100% reporting” of defects and deficiencies does not occur in practice. A common understanding and code of practice is needed across the whole Combat System project, so that ‘like for like’ comparisons can be made. This should be a key part of the CM plan.

7.6.18.3.7 Once installed in the vessels, the population under assessment increases, leading to more complete information, allowing fuller assessment. Again, it is only after HAT(E) that equipment level information would be more reliable. Equipments still get disturbed during system level testing, so both equipment and system level information can be treated with more confidence following system HAT [HAT(S)]. The support information from the second vessel should be one of the best indicators of reliability growth, since there is less disturbance of the systems and, normally, more confidence all round in the installation, functionality, and performance of the equipment.

7.6.18.3.8 Assessment of ‘turn-round’ time on spares provision and repair loops needs to be done with care in these early stages, since these are affected by several factors. For ‘development items’, where the support services/infrastructure are also ‘developing’, special provisions will probably be made within the industrial base rather than through the MoD ILS organisations. For ‘in service’ items, shore facilities can find themselves, correctly, lower down the service priorities than operational vessels or training facilities.


7.6.19.1 CM is fundamental to the integration, test and trials programme and is set to become an increasingly important issue as evolutionary equipment development and greater use of COTS components becomes the norm. CM during this stage ensures:

a) Compatibility between test environments;

b) Repeatability of test conditions;

c) Consistent and compatible build states of items under test/review/assessment;

d) Delivered build state is compatible with specifications and deliverable documentation (user and support);

e) Ability to return to previous build states and test environments in case of catastrophic failures or problems.

7.6.19.2 It is worthless undertaking Integration, Test, Evaluation and Acceptance activities without knowing the exact state of the ‘items under test’. When developing equipment is involved in the testing process, it is important to understand what changes are anticipated and the implications of those changes across the system and the supporting systems. After a development test, the equipments of the Combat System must be restored to their previous states or the configuration record updated to a new state. The CM process in place must be able to handle the tracking of change and the assessment, forward planning, and then implementation of co-ordinated changes to the system.


Unclassified 192

7.6.19.3 Declaration of the configuration of the system under test, and all-ancillary equipment and documentation, is a key feature of the pre-trial and post-trial meetings.

7.6.19.4 It is important to anticipate the major issues such as the alignment of the ‘design state’, the design team’s definition of the system configuration, with the ‘build state’ of the systems installed in the SIF and in the vessels and shore training facilities. An essential feature is the early adoption of a master record database and a soundly workable integration mechanism for aligning information between the ‘system level’ database and the equipment suppliers’ databases. As time goes on, and more systems are being commissioned or introduced into service, the methods of dealing with spares in store and items in the repair loops becomes increasingly important.

7.6.19.5 It is vital that the integration, test and trials team has input into the CM plan and is closely involved in its maintenance and update. This task is not to be left to the ILS Team. They may provide the service, but the integration, test and trials team is a major customer for that service. The CM plan is likely to contain details of:

a) Combat System configuration in the SIF including simulators and stimulators;

b) Combat System configuration in the platform;

c) Hardware and software build state records (for SIF and platforms);

d) Hardware and software design state records (for SIF and platforms);

e) Combat System documentation including test and acceptance plans.

7.7 Practical Integration, Test, Evaluation and Acceptance

7.7.1 Shore Facilities - Objectives

7.7.1.1 The SDF/SIF provide a unique test-bed for the controlled testing and integration of individual equipments and subsystems (without weapons items such as missiles, guns, sonar transducers etc). They provide the following benefits:

a) Identification, exploration and resolution of problems by providing a representative environment with easier access and greater resources availability;

b) Evaluation of the functional and physical characteristics of the whole Combat System prior to installation, setting to work and integration onboard ship or submarine;

c) Simulation of Combat System operation in repeatable scenarios which it would be impossible to exercise in an operational environment;

d) Support role for the Contract and SA process.

7.7.1.2 The use of a SIF is an important risk reduction measure which serves to reduce the time required onboard to integrate, test and accept Combat Systems. It also provides a hardware and software support facility, which enables new equipment and retrospective changes/modifications to be developed and proved ashore, thereby consuming less operational ship time. This role is likely to become more important as equipment development becomes more evolutionary in nature and more COTS components are utilised. Additionally, shore facilities provide a useful training site during periods where they are not being used for integration, test and trials. The present SDF/SIF is also networked via the Joint and Multi-national Interoperability Assessment Network (JMNIAN), which facilitates secure “end to end” communications to various UK facilities including the Joint Command and Battlespace Management Applied Research Technology Demonstrator (JCBM ARTD), RAF Waddington and LSRC Blandford test facilities. The JCBM ARTD connection allows onward connectivity to the other International test facilities via Combined Federated Battle Laboratories (CFBL) Net.


Unclassified 193

7.7.2 Shore Facilities


7.7.2.1.1 The precise role and responsibilities of the MoD CSM are dependent on the role of MoD SIF facilities in the Combat System programme which are determined by the integration and acceptance strategy. Where facilities (e.g. Portsdown) are required, the CSM must prepare the way, setting up and facilitating communications between SIF administration, CSI and equipment supplier organisations.

7.7.2.1.2 Throughout the shore integration process, the CSM must keep close watch over all high-risk activities, to ensure that progress towards acceptance is achieved, and any significant problems are raised and resolved at the earliest opportunity. In pursuit of these objectives, the CSM needs to maintain an informed view on technical issues particular to the shore environment, which may qualify acceptance in any way. These technical issues are summarised below.

7.7.2.2 Scenario Definition

7.7.2.2.1 The definition of scenarios is a cornerstone of the system integration process, which provides a common basis for all dynamic modelling, and test activities, including the HAT(S). Overall policy encourages the use of a co-ordinated and consistent set of scenarios throughout the Combat System life cycle derived from the roles and tasks of the warship.

7.7.2.2.2 The scenarios required to demonstrate Military Capability, operability, effectiveness evaluation and integration should be based on agreed high level scenarios derived during concept and refined though subsequent stages through to development and production. Scenarios are developed with the objective of faithfully representing the requirements of the operational context, elements of which will determine the scope of real trials. This is to maximise compatibility, so that modelling, shore trials and sea trials have a common basis which will aid verification between the modelling, integration and testing processes.

7.7.2.2.3 The scenarios chosen should be used to the maximum extent in the SIF before undertaking the real trial to reduce the risk of a failed trial using expensive assets. Furthermore, the shore environment may be able to provide scenarios, which would be too expensive to recreate in an actual sea trial. This however is subject to how representative the SIF-fit is compared to the ship-fit Combat System and the fidelity of the real-world scenario simulation.

7.7.2.3 Scenario Generation

7.7.2.3.1 The provision of a scenario generator supports the progressive programme of integration testing with a varying mix of real equipments and emulations against a representative set of scenario conditions. A scenario generation enables the coherent exercising of equipments and emulations comprising the SIF/SDF for realistic system proving, by co-ordinating:

a) Simulations of Combat System equipments/functions (e.g. provision of own ships data);

b) Emulations (including the ‘stimulator’ as an inherent part of its interface functionality);

c) Real equipments with associated stimulators;

d) Stimulation consistent with generated scenario events (e.g. appearance of a new platform in the scenario);

e) Scenario events arising from Combat System equipment actions (e.g. firing chaff);

f) Simulation highway operation in support of the above, including data recording and playback.

7.7.2.3.2 Equipment stimulation and scenario generation will typically be controlled from some form of Centralised Computing Facility (CCF) in order to facilitate comprehensive, consistent and repeatable system wide tests.


Unclassified 194

7.7.2.4 Simulation

7.7.2.4.1 Simulation is the imitation or partial replication of a real or potential system involving the use of emulation, stimulation and/or modelling. Simulation is the representation of an equipment in the form of a ‘black box’ transfer function with outputs based on pre-prepared data, or derived from input values via a readily-definable relationship. Equipment simulations augment real equipments so that a full representation of the Combat System can be provided within the SIF. This is applicable for situations where the use of real equipment is either not required or not practical due to cost or other criteria.

7.7.2.5 Emulation

7.7.2.5.1 Emulation is a type of simulation, which involves the representation of equipment by its input/output transfer function, internal functionality to a specified level of fidelity, together with interface related components of the real equipment. Emulation may also include representative facilities for operator input/display where appropriate. The hardware/software interface (e.g. to a data highway) should be identical to the real equipment, and hence the real equipment (with associated stimulator, where appropriate) and its emulation are interchangeable in the context of the SIF.

7.7.2.6 Stimulation

7.7.2.6.1 Stimulation is the exercising of sensor-dependent equipment without the sensor. The stimulator (co-ordinated by a scenario generator) replaces the functions that would be provided by a sensor (e.g. radar dish) operating in the real-world environment. The stimulator comprises hardware and software, and may be equipment-specific, or generic (i.e. designed for use in conjunction with a range of radar types via software and/or cabling changes for example). The nature of the stimulator will influence the potential procurement route, with equipment-specific stimulators generally, though not necessarily, being provided by the equipment supplier.

7.7.2.6.2 To achieve a co-ordinated approach to stimulation and maintain maximum control over the integration process, a number of equipments will provide their own means of stimulation, which can be controlled externally (e.g. from a central facility). Stimulation may be implemented as an in-built function provided for other purposes (e.g. onboard training), or by some external equipment (e.g. dedicated PC).

7.7.2.7 Recording and Analysis

7.7.2.7.1 The main objectives of data recording is to collect evidence of the Combat System operation during tests and trials to determine compliance to specification by subsequent analysis and for fault identification. The key activity is to identify the data to be recorded to support the analysis.

7.7.2.7.2 Since recording and analysis is required both in the SIF and on-board, there are benefits to be gained by using the same equipment(s)/facility for both applications. Wherever possible, ‘in-service’ or standard recording and analysis facilities should be utilised. Data recording, data reduction and time synchronisation requirements need to be addressed through complementary use of facilities provided by:

a) Combat System equipments which are capable of recording data as part of their normal mode of operation;

b) Additional recording and analysis equipment e.g. highway recorders, trials/range test recorders.

7.7.2.7.3 Extensive on-line monitoring and comprehensive analysis is required during testing to confirm that the basic operation of the equipments under test is proceeding to plan. On-line monitoring of tests during scenario execution should be provided by the scenario generator together with the communications status of all connected simulations/emulations/equipments.

7.7.2.8 Data Reduction

7.7.2.8.1 Data reduction is required where the volume of data collected is large and needs to be reduced to manageable proportions by the rejection/removal of superfluous data. Techniques for data reduction can be applied in two ways:


Unclassified 195

a) Pre-filtering (selection) at the recorder to limit the data recorded to within user-specified parameters (e.g. within time bands, only specified message type(s), etc.) or in accordance with a selected default recording configuration. This is of particular relevance to data highway recording;

b) Post-recording selection to enable the analyst to only select data of current interest.

7.7.2.9 Data Analysis, Reporting and Action

7.7.2.9.1 Data analysis is required for the compilation of evidence demonstrating successful system operation and investigation of anomalous behaviour of one or more system components. Confirmation from each test is sought that the system configuration under test has behaved as specified in the test specification and related documentation. The data acquired through this activity should be used to verify and refine the through-life models, to support investigation and maintain the configured baseline definition of Combat System behaviour.

7.7.2.9.2 Where anomalies are discovered, action is required following issue of an analysis report. This action will generally take one (or more) of the following forms:

a) Reference back to the subsystem project, requiring the equipment to be modified to comply with the appropriate specification (e.g. a DES);

b) Regression testing;

c) Impact analysis, possibly leading to generation of an Engineering Change Proposal (ECP);

d) Concession application, the procedure for raising and approval/rejection of concessions relating to integration will be specified in the Combat System integration plan.

7.8 Integration, Test and Trials Stage Documentation

7.8.1 In the integration, test and trials phase of the programme, the main documentation activity is the generation, approval and use of the test documents needed to support the integration, test and trials activities in the shore facilities and on the vessels, in harbour and at sea. Equipment test documentation must be prepared in parallel with hardware/software development so that both equipment and documentation are available for testing to begin. Similarly, integration tests must be prepared in advance of the integration stage so that integration can commence as soon as equipments become available.

7.8.2 For vessels, especially the first of class vessel, there are stringent timescales laid down in the relevant sections of SSP 23 and BR 3018. Once the main milestones for the integration, test and trials programme are known, and the structure of the tests and trials documentation is defined in the draft test form index then the documentation generation programme can be developed. Because the test form index is generic, this can be done early in the programme to assist with resource planning in the later phases of the programme.

7.8.3 Table 5 summarises the requirements for the integration, test and trials documentation for the vessels. The documentation for use in the shore facility needs to feed in to the vessel programme and therefore forms a lead in to this vessel programme. The linking of the shore facility and vessel programmes is essential, despite the different ownership (respectively, CSM and warship manager in the MoD organisation; and technical directorate and production directorate in the shipbuilding or prime contract organisations).

7.8.4 Documentation also needs to be split into groups which will support the four ‘building blocks’: design verification and qualification; first off production; first off installation and build, and routine (in service) system testing. This should be identified in the documentation requirements relating to the integration, test and trials management plan and correlate with the list of integration, test and trials events in the various phases of the overall programme.

7.8.5 Other Documentation

7.8.5.1 The increasing use of the installed system in the shore facility and the vessels means that the information, which forms the whole of the handbook series, is used and assessed from an early stage. This


Unclassified 196

provides a valuable input into the ‘handbook verification’ process. The use by ships staff standing-by of the BR/CB documentation in draft form also provides an RN operator and maintainer view on the adequacy and correctness of the information provided at equipment, system and ship level. However, there is a continuing shift away from BRs/CBs to more commercial user/maintainer handbooks especially for OTS equipments.

7.8.5.2 The other main set of documentation, which is involved in this phase, is the acceptance documentation. The Integration, Test, Evaluation and Acceptance documentation must always support the evaluation of the system requirements, logged in the SRD. The results from the integration, test and trials activities must provide the acceptance evidence needed to show that requirements have been fully satisfied, this evidence being logged in the acceptance check lists, or a supporting database, the Acceptance Database (ADB).



Systems Engineering (Technical) and Project Management

General and Specific Policy and Strategy Papers (including Data Management Policy, ITEAP, COTS Policy)

Management Plans (CMP, RMP, Risk Register, Risk Analysis, RRAP, Quality Plans, SEMP, PMP)

Lower tier Management Plans (Integration Management Plan, Test and Trials Plan, Acceptance Management Plan, Support Plan, Disposal Plan)

Configuration Management Plan Life Cycle Costings

Updated to keep abreast of developments during the stage. Updated, reviewed and agreed by CSM.

Design Change Management

URD, SRD, ITEAP and TLMP System (Design) Specification Coherence Specification Subsystem Specifications Interface Documentation (SIDs, IFSs, DESs) Verification Cross Reference Index/Acceptance Database (ADB)

Requirements configured as baseline data, together with acceptance data) in a combined database

Change investigation and impact analysis performed by CSI. Agreed changes promulgated down the contractual hierarchy as necessary.


Unclassified 197



Combat System Integration

Trials Policy, Trials Requirements, Trials Management Plan Integration, Test, Evaluation and Acceptance Plan Shore Integration Plan Integration, Test and Trials Programme Test Documentation Preparation Programme

Integration, Test, Evaluation and Acceptance Requirements/ SpecificationsTest Equipment Specifications Subsystem Integration Test Procedures and Reports Combat System Integration Test Procedures and Reports

Developed by CSI under direction of CSM. Reviewed and agreed by CSM.

Equipment II/Trial Procedures and Reports Equipment PFA/STW Procedures and Reports Equipment HAT(E) Procedures and Reports Equipment Integration (PINT/SINT/FINT) Test Procedures and Reports


Combat System Test and Trials

Test Form Index (TFI) Working Issues of Test Form Index. Final Issue to Datum Pack for each Vessel.

Test Forms Development from Draft to Design Authority and Test Group Approved Issues.

Test Schedules (Detailed Operating Instructions for Functional and Performance Tests & Trials)

Development from Draft to Design Authority and Test Group Approved Issues.

Certified Test Forms Gathered and Maintained as Acceptance Evidence from Integration, Test and Trials activities. Final Issue to Datum Pack for each Vessel.

Trouble Reports/Defect Reports Records of problems experience with conduct of test or trial, or with system. Register of recorded problems and defects still requiring action to clear them, as part of the D3B (Defects and Deficiencies Database).


Unclassified 198



Design Queries (for those issues which cannot be understood from available documentation)

Raised during Integration, Test and Trials activities where issue is ‘not trouble’ and ‘not a defect’. May lead to Defect Reports, Trouble Reports, or Change Requests.

Trials Plans (Requirements for Use of RN and Other Assets, often known as the ‘Form ALPHA’)

Documents worked up from Draft responding to Trials Requirements to Authorised Issue with appropriate Operating and Project Authorities. Delivered as part of Datum Pack or appendices to the Acceptance Database (ADB).

Trials Orders (Trials Schedules for vessel-based activities)

Documents worked up from Draft (in response to Trials Requirements) to Authorised Issue with appropriate Operating and Project Authorities.

Trials Operation Plans (for range facility usage, especially NATO Ranges or AUTEC)

Documents worked up from Draft (in response to Trials Requirements) to Authorised Issue with appropriate Operating and Project Authorities.

Trials Reports (Records of trials outcome produced by Conducting Authority)

Part of Project Acceptance records (appendices to ADB).

Trials Analysis Reports (Records of trials outcome produced by Presenting (Design) Authority)

Part of Project Acceptance records (appendices to ADB).

Acceptance URD, SRD with Traceability to results from ITEAP Acceptance Check List

Completed by CSI under direction of CSM. Reviewed and agreed by CSM/CWG(A).

System Studies Operational Scenario Definitions Effectiveness Models Effectiveness Assessment Reports

Conducted by CSI under direction of CSM. Reviewed by DEC

Operability Demonstrators System Operability Assessment Reports

Conducted by CSI under direction of CSM. Reviewed by DEC, FLEET and FOSM/FOSF.

Functional Models Architecture Assessment Reports AR&M Modelling and Reports Trade-off Study Reports

Developed by CSI under direction of CSM. Subsystem/Equipment-level modelling conducted by EDC under direction of CSI/CSM. Reviewed and agreed by CSM.


Unclassified 199



Integrated Logistic Support

Upkeep Policy and Plan Ship Weapon Upkeep Planning Information Weapon Upkeep Disclosure Documents STTE (Special To Type Equipment) Report CRETE (Common Range Electrical Test Equipment) Report Handtools Reports Stowage Requirements and Reports Spares Reports Planned Maintenance Reports Defect Reporting Procedure Ship Fit Configuration MRI (Master Record Index ) WEDP (Weapon Equipment Development Programme) A&A Proposal Forms Software Issue Control Report

Developed by CSI (or CSSC dependent on support arrangements) under direction of ILSM/CSM. Reviewed and agreed by ILSM/CSM.

Training Plan and Reports Training Course Material Combat System Handbooks (CBs/BRs) Planned Maintenance Schedule (PMS) Weapon Repair Facility Statements (WRFS)

Developed by CSI (or CSSC dependent on support arrangements) under direction of ILSM/CSM. Reviewed and agreed by ILSM/CSM and FOSM/FOSF.

Equipment Upkeep Plans and Policy Equipment Training Plans and Course Material Equipment Spares Reports Equipment Handbooks Establishment Lists (E Lists)

Developed by EDC under direction of CSI (or EPM dependent on Design Authority arrangements) under direction of ILSM/CSM. Reviewed and agreed by ILSM/CSM.

Configuration Management

Configuration Definitions Established in Design Phases. Maintained as Design States. Approved Design State at Acceptance.

Configuration Accounts Established during earlier phases leading up to Installation. Maintained as Build state throughout the Integration, Test and Trials programme as baseline definition for all tests and trials. Delivered as Build State Definition at Acceptance.

Configuration Audit Reports Maintained as part of Project/ Configuration Management Files.


Unclassified 200



Change Control Documentation Register of items which will affect Integration, Test and Trials aspects, Change Note Files. Register of Outstanding Changes forms part of D3B Records.

Relevant Change Note Files handed over to ILS Design Group at Acceptance.

7.9 Integration, Test and Trials Stage Specialist Discipline Support

7.9.1 Objectives

7.9.1.1 In the integration, test and trials phase of the programme, many of the specialists are able to gather real-life information on the implementation of their designs. The following table provides a view of the level of specialist support required by the integration, test and trials team for activities in the shore facilities, for vessels in harbour, and vessels at sea.

Table 7.6 Specialist Discipline Support

Specialist Discipline Support through Engineering Discipline

Shore Facility Activities Harbour Activities At Sea Activities

Air Engineering TBD - if applicable Aircraft Handling and Movement Trials, Aircraft Fuelling/Defuelling Trials Air Weapons Handling Aircraft Arming Trials.

Flying Trials for all types of rotary and fixed wing aircraft embarked.

Alignment Assessment of the use of Alignment Data by all relevant system components.

Measurement of real-life values for insertion into Operational and Test Software for static testing.

Measurement of real-life values for insertion into Operational and Test Software for dynamic testing.

AR&M Initial assessment of AR&M performance based on the partial system fit in the shore facility. Initial assessment of maintainability of installed equipment.

Initial assessment of AR&M performance based on the full system fit in the vessel. Assessment of Removal Routes and maintainability.

Assessment of AR&M performance growth based on the full system fit in the vessel. Assessment of maintainability at sea. In-service R&M data collection.

Cost Maintain and Refine Through Life Costings/LCCs as Integration, Test & Trials progresses.

Influence the Integration, Test and Trials process to optimise LCCs.


Unclassified 201




Documentation Delivery of Handbooks and other documentation in time to support the Integration, Test & Trials programme.

Turning round amendments to handbooks and other user documentation arising from use in the facility.

Electro- Magnetic Engineering

Assessment of problem areas which are realistically represented in the shore facilities

RADHAZ Trials Transient Trials (EMI) EMC Part 1 Tempest Trials

Degaussing Trials EMC Part 2 WEMIT

Environmental Factors

Probably Not Applicable here. Shore facilities are benign environments. Environmental testing usually done in development phase.

Assessment of problem areas and assistance with rectification.

Assessment of problem areas and assistance with rectification.

HF/Operability Assessment of areas which are realistically represented in the shore facilities, especially after RN Training, using RN Operator and Warfare Teams.

Assessment of HF/Operability Performance issues in the real vessel environment.

Crew Certification for Sea Trials.

Assessment of HF/Operability Performance issues in the real operating environment.

Crew Certification for Operations.

ILS Operation of interim ILS facilities for Maintenance, replacement items, and repair loops.

Initial assessment of ILS Provisions based on support of partial system fitted in the shore facility.

Operation of Commissioning/ Test & Tuning’ ILS facilities for Maintenance, replacement items, and repair loops.

Assessment of ILS Provisions based on support of full system fitted in vessel, operated by contractor.

Operation of ‘Commissioning/ Test & Tuning’ ILS facilities for Maintenance, replacement items, and repair loops.

Assessment of ILS Provisions based on support of full system fitted in vessel, operated by RN Staff.

Installation & Ship Services

Resolution of Trouble Reports.

Initial assessment of measured ‘Demands’ and ‘Loads’ vs the predicted loads developed in the Design Phases using the partial system fitted in the shore facility.

Support to proving of Ships Services Interfaces under static conditions.


Assessment of measured static ‘Demands’ and ‘Loads’ vs the predicted loads developed in the Design Phases.

Support to proving of Ships Services Interfaces under dynamic conditions.


Assessment of measured dynamic ‘Demands’ and ‘Loads’ vs the predicted loads developed in the Design Phases.


Unclassified 202




Interface Data Management

Design Verification of the Interfaces available in the partial system fitted in the shore facility.

System Network loading assessment.

Resolution of Trouble Reports and Defect Reports.

Design Verification of all Interfaces available in full system fitted in the vessel, under static and stimulated dynamic conditions.



Design Verification of all Interfaces available in full system fitted in the vessel, under typical dynamic conditions.



Modelling and Analysis

Assistance with development of trials scenarios for the scenario-based sub-system and system level trials.

Prediction of expected results for trials.

Assistance with investigation into anomalous system behaviour and analysis of trials data.

Assistance with development of trials scenarios appropriate to the nominated trials ranges [At this stage, all trials are scenario-based sub-system and system level trials].



Assistance with development of trials scenarios appropriate to the nominated trials ranges [At this stage, all trials are scenario-based sub-system and system level trials].



Naval Architecture

The Naval Architects are unlikely to have any input into this phase. Naval Architects will be collecting ‘measured data’ on weights and C’s of G to balance the vessel in inclining and trim experiments.

Operational Performance

Assessment of the Operational Performance against earlier predictions, if information which is significant in determining system performance is available from the facility, e.g. system data transfer latencies and accuracies.

As for the Shore facility, using system data from the full system installed in the vessel.

Assessment of the Operational Performance against earlier predictions, using ‘macro level’ data which is available through testing the system in the vessel at sea or on ranges, e.g. actual signal to noise ratios, beam patterns, tracking accuracies, etc.


Unclassified 203




Safety Independent Safety Assessment of the Hazard Log and the Safety Operating Procedures for the system in the facility and any contributions to the Pre-Construction Safety Report.

Particular attention to any Safety-Critical features.

Independent Safety Assessment of the Hazard Log and the Safety Operating Procedures for the system in the Vessel in harbour and any contributions to the Pre-Operations Safety Report.


Independent Safety Assessment of the Hazard Log and the Safety Operating Procedures for the system in the Vessel at sea and any contributions to the Pre-Operations Safety Report.


Security Initial Assessment of Security policy implementation using system components available.

Intermediate Assessment of Security policy implementation for the system installed in the vessel operated in harbour.

Final Assessment of the security Policy implementation for the system in the vessel operated at sea within the RN Operating and Support environments.

Noise, Shock & Vibration

Assessment of problem areas which are realistically represented in the shore facilities.

Shock aspects are usually dealt with as part of the design and development activities.

Assessment of installed Noise and Vibration performance in the vessel.

Assistance with resolving any issues arising.

Assessment of installed Noise and Vibration performance in the vessel.

Assistance with resolving any issues arising.

Software Development

Assessment of the Quality of delivered software. Resolution of technology and infrastructure problems arising. Independent audit of software functionality. Independent assessment of any safety critical features.

Structural Engineering

Not applicable in this phase. Structural aspects are usually dealt with as part of the design and development activities. There may be problems to resolve in the installation phase or following sea trials.


Unclassified 204




Training Use of shore facility system for preparation of training course material for operator and team level training, and for ‘training the Trainers’.

Use of shore facility for training ships staff, if Operator, Maintainer, and Command Team Trainers are not available for the early courses.

Training of Ship’s Staff and MCTA to support equipment and system level trials.


Unclassified 205

Annex A Normative References

A.1 Reference Information

A.1.1 The publications shown below are referred to in the text of this standard. Publications are grouped and listed in alpha-numeric order.

AECMA 2000M International Specification for Technical Publications Utilising a Common Source Database – Initial Provisioning

ANEP 54 ‘Guidelines for COTS Insertion’

BR 3018 Nuclear Safety Management of Naval Reactor Plant (Formerly TOANS)

BR 3021 Shock Manual (Metric) Vol 1 & Shock Manual (Metric) Vol 2

BR 4050 Instructions for the Conduct of Naval Weapons Inspections and Trials

BR 8593(17) Alterations and Additions (A&As) Procedures for Surface Ships and Submarines (Supersedes Oct 98 Edition)

BS ISO/IEC 15288 Systems Engineering – System Life Cycle Processes

BS EN ISO 14001 Environmental management systems. Requirements with guidance for use

BS 5750 ISO 9000 Quality Management and Quality Assurance

COPA Control Of Pollution Act (COPA) 1974

D448 Report of Material and Trials State of New Construction and Post LOP(R)

Def Stan 00-19 The ASWE Serial Highway

Def Stan 00-41 Reliability and Maintainability, MoD Guide Practices and Procedures

Def Stan 00-55 Requirement for Safety Related Software in Defence Systems (Obsolescent 04-2005)

Def Stan 00-60 Integrated Logistic Support:

Part 0 – Application of Integrated logistic Support (ILS)

Part 1 – Logistic Support Analysis (LSA) and Logistic Support Analysis Record (LSAR)

Part 3 – Guidance for Application of Software Support

Part 10 – Electronic Documentation

Part 20 – Application of Supply Support Procedures

Part 21 – Procedures for Initial Provisioning

Part 22 – Procedures for Codification

Def Stan 05-57 Configuration Management of Defence Material.

Def Stan 05-91 Quality Systems Requirements for Design, Development, Production, Installation and Servicing (Obsolescent 03-2004)

Def Stan 05-95 Quality System Requirements for Design, Development, Supply and Maintenance of


Unclassified 206

Software

Def Stan 08-107 General Requirements For The Design of Electrotechnical and Naval Weapon Equipment (Cat 2)

Def Stan 08-124 Radio Frequency Environment and Acceptance Criteria for Naval Stores Containing Electro-explosive Devices (Cat 1)

Def Stan 08-136 Requirements for Electromagnetic Engineering:

Part 1 - Electromagnetic Engineering Management (Cat 2)

Part 2 – Guide to the Application of Electromagnetic Engineering (Cat 2)

Def Stan 21-13 Combat System Interface and Link Documentation:

Part 1 – Management Guide (Cat 3)

Part 2 – Interface Documentation (Cat 3)

Part 3 – Link Document Guide (Cat 3)

Def Stan 21-24 Data Transmission – Direct Memory Access Interface Standards:

Part 1 – The Software Interface (Cat 1)

Part 2 – The Electrical Interface (Cat 1)

Part 3 – The Enhanced Software Interface (Cat 1)

Part 4 – The Enhanced Electrical Interface (Cat 1)

Def Stan 21-26 Requirements for the Digital Representation of Shipboard Data Parameters:

Part 1 – Standard Units, Parameter representations and Element Formatting (Cat !)

Part 2 – Guide to the Use of NES 1026 Part 1 (Cat 1)

Def Stan 21-28 Standard for Inter-system Communications Protocols:

Part 1 – High Level Language of Message Construction (Cat 1)

Part 2 – Rules and Protocols for Intercommunication Across Digital Highways (Cat 1)

Part 4 – Rules and Protocols for the Passing of Large Blocks of Data Across Digital Highways (Cat 1)

Def Stan 21-43 Data Recording and Analysis for Maritime Platforms, Systems and Equipments, Guidance for User Requirement Capture (Cat 3)

Def Stan 21-88 Policies and Procedures for Combat System Integration in Surface Ships

DOORS Requirements Database

E Lists (Establishment Lists)

EIA 632 Electronics Industries Alliance (EIA) Standard: Processes for Engineering a System

EPA Environmental Protection Act 1990

Form ERC/S2022 Defect and Deficiency Form

Health and Safety at Work Act 1974

IEEE 1220 IEEE Standard for Application and Management of the Systems Engineering Process, Institute of Electrical and Electronics Engineers. Revision to align IEEE 1220 with ISO/IEC 15288 and provide a lower level of detail than 15288 is underway and due out December 2004. After that it will be fast tracked as an ISO standard


Unclassified 207

Joint Customer 2 Handbook

JSP 336 Defence Supply Manual Part 9 – Disposal by Sale

JSP 430 Ship Safety Management System Parts 1 to 4

JSP 440 The Defence Manual of Security

JSP 454 Procedures for Land Systems Equipment Safety Assurance

JSP 520 Ordnance Munitions Explosives Safety

Part 1 – Management System

Part 2 – Operational Procedures

JSP 553 Military Airworthiness Regulations (formerly JSP 318B)

JSP 600 Information Coherence Directions

JSP 602 The Data Imperative

MIEA 5929 (UK-RN-N-5929) Information Exchange Programme

MILSTD 499B Systems Engineering

MoD AMS Acquisition Management System – Human Factors Integration – Practical Guidance for Integrated Project Teams

Montreal Protocol

S2022 Ship Report of Shortcomings In Material Design Or Support

Smart Approvals Guidance Version 9.1

STG Maritime System Maturity

SSP 23 Design of Surface Ship Structures (Vols. 1 & 2)

SSP 38 Technical Procedural Requirements for the Procurement & Upkeep of Naval Weapon Systems

STG Publication 10 HFI Management Guide

STG Publication 11 HFI Technical Guide

UNECE United Nations Economic Commission for Europe Protocol

WSAF 129 Shore Support Arrangements for Warships (formerly known as 'Form Alpha)

A.1.2 Reference in this Standard to any normative references means in any Invitation to Tender or contract the edition and all amendments current at the date of such tender or contract unless a specific edition is indicated.

A.1.3 In consideration of clause A.1.2 above, users shall be fully aware of the issue and amendment status of all normative references, particularly when forming part of an Invitation to Tender or contract. Responsibility for the correct application of standards rests with users.


Unclassified 208

A.1.4 DStan can advise regarding where normative references documents are obtained from. Requests for such information can be made to the DStan Helpdesk. How to contact the helpdesk is shown on the outside rear cover of Def Stans.


Unclassified 209

Annex B Abbreviations

A&A Alteration and Addition

ACs Acceptance Criteria (formerly Acceptance Characteristics)

ADB Acceptance Database

ADS Accreditation Documentation Set

AKA Also Known As

ALARP As Low As Reasonable Practicable

ALSP Aggregate Level Simulation Protocol/Process

AMP Acceptance Management Plan; Assisted Maintenance Period

AMS Acquisition Management System

ANEP Allied Naval Engineering Publication

AP Assessment Phase

AQ Acceptance Questionnaire

AR&M Availability, Reliability and Maintainability

ARP Applied Research Programme

AUSCANZUKUS Australia, Canada, New Zealand, United Kingdom, United States

AUTEC Atlantic Undersea Test and Evaluation Centre

BITE Built-In Test Equipment

BoI Balance of Investment

BR Book of Reference

BS British Standard

C2W Command and Control Warfare

C4I Command, Control, Communications, Computers and Intelligence

CAD Computer Aided Design

CADMID Concept, Assessment, Demonstration, Manufacture, In-Service, Disposal

CALS Continuous Acquisition and Life-cycle Support

CAWG Capability Acceptance Working Group

CB Confidential Book


Unclassified 210

CCB Change/Configuration Control Book

CCF Centralised Computing Facility

CCU Certificate of Clearance for Use

CDMA Central Data Management Authority

CDP Cardinal Date Programme; Chief of Defence Procurement

CDPIs Chief of Defence Procurement Instructions

CESG Communications Electronics Security Group

CFBL Combined Federated Battle Laboratories

CFC Chlorofluorocarbons

CINCFLEET Commander in Chief Fleet

CIS Communications and Information Systems; CIS Integration Strategy

CL Core Leadership

CLS Contractor Logistic Support

CM Configuration Management

CMM Capability Maturity Model

CMP Contract Management Plan

CMS Command Management System

COA Concept of Analysis

CofG Centre of Gravity

COEIA Combine Operational Effectiveness and Investment Appraisal

CONEMP Concept of Employment

CONOPS Concept of Operations

CONUSE Concept of Use

COO Cost of Ownership

COPA Control of Pollution Act

COTS Commercial Off The Shelf

CRETE Common Range Electrical Test Equipment

CSA Customer Supplier Agreement

CSDA Combat System Design Authority

CSED Combat System Engineering Database


Unclassified 211

CSI Combat System Integrator

CSM Combat Systems Manager

CSSC Combat System Support Contractor

CSTs Contractor Sea Trials

CTT Command/Combat Team Trainer

CUPs Capability Update Periods

CVSG Carrier Vertical Strike Guide

CWG Capability Working Group

CWTA Captain Weapons Trials Assessment

D&M Demonstration & Manufacture

D3B Defect and Deficiency Database

DA Design Authority

DAMPs Docking Assisted Maintenance Periods

DARR Design Authority Refit Requirements

DCDS Deputy Chief of Defence Staff

DCIs Defence Council Instructions

DCSA Defence Communication Services Agency

DD Detailed Design

DDR Defence Data Repository

DEC Director Equipment Capability

Def Stan Defence Standard

DER Data Exchange Requirement

DES Data Exchange Specifications

DESO Defence Export Sales Organisation

DIS Distributed Interactive Simulation

DLO Defence Logistics Organisation

DLOD Defence Lines Of Development

DoD Department of Defense (US)

DOSG Defence Ordnance Safety Group

DOSGTS Defence Ordnance Safety Group Technical Support


Unclassified 212

DP Documentation Production

DPA Defence Procurement Agency

DPMGs Defence Procurement Management Guide

DR&A Data Recording, Reduction, Assessment and Analysis

DSA Disposal Sales Agency

DSSO Defence Security Standards Organisation

DStan Defence Standards Organisation

DSTL Defence Science & Technology Laboratories

E3 Electro-Magnetic Environmental Effects

EAC Equipment Approval Committee

EC Equipment Capability

ECC Equipment Capability Customer

ECP Engineering Change Proposal

EDC Equipment Design Contractor

EDI Electronic Data Interchange

EDM Electronic Data Management

EIA Electronics Industry Association

EI&IS Engineering, Interoperability & Information Services

EMC Electromagnetic Compatibility

EMI Electromagnetic Interference

EMP Electromagnetic Pulse

EP Equipment Programme

EPA Environmental Protection Act

EPM Equipment Project Manager

ERC Equipment Reference Code; Event Record Card

ESCROW n. Law. A deed held by a third party until certain conditions are fulfilled.

EU European union

EW Electronic Warfare

FAT Factory Acceptance Trials/Test

FBG Future Business Group


Unclassified 213

FDIP Full Development and Initial Production

FDP Full Development & Production; Flight Data Processing; Full Development Programme; Final Design Phase etc.

FGI Final Guidance Information

FII Final Installation Inspection

FINT Final/Functional Integration Test

FMECA Failure Modes Effects and Criticality Analysis

FOC First Of Class/Full Operational Capability

FON Follow On (Vessels)

FOSF Flag Officer Surface Flotilla

FOSM Flag Officer Submarines

FWA Fleet Weapon Acceptance – (obsolete term) now know as System Acceptance

GA General Arrangement (Drawings)

GFA Government Furnished Assets

GFE Government Furnished Equipment

GFI Guideline For Industry

HASAWA Health and Safety at Work Act

HATs Harbour Acceptance Trials

HCI Human Computer Interface

HF Human Factors

HFI Human Factors Integration

HFIP Human Factors Integration Plan

HLA High Level Architecture

IAB Investment Appraisal Board

ICD Interface Control Documents

IEEE Institute of Electrical and Electronic Engineers

IER Information Exchange Requirements

IFS Interface Specification

IG Initial Gate

IGI Initial/Interim Guidance Information; Installation Guidance Information


Unclassified 214

IGP Installation Guidance Package

II Installation Inspection

ILS Integrated Logistic Support

ILSM Integrated Logistic Support Manager (MoD)

ILSP Integrated Logistic Support Plan

INFOSEC Information System Security

INR Initial/Ionising Nuclear Radiation

IOR Interoperability Requirements

IP Initial Provisions

IPR Intellectual Property Rights

IPT Integrated Project Team

IPTL Integrated Project Team Leader

IRC International Research Collaboration

IS In-Service

ISD In-Service Date

ISMART Interoperable Systems Management and Requirements Transformation

ISO International Standards Organisation

ISP Integrated Support Plan

ISS In-Service Support

ISSP Integrated Supply Support Procedures, Initial System security Policy

ISTAR Intelligence, Surveillance, Target Acquisition and Reconnaissance Equipment

ISTD In-Service Target Date

IT Information Technology

ITEAP Integrated Test Evaluation and Acceptance Plan

ITT Invitation To Tender

JCBM ARTD Joint Command Battlespace Management Applied Research Technology Demonstrator

JDCC Joint Doctrine & Concepts Centre

JETL Joint Essential Task List

JIFM Joint Information Flow Model


Unclassified 215

JMNIAN Joint and Multi-national Interoperability Assessment Network

JSPs Joint Service Publications

KSRs Key System Requirements

KURs Key User Requirements

LAN Local Area Network

LBTS Land Based Test Sites

LCC Life Cycle Costing

LDs Liquidated Damages

LODs Lines of Development

LORA Level Of Repair Analysis

LPD(R) Landing Platform Dock (Replacement)

LPH Landing Platform Helicopter

LSA Logistics Support Analysis

LSAR Logistics Support Analysis Record

LSRC Land Systems Reference Centre

LTC Long Term Costing

M&S Modelling and Simulation

MC Military Capability

MCTA Maritime Commissioning, Trials & Assessment

MDAL Master Data and Assumptions List

METL Maritime Essential Task List

MG Main Gate

Mil Std Military Standard (US)

MILCAP Military Capability

MLU Mid Life Update

MMI Man Machine Interface

MoD Ministry of Defence

MODAF MoD Architectural Framework

MODAR MoD Architectures Repository

MOE Measures of Effectiveness


Unclassified 216

MOP Measures of Performance

MRI Master Records Index

MTA Maintenance Task Analysis

NATO North Atlantic Treaty Organisation

NDIs Non-Development Items

NEC Network Enabled Capability

NSC Naval Support Command

NWHT Naval Weapon Harbour Trial

NWST Naval Weapon Sea Trial

OA Operational Analysis; Order Administration

OME Ordnance, Munitions, Explosives

OPCON Operation Control

OPDEF Operational Defect

ORBAT Organisation for Battle

OSD Out of Service Date

OTS Off The Shelf

PC Personal Computer

PCT Performance, Cost, Time

PD Project Definition; Preliminary Design

PDC Platform Design Contractor

PDG Procurement Development Group

PDS Post Design Services

PECs Printed Electronic Circuits

PFA Preparation For Acceptance

PFG Procurement Finance Group

PINT Primary Integration Test

PM Pivotal Management; Project Manager

PMP Project Management Plan

PMS Planned Maintenance Schedule

POEMS Project-Oriented Environmental Management System


Unclassified 217

POSMS Project-Oriented Safety Management System

PP Procurement Planning

PPM Platform Project Manager

PR&A Project Review and Assurance

PSU Power Supply Unit

QA Quality Assurance

QAR Quality Assurance Representative

R&D Research and Development

RADHAZ Radiation Hazard

RAF Royal Air Force

RAMP Requirements and Acceptance Management Plan

RAO Research Acquisition Organisation

RCM Reliability Centred Maintenance

RDB Requirements Database

RE Requirement Engineering

RFA Royal Fleet Auxiliaries

RFQ Request For Quote

RM Requirement Manager

RMP Risk Management Plan

RN Royal Navy

RRAP Risk Reduction Action Plan

S&TE Support and Test Equipment

S+T Science and Technology; Swiftsure and Trafalgar (Class Submarines)

SA Systems Analysis, System Acceptance

SAG Studies Assumption Group; Standardisation Advisory Group

SATs Sea Acceptance Trials

SD Systems Design

SDE Shared Data Environment; Shared Development Environment

SDF Shore Development Facility

SE Systems Engineering


Unclassified 218

SEISP System Electronic Information Security Policy

SEMP Systems Engineering Management Plan

SID System Interface Diagram

SIF Shore Integration Facility

SINT Secondary Integration Test

SIP Standardisation Implementation Plan

SMH Safety Management Handbook

SMO Safety Management Office

SMP Standardisation Management Plan; Self Maintenance Period; Safety Management Plan

SOW Statement Of Work

SPS Special Procurement Services

SRD Systems Requirements Document

SRL System Readiness Level

SRM Supplier Relationship Management; Stakeholder Responsibility Matrix

SS System Studies

SSE Support Solutions Envelope

SSIs Shipbuilder Supplies Item

SSMS Ship Safety Management System

SSON Single Statement of (User) Need

SSP Sea Systems Publication

STG Sea Technology Group

STGP Sea Technology Group Publications

STIR Statement of Technical Information Required

STP Short Term Programme

STTE Special To Type Equipment

STW Setting To Work

SWUPI Ship Weapons Upkeep Planning Information

SyOPs Security Operating Procedure

SYSINT (Staged) System Integration Test


Unclassified 219

T23 Type 23 Frigate

T42 Type 42 Destroyer

T45 Type 45 Destroyer

T&E Test and Evaluation

TDL Tactical Data Link

TDP Technology Demonstrator Programmes

TES Technical Enabling Service

TES-SSG Technical Enabling Services – Sea Systems Group

TFI Test Form Index

TLC Through-Life Cost

TLMP Through-Life Management Plan

TPMs Technical Performance Measures

TRL Technology Readiness Level

TS Technical Support

TTCP The Technical Co-operation Program

TULIP Through Life Interoperability Planning (Predecessor of ISMART)

UK United Kingdom

UNECE United Nations Economic Commission for Europe

URD User Requirement Document

URS User Requirement Specification

US United States

VCDS Vice Chief of the Defence Staff

VCRI Verification Cross Reference Index

VFM Value for Money

WAN Wide Area Network

WBS Work Breakdown Structure

WEDP Weapon Equipment Demonstration Programme; Weapon Equipment Delivery Programme

WEMIT Weapons Electrical Mutual Interference Trial

WES Weapon Equipment Schedule


Unclassified 220

WLC Whole Life Costs

WPM Warship Project Manager

WRFS Weapon Repair Facility Statements

WSA Warship Support Agency

WSAC Weapon System Acceptance Committee

WSAF Warship Support Agency Form

WSIA Weapon System Integration Authority

WSM Weapon System Manager


Unclassified 221

Inside rear cover

©Crown Copyright 2006

Copying Only as Agreed with DStan

Defence Standards are Published by and Obtainable from:

Defence Procurement Agency

An Executive Agency of The Ministry of Defence

UK Defence Standardization

Kentigern House

65 Brown Street

GLASGOW G2 8EX

DStan Helpdesk

Tel 0141 224 2531/2

Fax 0141 224 2503

Internet e-mail [email protected]

File Reference

The DStan file reference relating to work on this standard is .

Contract Requirements

When Defence Standards are incorporated into contracts users are responsible for their correct application and for complying with contractual and statutory requirements. Compliance with a Defence Standard does not in itself confer immunity from legal obligations.

Revision of Defence Standards

Defence Standards are revised as necessary by an up issue or amendment. It is important that users of Defence Standards should ascertain that they are in possession of the latest issue or amendment. Information on all Defence Standards can be found on the DStan Website www.dstan.mod.uk, updated weekly and supplemented regularly by Standards in Defence News (SID News). Any person who, when making use of a Defence Standard encounters an inaccuracy or ambiguity is requested to notify UK Defence Standardization (DStan) without delay in order that the matter may be investigated and appropriate action taken.