EMIS 7307

T&E Part 2

1

• Documents in flux. • MNS - Mission need statement

– Non system specific, a needed capability.

• Being replaced by Initial Capabilities Document (ICD). – ICD identifies the capability gap, describes the

evaluation of different approaches and proposes an approach to pursue. - Supports the MS A decision.

DOD 5000 Changes

EMIS 7307

T&E Part 2

2

• ORD - Operational requirements document• Minimum acceptable capability of possible system.

• ORD is being replaced by Capabilities Development Document (CDD) at MS B and Capabilities Production Document (CPD) for MS C.

• CDD- Contains operational performance parameters to design and build an affordable and useful increment includes KPPs (Key performance parameters).

• CPD - Contains production attributes of a single increment.

DOD 5000 Changes

EMIS 7307

T&E Part 2

3

Revised DoD 5000

ICD CDD CPD

EMIS 7307

T&E Part 2

4

DoD 5000 with planned upgraded capabilities called increments. A spiral is used when

upgrades are expected but are TBD.

AoA Analysis of Alternatives

DAB Defense Acquisition Board

JROC Joint Requirements Oversight Council

EMIS 7307

T&E Part 2

5

A significant number of documents have relevancy to T&E.

EMIS 7307

T&E Part 2

6

Test Documentation Relationships

Analogous to A, B and C spec relationships

EMIS 7307

T&E Part 2

7

EMIS 7307

T&E Part 2

8

Test Program Documentation

EMIS 7307

T&E Part 2

9

NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS) Test

and Evaluation Master Plan (TEMP)

EMIS 7307

T&E Part 2

10

NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS)

TEMP

EMIS 7307

T&E Part 2

11

NATIONAL POLAR-ORBITING OPERATIONAL ENVIRONMENTAL SATELLITE SYSTEM (NPOESS)

TEMP

EMIS 7307

T&E Part 2

12

Test Program Documentation

• System Test Plan– Describes ‘big picture’ of all the testing for the whole system,

not a management document.

– This is the foundational technical test document from which subservient plans are derived.

– Includes laboratory, component, subsystem, system, software etc.

• Detail Test Plan– In a large system, each of these topics (laboratory,

component, subsystem, system, software etc.) is likely to be a volume or a book of its own.

EMIS 7307

T&E Part 2

13

EMIS 7307

T&E Part 2

14

Other Documents

• Test Procedures– Implements a specific test with carefully constructed step

by step directions.– Completeness is paramount as unplanned steps may

invalidate the data being collected. – May be augmented by test cards or scripts.– Formality in execution may require QC and/or customer

sign-off.– Dry run - find problems in procedure or UUT and fix.– Document changes during execution.

EMIS 7307

T&E Part 2

15

Other Documents

• Test Reports– Quick look - very important for management

• Based on observation and limited, if any, data analysis

• May affect next day’s testing - consider email?

– Final report includes:

EMIS 7307

T&E Part 2

16

Test/Program Interrelationships

EMIS 7307

T&E Part 2

17

More about types of testing.

• DT&E again.

• Contractor perspective.– Demonstrates design and development and in

the end-game, completeness i.e. meets spec.– Reduce risks.

• High-risk components.

• Intra-subsystems.

• Inter-subsystem.

EMIS 7307

T&E Part 2

18

• DT&E government perspective.– Demonstrates spec compliance.– Used for acceptance of the EDM(s).– Estimate the military utility.– Confirms ready for field testing (OT&E).

EMIS 7307

T&E Part 2

19

• DT&E by phase.– Prior to program start.

• Determine or at least estimate that the technology is feasible.

– Concept and technology development.• Assist in selecting between possible alternative

system concepts.

• Info for SRR and program start decision.

EMIS 7307

T&E Part 2

20

• DT&E by phase.– During System Development and Demonstration.

• Risks can be reduced to acceptable levels.• From here on engineering efforts instead of experimental

efforts.• Supports decision to build an EDM.• System I&T is a part of this.• Includes EDM acceptance testing.• Includes the testing of the EDM to provide data for the

LRIP decision.

EMIS 7307

T&E Part 2

21

• DT&E by phase.– During production and deployment phase.

• Prelude to IOT&E.

• Navy calls TECHEVAL.

– During full rate production and operations.• To verify corrections and/or improvements.

• In terms of calendar time and manhours consumed, DT&E is larger than OT&E and has mostly contractor involvement.

EMIS 7307

T&E Part 2

22

Another view of DT&E

EMIS 7307

T&E Part 2

23

DT&E For the MSs

EMIS 7307

T&E Part 2

24

OT&E is complex in it’s simplicity!

EMIS 7307

T&E Part 2

25

Types of OT&E

• Early Operational Assessment (EOA).– Performed for the Program Office and other

decision makers as a indicator of how operational things are “shaping up”.

– Done by operational testers but may be mostly analytical based on DT&E provided info.

• Initial OT&E– Supports full rate production decision.– Called OPEVAL by Navy.

EMIS 7307

T&E Part 2

26

Types of OT&E cont’d.

• Follow-on OT&E (FOT&E)– After full rate production.– Conducted during fielding.– Checks on correction of IOT&E deficiencies.

EMIS 7307

T&E Part 2

27

More comparisons

EMIS 7307

T&E Part 2

28

Lessons for the PM that you should know too.