1 Operational Test & Evaluation AugustaWestland 101 Air Operations Inspectorate Concept: Acquisition – Acceptance – OT&E – Education & Training – Operations

1

Operational Test & EvaluationAugustaWestland 101

Air Operations Inspectorate

Concept: Acquisition – Acceptance – OT&E – Education & Training – Operations

Brief for the Sola Pre-Conference by Col Alfheim CO OT&E AW101

2015-09-21Mission First, Safety Always

LuftforsvaretOT&E AW101

2

Luftforsvaret

Roadmap

• Introduction and Historical Back Ground– Why OT&E– NAWSARH

• OT&E AW101– The Concept– The OT&E-process– Education & Training– Basing and Operations– Risk and Safety– Simulation

• Summary• Q&A

Mission First, Safety Always 2015-09-21

Luftforsvaret

3

OT&E AW101

Why OT&E?

1951 – 1958 Phase Tests

1. Airworthiness (Contractor)

2. Contractor Compliance (AF)

3. Design Refinement (Contractor)

4. Performance and Stability (AF)

5. All Weather (AF)

6. Functional Development (AF)

7. Operational Suitability (User)

8. Unit Operational Employment Testing (User)

1958 – 1972 Category Tests

1. Subsystem Development & Initial Airworthiness ”Prototype or early production aircraft” (Contractor)

2. System Development & Integration of subsystems into complete aircraft ”Early production aircraft” (AFFTC and User)

3. Operational ”Production aircraft” (User with some contractor support)


• OT&E tended to be a QA evaluation after the procurement decision had been made

• Poor performance of weapon systems during Vietnam War


4

Luftforsvaret

Why OT&E ..

• US Congress mandated that an independent, operational test and evaluation be done prior to a production decision.

• Each US military service established an Operational Test Agency reporting directly to their Chief of Staff

• Independent of program office, DT&E, contractor…• Current Users—operators and maintainers• Evaluation done in Real world conditions

• Joint/Integrated Test Teams - Contractor, DT&E, and User Combined Test Forces



5

Luftforsvaret

Evolutionary Test Strategy


Concept Technology Development

System Development

Production & Deployment

Operations & Support

User Needs, Technology, Opportunities, and Politics…

IOC FOCDT&E

OT&E

EOA

Integrated DT&E and OT&E

TD&EIOA FOT&EIOT&

E

AT&EQT&E


6

Luftforsvaret

Kebnekaise…


Luftforsvaret

7

OT&E AW101

OT&E Purpose

Determine system operational effectiveness & suitability• Operationally realistic

conditions• Operated and maintained by

typical Users• Production representative

systems• Using delivered equipments,

training, data, mission planning, etc

• Use of certified models, simulations, and data bases

Answer Critical Operational Issues• Documenting deficiencies• Recommend system

enhancements• Provide information for

refining:– Logistics support requirements– Software support– Training– Doctrine– Tactics,Techniques, and

Procedures


All hardware/software modifications that change delivered system performance must be tested and evaluated

Luftforsvaret OT&E AW101LOI

8

Air Force Regulations



9

Luftforsvaret

OrganizationSquadron• “Small/simple OT&E”• P-3 MX-20 (EO/IR) og TCDL• Raven UAS

Multi National OT&E• “Same type config” • F-16 (M6.5)• F-35• C-130J (Block 7.0/8.1)

Dedicated OT&E unit• “Large/complex OT&E” • NH90 (2010)• AW101 (2015)



10

Contract signed 19. Dec 2013

MoJ & AW

NAWSARH-PROJECT – TIMELINE

• “Pre-phase” (White paper, studies, studies new studies, NH 90 option etc, etc) 1997 – 2007• Pre-study (“QA1 study & QA2 study”) 2007 – 2010• Parliament decision Dec 2010• Competition Timeline:

• Pre Qualification (AW, EC, SK, NHI, RHI) Dec 2011

User Needs, Technology, Opportunities, and Politics



11

Luftforsvaret

BACKGROUND - CONTRACT

• AW101 – 612 to replace Sea King by end of 2020

– First delivery early 2017

– RNoAF to remain the operator

• SAR-service to be maintained on all 6 bases

• «Turn Key provider» solution

– Availability as main KPI

– All new Integrated Logistic Support solution (PBL)

– Train-the-trainer concept.


Luftforsvaret

Full Ice Protection 30minute Run Dry Transmission

Sliding door, Crew Door & Ramp

Ground Clearance Ca. 500mm

183 cm Cabin height

3 Observation Windows

AW101-612 – KEY FEATURES

Active Vibration Attenuation (ACSR)

30minute Dry Run capability XMSNs

BERP III MRB

3 Observation Windows

Crashworthy Structure, Seats,

Fuel System

AW101-611 “CSAR” as baseline for

certification

Luftforsvaret

No folding MRB

Engine Configuration (3x GE CT7-8E)

New Automatic Flight Control System*

Fully articulated Tail Rotor (NEW)

New AESA 3x Flat Panel Radar

Low maintenance workload program

* Including Hover Trim Controller (HTC)

AW101-612 – KEY FEATURES (NORWAY)

Luftforsvaret

DETECTION AESA Radar (Simultaneous Modes)

Independent Weather Radar HD Electro Optics/IR Sensor DF & Mobile Phone Detection

TERRAIN AWARENESS HTAWS (Fully Integrated)

3 x Digital Map (Elevation Data) Integrated Synthetic Vision

Low Height System (2 x Rad Alt) Configurable S.A. Overlays

OBSTACLE DETECTION Laser Obstacle Warning System

Obstacle Proximity LiDAR Integrated Synthetic Vision

AREA NAVIGATION 3 x Integrated Digital Maps

High Integrity Navigation Suite 2 x Civil GPS, Integrated RadNav

Automatic Nav. Accuracy Monitoring Jeppesen Database

VFR / IFR Navigation AFCS Modes

TRAFFIC AWARENESS Integrated TCAS II (Overlay on

Map) ADS-B Tx Transmissions

AIS Tx/Rx Capability IFF Transponder

Growth Option for Link 16

AW101-612 – KEY SA FEATURES

Luftforsvaret

SAR LAYOUT

Flexible Cabin Arrangement

Wireless Passenger & Crew Intercom

Crew Galley & Relief Facilities

Rear Ramp Stowage Area

Adjustable, Rotating & Traversing Crew

Seats

Flexible / Adaptable Medical Rail System

Wet Storage Adjacent to Rescue

Station

Stretcher Compatibility with

Ground Ambulance

Dedicated System Operator Console

Area

Dual Trauma Station (Gas & Air Provisions)

AW101-612 – KEY CABIN FEATURES


16

Luftforsvaret

ILS

P7661

HRS - communication

Training

SAR Bases

Hospital Helipads

GFE / GFA

Florø

Helicopters

The NAWSARH Project



17

Luftforsvaret

OT&E AW101

Sola (40)

COM IAO

OT&E

CoS RNoAF

Air Staff

Training/Sim

Tech/Maint

P7661Air Sys /Heli

LogisticsNAWSARH



18

Luftforsvaret

NAWSARH / OT&E AW101 - timeline

• 2014 PDR• 2015 CDR • 2016

Planning, preparation, and training for The Mission:

1. OT&E AW101

2. Train 330 SQN


• Initial OT&E AW101 org• Final OT&E AW101 org


19

Luftforsvaret


”The Big Picture”2015 2016 2017 2018 2019 2020 2021

1 Initial

FFS and Helicopters

OT&Ephases

TrainingSchedule (Base based)

3 Final

2 Follow On

Train 330 SQN


20

Base 0 Sola#1 2017 OT&E and Training

Base 1 Sola#1 2018 Base 2 Ørland#2 2018

Base 3 Banak#1 2019

Base 4 Bodø#2 2019

Base 5 Rygge#1 2020

Base 6 Florø#2 2020

1 - 2018

2 - 2018

1 - 2019

2 - 2019

1 -2020

2 - 2020

Sea King RoA 20 rescuees~ 53 nm

Sea King RoA 2 rescues ~ 210+ nm

AW101 RoA 2 rescuees. >310nmRoA 20 rescuees >250 nm



21

Luftforsvaret


”A few details”2015 2016 2017 2018 2019 2020 2021

1 Initial

FFS and Helicopters

OT&Ephases

UKUK / ZVCN / ZVTrainingSchedule (Base based)

3 Final

2 Follow On

ZVOL

NA

BORY

FL


22

Luftforsvaret

OT&E Objectives

• Determine the AW101 helicopter actual capabilities, limitations, operational effectiveness and suitability

• Determine if the requirements in the Statement Of Intent (SOI) have been met

• Verify that the AW101 helicopter is properly manned and equipped

• Verify that the quality of training and logistics meet operational requirements for employment as a SAR system (operational suitability)

• Develop tactics, techniques and procedures (TTPs) • Establish the basis for further development of

organization, systems, TTPs by follow on OT&E activities at a later stage



23

Luftforsvaret

AW101 OT&E Phases

Phase 1• Initial – IOT&E (Basic SAR capability day & night).

Phase 2• Follow on – FOT&E (Full SAR capability day &

night/all weather. It will run in parallel with TR/TQP).

Phase 3• Final – FOT&E is planned to establish the full potential

of the AW101 for the remaining capabilities for military missions, support to special police missions, specialized firefighting, emergency relief, environmental protection, and other missions of public interest aimed at protecting assets or the environment.



24

Luftforsvaret

Critical Operational Issues (COI)

• COI-1: Is the AW101 system capable and available to meet national readiness requirements?

• COI-2: Can the AW101 system provide Search and Rescue capability in the Norwegian maritime environment?

• COI-3: Can the AW101 system provide Search and Rescue capability in the Norwegian land environment?

• COI-4: Can the AW101 system provide Helicopter Emergency Medical Service (HEMS) capability across NSRR?

• COI-5: Is the AW101 system capable of conducting and supporting national contingency operations?



25

Luftforsvaret

COI/MOE-MOS/MOP Structure


COI 1COI 1

COI 1COI 1

COI 5

MOS 1MOE 1COI 1

COI 1COI 1

COI 1MOS 5.6

MOE 1MOE 5.3

MOP 5.3.2 MOP 5.6.1 MOP 5.6.2MOP 5.3.1

Critical Operational Issue (COI)

Measure of Effectiveness (MOE) Measure of Suitability (MOS)

• Measure of Performance (MOP)

» Demonstrate – show operation» Determine/Measure - quantitative» Assess/Evaluate – qualitative

» Verify – quantitative versus ORCD» Determine – quantitative versus prediction


26

Luftforsvaret

RNoAF OT&E Tool

SharkScribe• Cooperation w/HOTEF, Canada

Aim• Tool to manage OT&E activity

• Automates OT&E process– Make test concept– Plan test activity (Sorties)– Log test data– Generate test card– Generate reports

• Standardize processes– COI – Critical Operational Issue– MOE – Measure of Effectiveness– MOS – Measure of Suitability– MOP – Measure of Performance / (Assessment)



27

Luftforsvaret

Conclusion Recommendation Terminology

ConclusionRecommendat

ionTerminology

RecommendationLevel

ACCEPTABLE No action No action

ACCEPTABLE / SATISFACTORY

SomethingCOULDbe done

DESIRABLE

ACCEPTABLE / UNSATISFACTOR

Y

SomethingSHOULDbe done

HIGHLYDESIRABLE

UNACCEPTABLE

SomethingMUST

be doneESSENTIAL



28

Luftforsvaret

AW101 OT&E Reporting


• CO OT&E AW101 is reporting to COM IAO after the completion of each campaign, COI and phase.

• COM IAO reports to CoS RNoAF the AW101 helicopter’s actual capabilities, operational effectiveness and operational suitability.

• Chief of Staff RNoAF will decide, based on the recommendations derived from the OT&E process if, how, and when the RNoAF can deploy the AW101 helicopter for live operations.

CO OT&E AW101 COM IAO CoS RNoAF

Luftforsvaret

29

OT&E AW101

Credible– Objective– Unbiased

Realistic– Typical operators– Realistic environment– Delivered equipments


Cost Effective– Efficient– Effective– Timely


30

Luftforsvaret

OT&E Tenents

• Objective, Dedicated and Independent– Representing the operational users– Separate from developing, procuring, and contractors– Minimize influence of acquisition managers

• Integral part of combined test teams– Involved as early as possible with DT&E



31

Luftforsvaret

AW101 OT&E

• Introduction of a new system tends to introduce new risks – OT&E Effectiveness and Suitability testing tends to be

hazardous– More people injured in other tests than in flight tests…

• Commence testing from a known and safe point• Incremental approach• Conduct a risk management analysis• Pre-determine actions for undesirable results• Have defined “knock-it-off” test criteria• Analyze results before proceeding to next test point



32

Luftforsvaret

RiskMission First, Safety Always


1 Identify Risk (Hazard)2 Analyze Risk (Hazard) – Cause & effect3 Eliminate/Control Risk (Hazard) – minimize probability & severity4 Establish emergency procedure


33

Luftforsvaret Risk MatrixMission First, Safety Always

RAC 1: Extremely High (R<=25)RAC 2: High (R<=20)RAC 3: Medium (R<=15)RAC 4: Low (R<=10)RAC 5: Acceptable (R<=5)


Experience/Expertise

Residual Risk =Assumed + Unidentified


34

Luftforsvaret

OT&E focus areas



35

Luftforsvaret

System Concepts – Hardware/Software

• Segregated

• Federated– Interfaces between subsystems– 1553 bus (typical)

• Integrated

• Complex• More difficult to relate cause and effect

– Automatic Diagnostic System (ADS)



36

Luftforsvaret

Simulation (FFS)

• Helps to understand the system• Predict performance• Increased efficiency / Reduce time and cost

Limited test articles / locations• Provide otherwise unattainable and immeasurable

data (Safety)• Be prepared – “What if?” Plan a “build-up” approach• Develop and train for test techniques• Development of test objectives, constrains, and criteria

Simulation models are designed to provide answers at a given abstraction level



37

Luftforsvaret

”In Short”

Concept for fielding of the NAWSARH AW101 in RNoAF

• OT&E AW101 will as a dedicated unit:– Conduct OT&E planned in three phases (IOT&E, FOT&E, and

“Final OT&E”)– Train 330 Sqn personnel (in phase two) to stand up one

SAR-base at the time (ZV-OL-NA-BO-RY-FL)

– Utilize FFS at ZV for:• Initial training TR/TQP• OT&E msn prep• Annual training ATP/CTP

– Support the safe and efficient introduction of AW101 into service


38

Operational Test & EvaluationAugustaWestland 101

Air Operations Inspectorate

OT&E AW101: “To Transform 40+ yrs of Sea King SAR-experience to AW101 in 4 yrs”

Brief for the Sola Pre-Conference by Col Alfheim CO OT&E AW101

2015-09-21Mission First, Safety Always

Q&A

Documents

1 Operational Test & Evaluation AugustaWestland 101 Air Operations Inspectorate Concept: Acquisition – Acceptance – OT&E – Education & Training – Operations