Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Verification and Validation ofMission Data Libraries for
Electronic Warfare Operational Support
Flight Lieutenant Charles WinsorRoyal Australian Air Force
Dr Thomas MillhouseNova Systems, Australia
Introduction
Electronic Support systems depend on libraries of accurate, current and relevant mission data to identify emitters and provide tactical threat warning to aircraft and ships
Electronic Warfare Operational Support is the process of updating these mission data libraries to adapt to changes in the operational environment
Introduction
Library reprogramming has historically been
treated like software development
Software engineering standards demand
time-consuming Verification and Validation
for mission critical systems
Verification and Validation provide assurance
that an Electronic Support system will perform
in its intended operational environment
Introduction
Excessive Verification and Validation
may compromise the essential
responsiveness of Electronic Warfare
Operational Support
We propose a new approach to balance
the rigour of software engineering with
the responsiveness of Electronic
Warfare Operational Support
Outline
Electronic Support
Mission Data Libraries
Electronic Warfare Operational Support
Verification and Validation
Finding the Balance
A New Approach
Electronic Support
Electronic Support is the branch of
Electronic Warfare that deals with the
passive intercept, analysis and
exploitation of electromagnetic radiation
Electronic Support is used in modern
warfare for surveillance, intelligence
collection and tactical threat warning of
radar-guided weapons
Electronic Support
Electronic Support systems aim to:
intercept and detect signals
locate, identify and report emitters
ReceiverSignal
ProcessorOperatorDisplay
Mission DataLibrary
INTERCEPT
DETECT
LOCATE
IDENTIFY
REPORT
Mission Data Libraries
Most Electronic Support systems are loaded pre-mission with reprogrammable mission data libraries
These libraries enable automatic identification of detected emitters
High quality libraries are essential to achieve timely threat warning: manual emitter identification by an
operator is slow
fast reactions needed against threats
Mission Data Libraries
Emitter parameters form the basis of a mission data library:
enables identification of emitters from detected signals
Emitter-to-platform associations improve situational awareness:
identified emitters may be correlated to known platforms or locations
Additional information can be associated to emitters to improve situational understanding
Mission Data Libraries
Mission data quality characteristics:
Accuracy
Currency
Relevance
All share a common dependence
on time
All are interdependent
Mission Data Libraries
Mission data libraries are typically specified to suit an Area of Operations: libraries should be updated as emitters enter or
leave the specified region
Mission data is typically based on observed emitter parameters: libraries should be updated as emitter
parameters change
particularly important as we enter the era of software-defined radio
Consequences
No identification:
no threat warning
countermeasures not initiated
platform loss
Misidentification or ambiguous identification:
false alarm
wrong countermeasures initiated
mission abort
EW Operational Support
AimTo update libraries before they become ineffective or
unsuitable for operational use
ProblemThe electromagnetic environment may change significantly
on a daily basis
ChallengeTo complete post-mission analysis, update library
requirements, develop an updated library, and verify and validate that library to a suitable extent… all before the
current library becomes an operational liability!
EW Operational Support
ANALYSE
REPROGRAM
OPERATE
Pre-MissionUpload
Post-MissionDownload
MissionReplay
ResolveAnomalies
RecommendChanges
DesignVerifyDevelop
Validate
Verification & Validation
Australian Technical Airworthiness regulations treat mission data libraries as software (not as data):
similar regulations are applied in the Australian maritime and land domains
Software engineering standards like DO-178B are applied to routine library reprogramming:
threat warning sensors are mission critical systems
libraries may be assessed to need high Software Assurance Levels
Assurance of mission critical software demands time-consuming Verification and Validation
Verification & Validation
VerificationThe evaluation of whether or not a product complies with a requirement specification“The library was developed correctly”
ValidationThe assurance that a product meets the needs
of the customer“The correct library was developed”
[IEEE 1490-2011]
Verification & Validation
Verification and Validation are important to
provide assurance that mission data
requirements are satisfied by a library...
...but those mission data requirements change
whilst library Verification and Validation is
being conducted
An optimal balance must be found between
rigour and responsiveness
Finding the Balance
1. Programming errors
found and resolved
2. System performance
optimisation
3. Mission data currency
degradation
4. Mission data
relevance degradation
Verification
Software-only testing:
basic error checking
basic performance estimation
“Was the library implemented as designed?”
Hardware-in-the-loop testing:
advanced error checking
advanced performance estimation
assess emitter interactions
“Does the library design satisfy its requirements?”
Limitations of Verification
The same mission data is used to develop and
verify the library
The performance of a library is fundamentally
limited by the quality of the available mission
data
Verification predicts library performance
assuming mission data is correct
Library verification provides no assurance of
underlying mission data quality
Validation
Field testing:
“Were the library requirements fit for purpose?”
Options:
controlled or mission conditions?
generic test scenario or mission rehearsal?
emitters of interest only, or “background
emissions” as well?
Limitations of Validation
High cost
If emitter emulators are used, same fundamental limitation as verification:
predicts library performance assuming emitter emulation is correct
If authentic emitters are used, still not a true representation of the operational environment:
no two radars emit exactly the same signal
when does “close enough” become “good enough”?
A New Approach
Verification and Validation are activities
performed as part of the library reprogramming
process
Library reprogramming is an activity performed
within the EWOS cycle
What if we treated the entire EWOS cycle as a
holistic Validation activity?
Can rapid, continuous execution of the EWOS
cycle fulfil the aim of Validation?
EWOS as Validation
Collect “test data” from operations: true mission conditions emitters of interest background emitters
Assess library performance in the operational environment through post-mission analysis
Assess the ongoing suitability of library requirements: “Were yesterday's library requirements still valid
during today's mission?” “If not, what needs to change?” “And how else do we expect tomorrow's
mission to change those library requirements?”
EWOS as Validation
Does rapid, continuous execution of the EWOS cycle mitigate the need for a formal Verification and Validation process?
A low quality library can still cause mission failure or the loss of life and materiel:
focussed testing of high risk emitters
Potential to compound errors:
initial library version must be of reasonable quality
Similar principles to the Spiral Model for software development [Boehm, 1988]
Implementation
Holistic consideration of Verification, Validation
and the entire EWOS cycle
Organisations, personnel and infrastructure
capable of sustaining highly responsive EWOS
Expertise to identify mission risks within
libraries and tailor Verification and Validation
processes to specifically address those risks:
instead of blanket-application of generic
software engineering standards to EWOS
Into the Future
Mission data library development processes for Electronic Support systems are currently similar to the Waterfall Model [Royce, 1970]
Rapid, continuous execution of the EWOS cycle could be considered similar to the Spiral Model [Boehm, 1988]
Many new software development models have emerged in the last few decades
Is it time for EWOS to move into the twenty-first century?
Conclusion
Libraries of accurate, current and relevant
mission data are essential to enable the
performance of Electronic Support systems
Low quality mission data libraries can result in
mission failure and other serious
consequences
Assurance of library quality is conventionally
achieved through application of software
engineering rigour
Conclusion
Conducting extensive Verification and
Validation processes to assure library
quality can actually reduce performance
The EWOS cycle must be responsive to
keep pace with an evolving
electromagnetic environment
A better balance must be found between
rigour and responsiveness
References
D.C. Schleher (1999), Electronic Warfare in the Information Age
R.G. Wiley (2006), Electronic Intelligence: The Interception and Analysis of Radar Signals
DO-178B, Software Considerations in Airborne Systems and Equipment Certification
IEEE 1490-2011, A Guide to the Project Management Body of Knowledge
W. Royce (1970), “Managing the Development of Large Software Systems”
B. Boehm (1988), “A Spiral Model of Software Development and Enhancement”
Verification and Validation ofMission Data Libraries for
Electronic Warfare Operational Support
Flight Lieutenant Charles WinsorRoyal Australian Air Force
Dr Thomas MillhouseNova Systems, Australia