Using the Performance Specification Process In Hazard Elimination and Control

Using the Performance Specification Process In Hazard Elimination and ControlPamela Wilkinson, MS

Technical Director, System Safety ESOH

1Background

Requirements Analysis

Performance Specification

Conclusion

Questions

Agenda2BackgroundPrior to 1994:Detail Specifications

Detailed the method for how something was built.

Proscriptive

1994 Present:Performance Specifications

Defines the functional requirements for the product, the environment in which it must operate, and interface characteristics.

States requirements in terms of the required results.

Does not state the methods for achieving the required results.3Performance SpecificationDefines the expected result

Ship

Written requirement that describes the functional performance criteria required for a particular equipment, material, or product.

4Hazard MitigationSo how does this relate to mitigating hazards?

If the contractor is required to meet specified safety performance requirements, they will more carefully select equipment in the design phase that will do so.5Requirements AnalysisSystem Safety related requirements oftenvague or missing.

Requirements are rarely defined explicitly.

MIL-STD-882 is an mandatory Requirement per DODI 5000.02. JCIDS DocumentsDerived - Implied requirements or requirements that must be followed due to higher level requirementsAllocated - A requirement that is established by dividing or otherwise allocating a high-level requirement into multiple lower-level requirements.6

Requirements AnalysisJCIDS Documents

KPPs

Federal Laws (EPA)

Military Standards (MIL-STD-1472G)Industry Standards (IEEE)

User needs/Operational RequirementsDetermine what existing requirements applyJoint Capabilities Integration and Development System (JCIDS)Key Performance Perameters

Correlate with the programs Requirements Traceability Matrix7Requirements Analysis Must be done in conjunction with other disciplines Requirements Traceability Analyses

Expect there to be Trade-offs to eliminate conflicts.

Likely will need to adjust some requirements to meet Cost, Schedule & Performance ConsiderationsCustomer Expectations - (What the customer wants the system to accomplish)Project & Enterprise Constraints - (Cost, schedule, available manpower, management decisions, etc.)External Constraints - (Available technology, public and international law, external equipment)Measures of Effectiveness - (Mission performance, safety, reliability, etc.)Measures of Suitability - (Maintainability, ease of use, etc.)System Boundaries - (What systems or components are under assessment, what falls outside of the area of control?)Interfaces - (What other equipment is necessary for the component or system to operate? What tools are necessary for maintenance or test?)Utilization Environments - (Weather, temperature extremes, vibration, noise, operational time of day, etc.)Lifecycle - (Operations, maintenance, test, disposal, etc.)Functional Requirements - (What the system must accomplish) Performance Requirements - (How the system must perform)Modes of Operation - (Types of operations and conditions under which they must operate)Technical Performance Measures - (Thresholds & objectives)Physical Characteristics - (Size, weight, type of coating, etc.)Human Systems Integration - (Noise, lighting, reach, space limits, ergonomics, etc.)

8

Requirements Analysis Where will the system be used?How will the system accomplish its mission objective?What are the critical system parameters to accomplish the mission?How are the various system components to be used?How effective or efficient must the system be in performing its mission?How long will the system be in use by the user?What environments will the system be expected to operate in an effective manner?

Basic Questions to ConsiderWhere will the system be used?How will the system accomplish its mission objective?What are the critical system parameters to accomplish the mission?How are the various system components to be used?How effective or efficient must the system be in performing its mission?How long will the system be in use by the user?What environments will the system be expected to operate in an effective manner?

Customer Expectations - (What the customer wants the system to accomplish)Project & Enterprise Constraints - (Cost, schedule, available manpower, management decisions, etc.)External Constraints - (Available technology, public and international law, external equipment)Measures of Effectiveness - (Mission performance, safety, reliability, etc.)Measures of Suitability - (Maintainability, ease of use, etc.)System Boundaries - (What systems or components are under assessment, what falls outside of the area of control?)Interfaces - (What other equipment is necessary for the component or system to operate? What tools are necessary for maintenance or test?)Utilization Environments - (Weather, temperature extremes, vibration, noise, operational time of day, etc.)Lifecycle - (Operations, maintenance, test, disposal, etc.)Functional Requirements - (What the system must accomplish) Performance Requirements - (How the system must perform)Modes of Operation - (Types of operations and conditions under which they must operate)Technical Performance Measures - (Thresholds & objectives)Physical Characteristics - (Size, weight, type of coating, etc.)Human Systems Integration - (Noise, lighting, reach, space limits, ergonomics, etc.)

9Requirements must be Verifiable

Allows the contractor to know what the government meant by the requirement.

Allows the government to know the contractor has achieved user requirements.

Requirements/P-SpecsThe prospective vendors are free to meet the requirements in any way they can. They can offer any material conforming to the specification, as long as the products they offer meet the performance criteria established in the specification.10Requirements/P-Specs Requirements must then be written into Performance Specifications (P-Specs)

Remain traceable:

DesignManufacturingOperationsNote: Picture of Acquisition timeline here11A well written P-Spec:Performance SpecificationArticulates user needs in terms that are:Output oriented OperationalFunctional Performance basedMeasurable

Functionally describes an acceptable product baseline.

Determines how to verify the product met that baseline.

12Performance Specification

In 1908, the United States Signal Corps drafted a general requirements document to identify the required specifications of the Wright Brothers heavier-than-air flying machine.

13The document included requirements such as:

be easily taken apart for transport in Army wagons and

be capable of being reassembled for operation in an hour,

carry 350 pounds for 125 miles, and

maintain 40 miles per hour in still air.

Performance Specification

This is an example of a set of performance specifications.14Performance Specification

The Wright Brothers won the contract, awarded about 2 months after the announcement, at a cost of $25,000.

15Quantitative (not Qualitative)

The mandrel shall be rugged.The mandrel shall have a hardness ofnot less than 60 and not more than 65 on the Rockwell C scale.(Measurable)(Subjective)Performance SpecificationSpecifications should be Quantitative not qualitative. Which one is which?16Human Systems IntegrationThe system shall be deployable by a two-man crew. No special tools or heavy lift devices shall be required.TemperatureThe system shall operate normally in temperatures from 0 to 40 degrees C (+32 to +104 degrees F). The system shall withstand storage and transportation in temperature extremes from 30 to +50 degrees C (-22 to 122 degrees F).Exposed EdgesThe equipment design shall have all exposed edges and corners rounded to prevent cuts or punctures.P-Spec ExamplesThree sections of a P-Spec.17ENVIRONMENTAL EXAMPLE:

Lithium batteries in the system must be UL certified and be able to pass NOSSA requirements for shipment via Navy Vessel with no additional testing.P-Spec Examples

Note add UL logo

The prospective vendors are free to meet the requirements in any way they can. They can offer any material conforming to the specification, as long as the products they offer meet the performance criteria established in the specification.18 Analysis Demonstration Examination Test Verification/ValidationThere are 4 ways to verify the specification has been met. Through Analysis, Demonstration, examination or testing. More than one type of verification may be used per specification.19Verification Ensures the system was built according to specification requirementsVerification/ Validation Note add picture of a chart

Verification by analysis uses established technical or mathematical models or simulations, algorithms, charts, graphs, circuit diagrams, other scientific principles and procedures, or reviews of applicable and adequate documentation to provide evidence that stated requirements were met.20Validation Ensures the system operates according to those specification requirementsVerification/ Validation Note add picture of a chart

Verification by analysis uses established technical or mathematical models or simulations, algorithms, charts, graphs, circuit diagrams, other scientific principles and procedures, or reviews of applicable and adequate documentation to provide evidence that stated requirements were met.21Analysis Technical models Charts Circuit diagrams DocumentationVerification/Validation Note add picture of a chart

Verification by analysis uses established technical or mathematical models or simulations, algorithms, charts, graphs, circuit diagrams, other scientific principles and procedures, or reviews of applicable and adequate documentation to provide evidence that stated requirements were met.22DemonstrationActual Operation of the Product

Verification/ValidationVerification by demonstration involves the actual operation of an item to provide evidence that the required functions were accomplished under specific scenarios. The items may be instrumented and performance monitored.23Simple physical manipulation Sight Touch Smell Hear Counting

Verification/ValidationExaminationVerification by examination is generally nondestructive and typically includes the use of sight, hearing, smell, and touch; simple physical manipulation; and mechanical and electrical gauging and measurement. 24TestActual testing of the system or product

Verified via the test reportVerification/ValidationTest is verification in which scientific principles and procedures are applied to determine the properties or functional capabilities of items.

Example: Shock and Vibration Testing25P-Spec. #1P-Spec. #2P-Spec. #3Verification #1Verification #2Verification #3Verification

For every specification there should be a corresponding means to verify the parameters in the specification have been met. Again, more than one type of verification can be used per specification.26First Article Test (FAT)System Integration Test (SIT)Performance Article Test (PAT)Joint Interoperability Test (JIT)Operational Testing (OT)Operational Environment Event (OEE)

Verification/Validation EventsWhen is the specification verified? Verification can be done at a number of events. FAT, SIT, PAT, or JIT. Verification can also be done at Operational Testing (OT).

27Human Systems IntegrationThe system shall be deployable by a two-man crew. No special tools or heavy lift devices shall be required.TemperatureThe system shall operate normally in temperatures from 0 to 40 degrees C (+32 to +104 degrees F). The system shall withstand storage and transportation in temperature extremes from 30 to +50 degrees C (-22 to 122 degrees F).Exposed EdgesThe equipment design shall have all exposed edges and corners rounded to prevent cuts or punctures.P-Spec ExamplesTemperatureThe system shall operate normally in temperatures from 0 to 40 degrees C (+32 to +104 degrees F). The system shall withstand storage and transportation in temperature extremes from 30 to +50 degrees C (-22 to 122 degrees F).TemperatureThe system shall operate normally in temperatures from 0 to 40 degrees C (+32 to +104 degrees F). The system shall withstand storage and transportation in temperature extremes from 30 to +50 degrees C (-22 to 122 degrees F).TemperatureThe system shall operate normally in temperatures from 0 to 40 degrees C (+32 to +104 degrees F). The system shall withstand storage and transportation in temperature extremes from 30 to +50 degrees C (-22 to 122 degrees F).Fix animation28A Analysis E Examination D Demonstration T Test FATSITPATJITDeploymentThe system shall be deployable by a two-man crew. No special tools or heavy lift devices shall be required.A/DTemperatureThe system shall operate normally in temperatures from 0 to 40 degrees C (+32 to +104 degrees F). A/TTemperatureThe system shall withstand storage in temperature extremes from 30 to +50 degrees C (-22 to 122 degrees F).A/TTemperatureThe system shall withstand transportation in temperature extremes from 30 to +50 degrees C (-22 to 122 degrees F).AExposed Edges

The equipment design shall have all exposed edges and corners rounded to prevent cuts or punctures.EVerification MatrixThe corresponding Verification. Notice the temperature requires 2 different verification statements. Also notice some of the specifications require 2 types of verification.29Conclusion

When system safety becomes a contract requirement and potential safety issues are tied to the performance of the system, many hazards can be eliminated or mitigated as contractors are considering solutions during the proposal stage.

The system safety professional will need to work closely with other systems engineers, test and evaluation personnel and acquisitions professionals to ensure an appropriate balance between safety, cost, schedule, and performance.

30Questions