Applying CMMI toApplying CMMI toSystem SafetySystem Safety

Tom Pfitzer

A-P-T Research, Inc.

M-05-01100-2

Good System Safety ProgramsGood System Safety ProgramsGood System Safety Programs

People

Practices Tools

A combination of factors related to people,practices and tools result in the goodness

of a system safety program

Each of the main factors can be evaluatedto predict the adequacy of the resulting

safety program

A-P-T Research, Inc.

M-05-01100-3

The CMM ConceptThe CMM ConceptThe CMM Concept

Maturity is measured by

Achievement Levels:0 – Incomplete/Entry-level or repeated

fledgling level analyses, casuallyperformed

1 – Pro forma/Perfunctorily2 – Managed (work guided and overseen

by trained Supv.)3 – Defined4 – Quantified (Metrics applied to various

determinants/discriminants)5 – Optimized (Superior)

People

Practices Tools

The maturity of an organization’s capabilitydepend upon 3 interrelated elements

A-P-T Research, Inc.

M-05-01100-4

Why CMM?Why CMM?Why CMM?

The Use of the CMMI approach could provide:

A. Government organizations a means tospecify or evaluate industry safetyprograms

B. Mature industry and governmentprograms a means to “certify” existingmaturity

C. Immature industry or Governmentprograms a way ahead toward morematurity

Capability MaturityModel Integration

“…the quality of a system orproduct is highly influenced bythe quality of the process usedto develop and maintain it.”

Mary Beth Chrissis, et al

“You take you car into a lousyshop, you’re gonna get a lousyjob!”

Tom & Ray Magliazi

A-P-T Research, Inc.

M-05-01100-5

The CMMI Approach to any discipline such as System SafetyThe CMMI Approach to any discipline such as System SafetyThe CMMI Approach to any discipline such as System Safety

sssssrrrrrqqqqqcccccbbbbbaaaaazzzzzyyyyyxxxxx5 - Optimized

ssssrrrrqqqqccccbbbbaaaazzzzyyyyxxxx4 – QuantitativelyManaged

sssrrrqqqcccbbbaaazzzyyyxxx3 – Defined

ssrrqqccbbaazzyyxx2 – Managed

srqcbazyx1 – Performed

NoneNoneNoneNoneNoneNoneNoneNoneNone0 - Incomplete

T3…T2T1M3…M2M1P3…P2P1

ToolsMethodsPersonnel

Notional

MeasurementCategories

MeasurementIndices

Levels of Maturity

A-P-T Research, Inc.

M-05-01100-6

PersonnelPersonnelPersonnel

Advanced Degree25 + YearsAdvanced Degreein System Safety5

15 – 25 Years4

CSP7 – 15 Years3

SSS Member3 – 7 Years3 – 5 ShortCourses2

1 – 3 Years1 Week Training1

0 - 1 FulltimeNoneNone0

P5 …P4 - Depth of StaffP3 - CredentialsP2 - ExperienceP1 - Training

Notional

A-P-T Research, Inc.

M-05-01100-7

MethodsMethodsMethods

4, + auditableevidence ofcloseout

4, + designchange usegenerouslyevident

3 & 4, +maintenance/calibration, etc.

4, +maintenance/calibration, etc.

Full Matrix(indicates/spans/Resolution)

3rd Party(>5 long-termsample)

5

Coupledw/Config.Mgmnt. orQuality Prgm

Use enforced3, + severitylevels tailored tocase

All significanttransients

Quantitativematrix scaling

Mgmnt(2nd level)4

Procedure-driven,documented

Used andMonitored

Two or more,case selected

TBDSubjectivematrix tailoring

Peer/Mgmnt(>1 or 1st levelmgmnt)

3

InformalUsed but notmonitored

Two, rote-selected

Modest, pro-forma (eg.,startup/run/stop)

Disciplinedmatrix selection

Peer (1)

2

NoneNot evidentPro-forma(ad-hoc)

NoneNone performedNone performed(solo Analysis)1

0

M7 – HazardTracking

M6 – Use ofRisk Tolerant

Limits

M5 – UseEffectiveness

Hierarchy

M4 – AssetSelection

M3 – MissionPhasing

M2 – MatrixTailoring

M1 – Reviewof Analysis

Notional

A-P-T Research, Inc.

M-05-01100-8

Methods (cont.)Methods (cont.)Methods (cont.)

3 &4, + FMEA orHAZOP, or FHA

RigorousTailored toprogram/systemneeds

Full-bore, readilyauditablew/Reliability,Availability

Designerstrained/intermediateapplication5

Operationalwalkthroughs

3, + Numericallydone

TBDTBDConcurrentengineering4

2, + Energy sourceinventory

Procedurallydocumented

TBDFormal, mandatorycross-feedw/Reliability

Frequent designreviews (e.g., ≈15%intervals)

3

1, + ChecklistSubjective, looselydisciplined

TBDModest, informalcross-feedw/Reliability

Infrequent designreviews (e.g.,30/60/90%)

2

“What-if”NonePro-formaNoneNone1

0

M12 – HazardIdentification

M11 – RiskSummation

M10 – Selection ofRisk Tolerant

Limits

M9 – CrossCoupled “illities”

M8 – Influence ofDesign

A-P-T Research, Inc.

M-05-01100-9

ToolsToolsTools

TBDCCA + (FTA or ETA)Top-Down + Bottom-Up5

TBDCCA (quantified)FMEA or FHA4

TBDFTA a/o ETA (quantified)PHA or HAZOP (w/matrix)3

TBDETA (unquantified)PHA (w/o matrix use)2

TBDFTA (unquantified)PHL1

0

T3 – Probalistic RiskAssessmentT2 – Logic Tree ToolsT1 – Hazard Inventory Tools

A-P-T Research, Inc.

M-05-01100-10

ConclusionConclusionConclusion

• If interest exists, G-48 could develop recommended standards tomeasure/evaluate System Safety program maturity.– APT will host a collegial workshop to define a strawman set of

measurement categories and indices for each.– Produce a report with recommended categories and indices.

A-P-T Research, Inc.

M-05-01100-11

Contact Information:

Tom Pfitzer

256.327.3388

A-P-T Research, Inc.

pfitzer@apt-research.com