Risk Acceptability (Tolerability) in System Safety: Concepts and
Methodology
Presented By:Elya B. Joffe
President, IEEE Product Safety Engineering Society
ARE YOU SAFE NOW???
Or…
� Could the ceiling fall?
� Could meteorite strike?
� Could fire start?
HOW SAFE ARE WE NOW?
Introduction: Why Risk Management?
Introduction: Why Risk Management?
Risk Management = Decision Making →→→→choosing an option that is perceived to have the
best benefit / cost ratio
No one takes a risk for the chance of lossOption A
Perceived Costs
Perceived Benefits
Introduction: Why Risk Management?
Option B
Perceived Costs
Perceived Benefits
The Ford Pinto Engineering Disaster
� Crash tests revealed defect ingas tank rear-end collisions over25mph resulted in rupture andexplosion
� The tank met legal standards,but Ford engineers knew itsdesign was flawed
� Cost-benefit analysis (CBA)estimation� Cost to pay for injuries:
� 180 Deaths, 180 Injured, 2100 Burned Cars = $49.5 million
� Cost to make safe cars:� $12.5 million cars x $11/car = $137 million
Cost
Safety
The Ford Pinto Engineering Disaster - Ford’s Dilemma
7
Outline� Introduction
� Buzzword Alert! Key Terms and Definitions
� Risk Assessment
� Risk Response Strategies
� Risk Acceptability
� Summary
Why do Men Die Younger ?
Do they Take Extra Risks?
Technology and Risks
� Technology has improved our level of well-being significantly
� But all technologies also have their potential downsides or risks…
� How should we decide about risky technologies?
� Or – What is an acceptable risk?
� There are a number of technical terms in this lecture
� Yes, you have to know them!� These terms have precise meaning, even though you will often see them MIS-used
� Since risk assessment is (or aims to be) a scientific activity we must agree on terminology
Buzzword Alert!Key Terms and Definitions
� Mishap (Accident)An unplanned event or series of events resulting indeath, injury, system damage, or loss of or damage toequipment or property
� Mishap LikelihoodLikelihood of mishap occurrence over a specifiedexposure interval. Probability is a component of riskand has no dimension but must be attached to aninterval of exposure (example: one operating year, amillion vehicle miles)
� Mishap Probability CategoryA categorization that provides arange of probabilities (or likelihoods) forthe occurrence of a mishap
Buzzword Alert!Key Terms and Definitions
� HarmPhysical injury or damage to the health of people, or damage to property or the environment
� HazardPotential source of harm, A condition prerequisite to a mishap
� Severity Measure of the possible consequences of a hazard; Severity is one component of risk
Buzzword Alert!Key Terms and Definitions
� Risk Combination of the probability of occurrence of harm and the severity of that harm
� Risk contains two elements:
� The likelihood of an event occurring
� The consequence and/or impact if it happens
� Residual Mishap Risk The mishap risk that remains after all approved control measures have been implemented and verified
� Safety Freedom from unacceptable risk
Buzzword Alert!Key Terms and Definitions
� Risk Acceptance:A decision to accept a risk. That level of residual risk that the managing authority is willing to assume on behalf of the agency, users and the public. Risk acceptance depends on Risk Criteria.
� Risk Criteria: Terms of reference by which the significance of risk is assessed
� Risk ControlProcess in which decisions are made andmeasures implemented by which risks are reduced to, or maintained within, specified levels
Buzzword Alert!Key Terms and Definitions
Introduction - What is “Risk”?
� Any uncertainty about the future� Technically can be both positive and
negative
� Safety questions focus only on negative outcomes
� Risk Assessment is an effective mean of identifying system or process safety risks� Characterizes hazards within risk areas and
critical technical processes
� Analyzes them for their potential mishap severity and probabilities of occurrence
� Prioritizes them for Risk Acceptance
� A scientific/mathematical discipline
� A substantive, changing and controversial field
� The most accepted tool is the Risk Assessment Matrix (RAM)� First used in 1662, when Blasé Pascal
created Pascal's concept of “Proportional Risk”
Risk Assessment
RelSafe Ltd
Quantitative Risk Assessment (QRA) Timeline
Blasé Pascal: Father of Risk Based Decision
� The frequency of the potential harm� The likelihood/probability that an
accident will occur due to the hazard
� The consequences of that loss� The most likely outcome/result of such
an accident
� The perception of the loss� The overall risk level of each hazard
� How seriously the stakeholders view the risk that might affect them
Risk AssessmentKey Concepts of Risk
� Risk is defined as a measure of frequency and severity of harm due to a hazard
� Safety is relative� It is a judgment of the acceptability of risk� An activity is considered safe if it’s risks are considered acceptable
� Risk by its nature can be considered a rare event
Risk AssessmentDefining and Calculating Risk
Risk Assessment Foundation
Historic experience
Analyticmethods
Knowledge & intuition
UnderstandingA Risk
UnderstandingA Risk
How likely is it to occur?How likely is it to occur?
What can happen?What can happen?
What are the impacts?What are
the impacts?
Risk AssessmentKey Concepts of Risk
Risk Assessment Risk Management
� What can happen?
� How likely is it to happen?
� What are the consequences if it happens?
� What can be done?
� What are the benefits, costs and risks of each option?
� What are the impactsof each option on future options?
Risk AssessmentAsk Six Questions
[ ],
r rP (P ( ( ,)) )|e o
o eR ee V o= ⋅ ⋅∑Risk
combination of…
probability of an event
probability of an outcome
given that event
the value of that event and outcome pair
For every event and outcome
Risk AssessmentThe Risk Equation
� The likely effect of a hazard may, for example, be rated:� (1) Major
� Death or major injury or illness causing long term disability
� (2) Serious� Injuries or illness causing short-term
disability
� (3) Slight� All other injuries or illnesses
Risk AssessmentAssessing The Risks Example
� The likelihood of harm may be rated� (1) High
� Where it is certain that harm will occur
� (2) Medium� Where harm will often occur
� (3) Low� Where harm will seldom occur
Risk AssessmentAssessing The Risks
Risk =
Severity of Harm x
Likelihood of occurrence
� Computation gives a risk value between 1 and 9 enabling a rough and ready comparison of risks
� The lower the number, the greater the risk
� Prioritizes hazards so that control action can be targeted at higher risks
Risk AssessmentAssessing The Risks
Risk AssessmentAssessing The Risks
� Incorrect: Confusing or combining Impact & Probability� “It is very unlikely, therefore the impact is
low”
� Correct: Keep Impact & Probability independent� “Probability is low, but if it happens, the
project will fail; therefore the impact is high”
Impact/Probability Matrix(or Risk Assessment Matrix (RAM))
Impact/Probability Matrix(or Risk Assessment Matrix (RAM))
� A common tool to conduct Risk Assessmentfor establishing system’s Risk Acceptance � Combines the two dimensions of a risk
(“probability-severitydoublet”):
� Probability of occurrence
� Its impact if it occurs
� Serves to: � Determine whether a risk is considered low, moderate, or
high
� Prioritizes hazards
� Determines either acceptability of the risk or appropriate management level to make the risk decision for tolerability
� The Severity and Probability dimensions of Risk define a Risk Plane
� The concept of iso-risk contour is useful to provide guides, convention and the Acceptance Limits for Risk Assessment
Impact/Probability Matrix(or Risk Assessment Matrix (RAM))
� In mathematical terms, therisk curve is thecomplementary cumulativedistribution function (CCDF),i.e., the frequency ofexceeding a given consequenceseverity
The Risk Plane
Logarithmicscales produces linear iso-risk
plots for R=S×P=constant
Risk is constantalong any
iso-risk contour
(S)
(P)
(R)
RAM – Iso-Risk Contour Uses
� Risk for a givenhazard can beassessed at anyseverity level
• Assess risk forthe worstcredible outcome
• An iso-riskcontour gives theprobability at alllesser severitylevels
Risk at A EqualsRisk at B
Further reductiondesirable
If risk for a givenhazard does notdisplay as an iso-risk contour, thenassess the severityand probability forthe Worst CredibleRisk
Risk Assessment Convention
Risk Assessment Guides
The Risk Plane Becomes A Matrix
Impact/Probability Matrix
The Risk PlaneSample Probability/Impact Matrix Example
Typical Impact/Probability Matrix(or Risk Assessment Matrix (RAM))
Frequent Individual: Occurs repeatedly in career
All: Continuous experienced> 1
Probable Individual: Occurs often in career
All: Occurs frequently1 ÷÷÷÷ 10-1
Occasional Individual: Occurs sometime in career
All: Occurs sporadically or several times10-1 ÷÷÷÷ 10-2
Remote Individual: Seldom chance of occurrence
All: Expected to occur sometime10-2 ÷÷÷÷ 10-4
Improbable Individual: Probably will not occur in career
All: Possible but not probable, rare10-4 ÷÷÷÷ 10-6
Incredible Individual: Occurs so implausibly as to elicit disbelief
All: Not plausible or believable< 10-6
based on IEC 60601-1-4
Risk Likelihood (Frequency Codes)
Risk AssessmentAssessing The Risks
Risk Severity(Severity of Consequence Codes)
Negligible First aid or minor supportive medical treatment, minor system impairment, minor property damage
MarginalMinor injury, lost workday accident, compensable injury or illness, minor system damage, minor property damage
CriticalPermanent partial disability, temporary total disability in excess of 3 months, major system damage, significant property damage
Catastrophic Death or permanent total disability, system loss, major property damage
Risk AssessmentAssessing The Risks
based on IEC 60601-1-4
� The process of developing options and actions to enhance opportunities and to reduce threats to the project objectives� Proactive, not reactive
� Appropriate to significance of risk
� Cost effective
� Timely
Risk Response Strategies
Risk Response StrategiesA-T-M
� Avoidance (A)Taking a conscious decision to apply specific, necessary measures to remove a potentialthreat by eliminating the cause of the risk
� Transference (T)The legal assignment of the negative impact of a threat, along with the ownership of the response, from one party to another (e.g., by insurance)
� Mitigation (M) / ReductionTaking actions to systematically reduce theexpected value/probability or impact of anadverse risk to an acceptable thresholdthrough control measures, according toa hierarchy of risk control
Risk Response StrategiesAcceptance
� AcceptanceRecognizing the existence of a specific risk and accepting the impact of the risk, should it occur
� Passive acceptance: no action , deal withthreats as they occur (workarounds)
� Active acceptance: establish a contingency reserve to handle risk
Impact
Prob
ability
Low
High
Low High
A – T - M
Pass
ive
Acc
ept
anc
e(w
orka
roun
d)
ActiveAcceptance(A – T – M)
Risk Response Strategies
�Avoidance (A)
�Transference (T)
�Mitigation (M) / Reduction
Tot
al o
rigi
nal ri
skMeasuresCategory
Policy, regulations,
land useplanning
RiskAvoidance
TechnicalMeasures,
Preparedness
RiskMitigation/Reduction
Living withRisk with or
Withoutcaution
Risk Acceptance
Residual risk
Acceptable risk level
Tolerable risk level
Risk Treatment
RiskTransference
Insurance
Cost Benefit Analysis (CBA)
� Simply put, CBA weighs costs against benefits to help determine the best course of action
Risk Acceptability/Tolerance
� Most Risks have associated benefits, however…
� Overall concept: Severe accidents are not acceptable – they should be avoided!!!
� As severe accidents never havezero probability, some form ofacceptance criteria is necessary
Even with properly identified hazards, someone may chose to operate outside design
limitations – a gamble at best!
Risk Acceptability/ToleranceThe “Challenger” Disaster
“The lowest temperature the system had previously experienced was 53ºF and both the primary and secondary component had failed to function as designed.“The predicted temperature for operation was approximately 26ºF. “
Morton-Thiokol VP of Engineering, STS-51L Accident Investigation, 1986
“…data below 53ºF was not available and [my] department could not prove it was unsafe to launch”
� O-ring Sealing problems� Engineers argued against
launch at low temperature� Management over-ruled the
engineers warnings� Shuttle exploded minutes into
the flight� 7 Lives lost � Setbacks to
the shuttleprogram
Risk Acceptability/ToleranceThe “Challenger” Disaster
Sr. VP to VP Engineering: “Take off your engineering hat and put on your management hat...”
� There is a difference between engineers and managers
� Engineers should adhere to their professional norms and hold safety paramount
Risk Acceptability/ToleranceThe “Challenger” Disaster
Murphy’s Law for Management
Risk Acceptability/ToleranceThe “Challenger” Disaster
Technology is dominated by those who manage what they don’t understand!
Ignoring risk doesn’t make the risk go
away!
Risk Acceptability/ToleranceGoverning Safety Using Quantitative Risk
Assessment
Risk Acceptability/Tolerance
Risk Acceptability/Tolerance
An acceptable risk is the risk associated with the best of available alternatives, not
with the best of alternatives which we would hope to have
available
� Factors that Determine Risk Acceptability� Personal
� Political / Social
� Economic
� Injustices� The process of determining the acceptability of
risk can be influenced by those with money and vested interests
� Setting a $ figure (in cost-benefit analyses) on a human life is considered by many to be unethical and unconscionable
� Remember the “Ford Pinto” Engineering Disaster?
Risk Acceptability/Tolerance
Factors Influencing Risk Acceptance
� “No Go” Alternative� Accept the Risk� Establish a “De Minimis Risk” Level
� Risks are so trivial that action to reduce risk generally would be unwarranted
� Establish a “De Manifestis Risk” Level� Risks are so high that they are manifestly
intolerable
� Perform a Cost-Benefit Analysis (CBA)� Perform Cost Effectiveness� Choose the Best Choice Among
Alternatives
Risk Acceptability/ToleranceRisk Acceptability Assessment Methods
� Risk levels, risk to individuals, societal risk, voluntary risk and involuntary risk, perception of risk
� Profound questions of Ethics and philosophy of life� What risk is acceptable?
� What is the value of life?
� Risk can be minimized at a cost even if can not be completely eliminated � How far should one go along this road?
Risk Acceptability/ToleranceAcceptability Criteria for Risk
� Define the alternatives � Specify the objectives and measures of effectiveness to indicate the degree to which they are achieved
� Identify the possible consequences of each alternative
� Quantify the values for the various consequences
� Analyze the alternatives toselect the best choice Source: Derby, Stephen L., Ralph L. Keeney. 1981. Risk Analysis: Understanding “How Safe Is Safe Enough?” Risk Analysis. V.1. No.3. Pp.217-224
Risk Acceptability/Tolerance5 Steps for Risk Acceptability Assessment
� IAW IEC Definition safety is the freedom from danger or risk of accidents� Absolute: Complete freedom from harm
� Basic: Freedom from unacceptable risk created by direct physical hazards when equipment is properly used under normal or reasonably foreseeable conditions
� It is impractical to expect equipment to be absolute safe(”free from risk”)� A compliant equipment shall ensure in
reality the freedom from unacceptablerisks
Risk AcceptanceRisk-Based Approach
“You must not say ‘never.’ That is a lazy slurring-over of the facts.
Actually, [risk analysis] predicts only probabilities.
A particular event may be infinitesimally probable, but the probability is always greater than
zero.”
“Second Foundation” (Isaac Asimov)
Because… →→→→
Risk Acceptability/Tolerance
� Everything That Can Go WrongWrongWrongWrong…� Sometimes Will Go WrongWrongWrongWrong !!!
Remember…Murphy was an OPTIMIST!!!OPTIMIST!!!OPTIMIST!!!OPTIMIST!!!
Risk Acceptability/Tolerance
… Murphy is the Patron Saint ofSafety Engineers...
Approaches to Risk-Based Governance
Benefits of Approach B:
• Reduces overall risk• Accepts greaternumber of desiredactions
• Defines area ofconcern for safetyprofessional to reducerisks
Like
lihoo
dConsequence
Approach A (1-step) Approach B (2-step)
Like
lihoo
d
Consequence
Not safe, can not proceed
Safe, proceed
Conduct analysis
Go No GoConductanalysis
GoNo Go
Go
Examine riskdrivers and
reduce
Analyze, decide
• Analyze, decide (if obvious)• Examine, reduce risk,
decide
Further review
Risk RegionRisk
Management Requirement
Broadly acceptable
None
Accept with mitigation
ALARP
Accept with national need
ALAP while meeting the operational requirement
“Broadly acceptable”
ALARPFre
quen
cyAs Low As Possible (ALAP)
• Acceptable• Significant risk management
Acceptable with
reasonable risk management
Higher
likelihoo
d
Consequence More severe
Greater undesirability of consequence
ALAP = As Low As PossibleALARP = As Low As Reasonably Possible
USA DoD Actual Concept
Mishap Probability Levels
Mishap Severity Categories
(1) Catastrophic
(2)
Critical
(3)
Marginal(4)
Negligible
(A) Frequent 1A 2A 3A 4A
(B) Probable 1B 2B 3B 4B
(C) Occasional 1C 2C 3C 4C
(D) Remote 1D 2D 3D 4D
(E) Improbable 1E 2E 3E 4E
Probability per System life
Multiplicative factor
> 10-1 No limit
10-2 ÷ 10-1 10
10-3 ÷ 10-2 10
10-6 ÷ 10-3 1000
< 10-6 No limit
Injury ≥≥≥≥ 1 deathSerious injury
Minor injury
Lost workday
Loss of Dollars > $1M$200k ÷
$1M$10k ÷$200k
$2k ÷$10k
Multiplicative factor
No limit 5 20 5
MIL-STD-882D RiskAcceptance Criteria (RAC) Matrix
MIL-STD-882 Risk Acceptance Criteria (RAC)
MIL-STD-882 Severity Categories Matrix
MIL-STD-882 Probability Levels Matrix
Risk Acceptance Matrix
The Risk Acceptance Matrix represents the tolerance level for acceptable and unacceptable risks
MIL-STD-882 Risk Acceptance Matrix and Authority
The Impact/Probability(Risk Assessment) Matrix Zones
� The Impact/Probability matrix zones indicate areas of:� “De minimis” risk acceptance
� Strict risk avoidance
� Both separated by an intermediate zone in which non-mandatory efforts should be devoted to further reducing risk according to “As Low as Reasonably Practicable” (ALARP)
NegligibleMarginalCriticalCatastrophic
Incredible
Improbable
Remote
Occasional
Probable
Frequent Intolerable (Strictly Unacceptable)
ALARPAs Low As Reasonably
Practicable
Broadly Acceptable
Based on ISO 14971, Fig E.1
Risk Regions
Severity
Likelihoo
d
The Impact/Probability(Risk Assessment) Matrix Zones
� ALARP (As Low As Reasonably Practicable)� The level of risk which can be further lowered
only by an increment in resource expenditure that cannot be justified by resulting decrement in risk
� Often identified or verified by formal or subjective application of cost-benefit analysis (CBA) or multi-attribute utility theory
� A region of risk between “strictlyacceptable” and “broadly unacceptable� Assumes we know where the acceptablelimit is
Risk Acceptability/TolerabilityThe ALARP Principle
� In the ALARP Range� Risk reduction is generally considered because
the risks are too high to be neglected, but –
� Risk reduction would be required only if feasible (e.g. cost effective) because the risks are not too high that they are manifestly intolerable
Risk Acceptability/TolerabilityThe ALARP Principle
Acceptable with endorsement of ProgramSafety Panel only if risk reduction isimpracticableControl measures must be introduced for riskreduction to drive residual risk towards thebroadly acceptable regionIf residual risk remains in this region, andsociety desires the benefit of this activity, theresidual risk is tolerable with endorsement ofthe Program Safety Panel if further riskreduction is impracticable or requires grosslydisproportionate action
The ALARPRegion
Increasing Individual Risk and Societal Concerns
Risks cannot be justifiedIntolerableRegion
NegligibleRegion
BroadlyTolerableRegion
Tolerable with endorsement of the normalproject reviews: Level of residual risk is notregarded significant and further effort andresources to reduce risk are likely to begrossly disproportionate to risk reductionachieved
Risk Level
$
Risk Acceptability/TolerabilityThe ALARP Tolerability Principle
Class A
Class B
Class C
Class D
Risk Score
Risk Descriptor
All categories of Risk except safety
or environment
Safety and Environmental Risks
12Extreme
Board Approval Required for Risk Intolerable
11
10
High
Senior Executive Approval Required
9 Level 2 Manager Approval Required
8Risk must be
managed in line with the ALARP Principle
Risk must be managed in line with the ALARP Principle
7
ModerateRisk must be managed in line with
the ALARP Principle6
5
4
LowNo approval required
but ongoing monitoring and management is required3
2
Risk Tolerability Criteria/Framework
Risk Tolerability Criteria/FrameworkHazard Reduction Order Of Preference
Risk Acceptance Criteria (RAC) Concept of de-Minimis Risk
� Risks judged to be too small to be of social concern, or to justify use of risk management resources for control� A de-minimis risk level of 10-6 (or “1 in a million”)
is frequently used by government agencies
� For increased risk of an adverse effect over a 70 year lifetime in a large population
� Many times below risks which people face daily
� A de-minimis risk level of 10-9 (or “1 in a billion”) per operating hour is often used forsystems where failure has a credibledirect potential for a catastrophicconsequence
“There is no point in getting into panic about the risks of life until you have compared the risks which worry you with
those that don’t, but perhaps should.”
(Lord Rothschild, “The Wall Street Journal”, 1979)
Risk Acceptance Criteria (RAC)Reality Check
Evaluation of Risk Acceptability
� Risk acceptability is controversial� Acceptability of risk depends on the nature of
the risk and on those who may bear it
� An acceptable level of risk always exists until someone tells you what it is
� There are levels of risk that people will accept, and other levels that they will not, risk acceptability carries all our social values
� Report on the inquiry of the Flixborough Accident(*) states:
� “… for what is or is not acceptable depends in the end upon current social tolerance and what is regarded as tolerable at one time may well be regarded as intolerable at another.”
� (*) An explosion at a chemical plant close to the village of Flixborough, England, on 1 June 1974. It killed 28 people and seriously injured 36.
� Society is much less willing to kill people in a single incident than from some factor over time, e.g., car crashes
� Voluntary risks are those we assume due to some perceived benefit, e.g., smoking, white water rafting.
� Involuntary risks are imposed on people by decisions made by others or by natural occurrence, e.g., second-hand smoke, and violent storms.
Society/Public Risk Acceptability
� The public takes other considerations into account in determining whether a risk is acceptable:� Fair distribution costs/benefits?� Risky activity freely chosen?� Available alternatives?
� Some risks can lead to enormous catastrophes, unacceptable, even if low probability
� Same concerns are shared by risk ethicists
Society/Public Risk Acceptability
1.to accept responsibility in making engineering decisions consistent with the safety, health and welfare of the public, and to disclose promptlyfactors that might endanger the public or the environment
Things to Remember…IEEE Code of Ethics
Summary� Risk is the potential harm that may arise
from some present process or future event
� A risk probability/impact assessment is used to analyze and prioritize the risks identified in the risk assessment.
� The risk acceptability/tolerance matrix represents your organization’s tolerance level for acceptable and unacceptablerisks
Things to Remember…
Murphy’s Law for Management
Technology is dominated by those who manage what they
don’t understand!
Thank You for your Thank You for your Thank You for your Thank You for your
AttentionAttentionAttentionAttention!!!!!!!!!!!!
Questions?
Comments?
Snide Remarks?