Qualitative Risk Assessment
Risk Analysis for Water Resources Planning and Management
Institute for Water Resources
2008
Risk AssessmentWhat can go wrong?How can it happen?How likely is it?What are the consequences?
Want to Improve Your Risk Analysis?
Use simple narratives that answer these questions honestly
Tell story of existing riskTell story of residual or transformed risk
The NeedManage risk intentionallyDo better than has been doneQuantitative risk assessment not always possible or necessaryQualitative risk assessment can be a viable option
Qualitative Risk AssessmentIs formal, organized, reproducible method based on science and sound evidenceFlexibleEasy to explain to othersSupports risk management decision making
Three Sample MethodsEnhanced Criteria RankingOperational Risk Management (Risk Matrix)CARVER + Shock
Enhanced Criteria Based Ranking
CriteriaRatingsAll Possible Combinations of RatingsRankingEvaluate Reasonableness of RankingAdd CriteriaNew Combinations of RatingsNew Ranking
Question?Which lock gates in division present the greatest potential risk to health and safety and therefore should be repaired first?
Step One: CriteriaAssume criteria equally important (or not).Reflect most important aspects of evaluating risk.Define H, M, L scenarios for each criterion. Use three or four evidence-based criteria.
H = Twenty and above Years of Age.M = Ten to Twenty Years of Age.L = Zero to Ten Years of Age.
H = Daily Use-approximately 365 times a year.M = Great than one and less than 365 times a year.L = Annual use-Once a Year.
H = Loss of Life and/or Property.M = Structure Damage.L = Minimal Loss of Property and/or Damage.
GATESCriteria #1: Age
Criteria #2: Frequency of Use.
Criteria #3: Consequence of Failure.
Step Two: RatingUse expert judgment to critically evaluate the available informationDevelop estimates for each “hazard” against criteriaUse letters or numbers but numbers do not represent an absolute measurement of risk only a relative means for comparison
Gate Criteria 1 Criteria 2 Criteria 3
Knightsbridge H L M
Steadly H M M
Redwood M H L
Jackflash M H L
Cantget L L L
Roughjustice H M L
IORR L M H
19 L H L
Step Three: All Possible Combinations
Greatest Risk HHH
HHM, HMH, MHHHHL, HLH, LHH, HMM, MMH, MHMHLM, MHL, HML, LMH, MLH, MMM,
LHMHLL, LHL, LLH, MML, LMM, MLMMLL, LML, LLM
Least Risk LLLThis is for equally weighted criteria. Unequal weightsyield different listings.
Step Four: Rank SubjectivelyEstablish rank according to descending relative riskIdentify subjective clusters.
Gate Rating Ranking
Steadly HMM Greatest Risk
Roughjustice HML
Jackflash MHL
Knightsbridge HLM Moderate
Risk
Redwood MHL
IORR LMH
19 LHL
Cantget LLL Least Risk
Step Five: Add Criteria?Look at rankings, do they make sense?Have you thought properly about this issue?If they do not, perhaps you did not consider all the most relevant criteriaA new criteria may be added to more accurately reflect the assessors rationale for ranking
Step Five: Add Criteria? (cont)Suppose the following was added to our exampleCriterion 4: Cost of emergency repair
H = Major disruptions to navigation or power, much higher costs to repairM = Much higher costs to repairL = Same as scheduled repair
Step Six: New RatingsGates Criteria #4 Rating
New Combined Ranking
Steadly H HMMH
Jackflash H MHLH
Knightsbridge H HLMH
Redwood M MHLM
IORR M LMHM
19 H LHLH
Roughjustice L HMLL
Cantget H LLLH
Step Seven: New RankingGates
New Combined Ranking
Criteria #4 Rating
Steadly HMMH Greatest Risk
Jackflash MHLH Greatest Risk
Knightsbridge HLMH Greatest Risk
Redwood MHLM Moderate Risk
IORR LMHM Moderate Risk
19 LHLH Moderate Risk
Roughjustice HMLL Moderate Risk
Cantget LLLH Least Risk
Operational Risk Management (ORM)
ORMAKA the risk matrixRisk ranking tool
Uses ranges of consequence and likelihoodCombinations created enable assessors to qualitatively estimate a risk
StepsDetermine purpose and use of matrix
Identify the question to be answered
Define consequences of interestIdentify consequence ranges and definitions Identify likelihood ranges and definitions Identify levels of risk in the cells of the matrix
Your DE Has Seen This“Mishap Risk”DOD "Standard Practice For System Safety”MIL-STD-882D 10 February 2000
Consequence Severities
Probability Levels
Risk Assessment Values
Each risk you assess is placed in a cell and managed accordingly
Risk Levels
Another Example
Source: Assessing Environmental Risk, A Lecture to the Irish Environmental Law Association By: L. M. Ó Cléirigh 29June 2004
Risk Matrix
Three AxiomsWeak consistencyBetweennessConsistent coloring3x3 and 4x4 should look like this to minimize problems
Low HighHigh
LowLow High
High
Low
Source: What’s wrong with risk Matrices? By Louis Anthony Cox, Risk Analysis Vol. 28 No.2, 2008
The Risk Management Point ofMatrix
CARVER + ShockVulnerability assessment method developed for Department of Defense CARVER is an acronym Criticality - measure of public health and economic impacts of an attackAccessibility – ability to physically access and egress from target
Recuperability – ability of system to recover from an attackVulnerability – ease of accomplishing attackEffect – amount of direct loss from an attack as measured by loss in productionRecognizability – ease of identifying target
SHOCKTechnique modified to include seventh attribute that combines health, economic, and psychological impacts of an attack
SHOCK attributes of target
Select a ProcessIdentify a critical process or infrastructure and assess vulnerability across nation, orAssess vulnerability of components of a single process
Critical
A target is critical when “loss” would have significant life, health or economic impacts
AccessibilityA target is accessible when an “attacker” can reach it to conduct the attack and then escape the target undetected
RecuperabilityThe time it will take for the specific facility to recover productivity is the target’s recuperability
VulnerabilityVulnerability measures the ease with which sufficient quantities of threat agents can be introduced to achieve the attacker’s purpose once the target has been accessed
EffectEffect is the percentage of system productivity damaged by an attack at a single facility
RecognizabilityRecognizability is the extent to which the target can be identified by an attacker without confusing it with other targets or components
ShockShock combines the measure of the health, psychological, and collateral national economic impacts of a successful attack on the target system
Sandia LabsUser friendly software has been developed for food defense by FDA and Sandia
http://www.cfsan.fda.gov/~dms/carver.html Process diagrams Interviews Results
Take Away PointsNot all risk assessment needs to be quantitativeDevelop a few consistent and well developed techniques for your usage