Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
W W W . C H I C A G O L A N D R I S K F O R U M . O R G W W W . C H I C A G O L A N D R I S K F O R U M . O R G
What’s New in Risk Assessment?
2 2
3 3
Risk Management Depends on Risk Assessment The simplest definition of Risk Management involves 3 steps:
Step 1
Step 2
Step 3
4 4
Risk Assessment Has Many Moving Parts!
6 6
Strategic Risk Assessment Question:
What is Important to Achieving Organizational Objectives and Not Under [complete] Control? Identifying threats and exposures without measurement generates lists of “Critical Risks” that frustrate management and fail to provide a blueprint for action. Strategic Risk Assessment starts with corporate objectives and considers assets at risk; identifies potential threats; and assesses the impact and the effectiveness of current controls to counter those threats.
– and points to new controls where objectives are threatened.
7 7
Strategic Risk Assessment Issues To be effective, Risk Assessment must be connected to business strategy. • Integrated in a way that provides timely and relevant risk
information to management.
• Owned by the business units and be embedded within the business cycle, starting with strategic planning.
• And: Risk assessment begins and ends with the organization’s specific objectives.
8
Strategic Risk Assessment
Qualita9ve Analysis Risk Register ◄
Risk Map ◄ Risk Categoriza9on ◄
Quan9ta9ve Analysis ► Decision Tree Analysis ► Scenario Analysis ► FMEA ► Simula9on & Modeling
9 9
Risk Assessment Basics • It is a matter of widespread understanding that risks should be
assessed in terms of the likelihood (probability) that an uncontrolled event will occur and the consequences (impact) on achieving one or more organizational objectives. – Applicable to both qualitative and quantitative methods of
assessment.
• Strategic Risk Assessment involves pursuing a systematic, logical process to identify the magnitude of hazards and exposures, assess threats, and implement controls to mitigate, eliminate or control high-risk conditions.
Qualitative Methods & Risk Maps Highlight Critical Threats
Risk Maps Are Primarily Qualitative Assessments
! Data gathering & representa9on
! Select appropriate techniques(s)
! Risk analysis & modeling ! Expert judgment
RISK
But Quantitative Methods are Often Required to Identify Corrective Actions
13 13
Risk Assessment Tools & Techniques Are Rapidly Evolving – RM’s Must Keep Up! Risk Assessment must move beyond Probability x Severity, risk registers and risk maps to quantitatively evaluate emerging issues, detectability, and critical threats to strategic objectives. • Over the past decade, developments in economic and financial
theory -- plus computing and “big data” advancements – provide new methods for quantitative risk assessment, as well as improvements to existing techniques.
• Risk Managers should understand available risk assessment techniques and adopt a set of tools they can apply to their organization's unique Risk Management requirements.
Several of the analytical methods discussed in this presentation are contained in the new Institutes designation program, Associate in Insurance Data Analysis (AIDA).
14 14
Three Basic Types of Quantitative Assessment Tools – In Order of Complexity
1. Comparative methods; 2. Temporal methods; and, 3. Functional methods.
15 15
Comparative Assessment Methods A Comparative Analysis takes an explicit standard – eg., “Best Practices” – and compares a system, process and/or set of procedures to that standard, producing a “Gap Analysis” as a result. A “good standard” is prepared and maintained as “the distillation of continually developing expert opinion and experience in the face of a continually changing environment”. • One of the strengths of Comparative Analysis is its simplicity.
Comparative methods can be ideal for organizations just beginning to focus attention on specific systems, processes or threats.
• A key weakness is the lack of an explicit list of threats as there is in other approaches.
16 16
Sample “Best Practices” Matrix – Claims Handling
O Managerial OversightP PrimaryS SecondaryC Consultative InputD Data Resource
Directo
r of Insurance
Directo
r of Legal Supp
ort &
Cla
ims
Executive
Vice
Preside
nt, A
on
Senio
r Vice
Preside
nt, Claims
Vice Pr
eside
nt, Claims
Assistant VP, Claim
s
Senio
r Con
sultant, Claims
Senio
r Client Sp
ecialist,
Claims (Megan)
Senio
r Client Sp
ecialist,
Claims (Martha)
Claim Assistant
CLAIM MANAGEMENT PROCEDURES
1)Establish formal claims service standards for TPA's, carriers and other vendors C P C P C C
2)Develop annual written service plan for TPA's and other vendors and monitor performance C P O P C C
3) Develop written Claims Procedures or Manual C O C P C C
4)Establish internal claims reporting and management procedures and monitor compliance C O C O P S
5)Develop claim reports, distribute and review with business units as necessary C O C O C P S S S D
6) Maintain listing of all insured claims O O O C P S S S D7) Maintain listing of all self-‐insured claims O O O C P S S S D8) Establish and monitor WC post-‐injury management program C O O O D P S9) Manage claims litigation process C O O P D S D10) Administer OCIP claims C O O O P S11) Administer non-‐litigated GL claims O O O P S S D D12) Administer auto claims O O O P S S D D13) Administer D&O, fidelity, fiduciary, EPL C P C14) Administer Litigated GL claims O O P D S D15) Administer Property claims O O O P D S D16) Pursue subrogation activities O O O P S S S D17) Review losses and identify trends C C C O C P S S S D18) Conduct/coordinate periodic claims audits D D19) Monitor large loss activity C O O P D D
20)
Review and adjust safety/loss control initiatives as needed to proactively treat risk and address trends observed in claims management activities
O C C C D C D
Management assures activ ity is addressed
Principally responsible for driv ing the activ ity
Responsible to perform or drive certain aspects of the activ ity , but is not the leader
Can prov ide guidance or feedback at a high level for activ ity
Prov ides data or information that is used in the activ ity
17 17
Sample “Gap Analysis” – RM Strategy
18 18
Temporal Analysis Methods A Temporal Assessment applies quantitative tests to a system, process or set of procedures. These “tests” involve analyzing the results of specific threats or attacks against actual protections and controls, subject to some constraints. • Since it is often impractical to test a system directly, a model of the
system is generally used instead. – However, a model introduces the question of fidelity: an
inaccurate model may not only confuse matters -- it may provide a false sense of security that is even worse than confusion.
• A key weakness of a temporal method is that it is not possible to model – or even list -- all possible threats.
19 19
One Temporal Method: Scenario Analysis
Scenario analysis considers the questions what might happen? and what should/would we do if it does?’ It not only highlights risks and opportunities in the short and long term; but also tests the effectiveness and efficiency of specific controls and plans. • The central idea is to consider a variety of possible futures that
include many of the important uncertainties in the system, rather than to focus on the accurate prediction of any particular outcome.
• A strength of scenario analysis is that it can consider “existential threats” that involve large swaths of the organization.
20 20
Four Critical Components of Scenario Analysis 1. Determine which factors the scenarios will be built around. In general,
analysts should focus on the two or three most critical factors. 2. Determine the number of scenarios to analyze for each factor. That
depends upon how different the scenarios are, and how well the results of each scenario can be forecast.
3. Estimate results – e.g., asset cash flows, control failures, unexpected breakdowns, etc. -- under each scenario.
4. Assign probabilities to each scenario. Note: this may not make sense if the scenarios do not cover the full spectrum of possibilities; in that case, the probabilities will not add up to 100%
21 21
Sample Scenario A Scenario Analysis to ensure effective and reliable insurance coverage. • Typically involves meeting with brokers, underwriters, lawyers,
adjusters and managers to analyze and discuss how each insurance policy is expected to respond to different circumstances.
• The results are compiled into systematic tables and charts that point out problem areas and suggest solutions.
One of the strengths of Scenario Analysis is that it tests the system itself (or a model), clearing away misconceptions and uncovering specific elements or issues needing attention.
22 22
Other Temporal Analysis Methods
The most important Temporal Assessment techniques use Predictive Analytics to not only determine what might happen, but how much it could impact objectives. • Two useful tools are:
– Decision Tree Analysis; and, – Simulation modeling.
23 23
Decision Tree Analysis A Decision Tree is a logical structure in which each internal node represents a "test" on an attribute; each “branch” represents the outcome of the test; and each “leaf” represents a decision taken after computing all previous attributes. The paths from root to leaf represent classification rules:
– A Root node represents the start of the decision tree, where a decision maker is faced with an uncertain outcome. The objective is to evaluate the overall net positive or negative outcomes at this node.
– Event nodes represent outcomes based upon the probable occurrence of various events.
– Decision branches represent choices that are made by the decision maker.
– End nodes represent final outcomes where a payoff value is identified.
24 24
Sample Decision Tree: Jenny Lind • Jenny Lind is a writer of romance novels. A movie company
and a TV network have both offered to buy exclusive rights to one of her more popular works.
• If she signs with the TV network, she will receive a definite lump sum; but if she signs with the movie company, the amount she will receive depends on the market response to her movie.
• What should she do?
Jenny Lind Decision Tree
Small Box Office
Medium Box Office
Large Box Office
Small Box Office
Medium Box Office
Large Box Office
Sign with Movie Co.
Sign with TV Network
$200,000
$1,000,000
$3,000,000
$900,000
$900,000
$900,000
.3
.6
.1
.3
.6
.1
Es+mated Outcomes Es+mated
Likelihood
Root Node
Event Nodes
Jenny Lind Decision Tree - Solved Small Box Office
Medium Box Office
Large Box Office
Small Box Office
Medium Box Office
Large Box Office
Sign with Movie Co.
Sign with TV Network
$200,000
$1,000,000
$3,000,000
$900,000
$900,000
$900,000
.3
.6
.1
.3
.6
.1
Expected $900,000
Expected $960,000
Best Result $960,000
27 27
Simulation Modeling Where Scenario Analysis and Decision Tree Analysis are techniques to assess discrete risk events, Simulation methods measure continuous risk exposures and outcomes. • Simulations yield a distribution of outcomes rather than a single point
estimate. • An important simulation tool is an “Exceedance Probability Curve”
that measures whether an outcome will exceed a specific estimate, based upon predetermined probabilities.
• Simulation has few limitations in terms of events, probabilities and outcomes – very robust models may be constructed, evaluated and displayed graphically.
28 28
Simulation Model Example: Quantifying the Risk of Natural Catastrophes How do companies prepare for the financial impact of natural catastrophes? How can they possibly determine the potential cost of events that haven't happened? Catastrophe Modeling can provide answers. A catastrophe model is generally composed of three parts: • The Hazard Module looks at the physical characteristics of potential
disasters and their frequency. • The Vulnerability Module assesses the vulnerability (“damageability”)
of buildings and their contents. • The Damage Module determines the overall loss distribution for a
specific event by multiplying building values by potential damage.
Catastrophe Modeling Functions
Expected Frequency of Major Storms
0 2 4 6 8 10 12 14 16 18 20
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
55%
60%
65%
70%
75%
80%
85%
90%
95%
100%
Probability of # Annual Storms
0 20 40 60 80 100 120 140 160 180
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Expected # of Annual Storms
At least 1 major storm is expected in 2 years out of 3
There is a 5% chance of 7 or more major
storms in 1 year
# of Annual Storms
# of
Ann
ual S
torm
s
# of
Yea
rs
Expected Business Interruption Losses Summary of Simulation Model Results
LIKELIHOOD EXPECTED LOSS ($MM) 1 in 10,000 $412.0
1 in 1,000 $103.2
1 in 100 $23.9
1 in 10 $3.5
1 in 3 No storm
Average Annual Loss $1.2
32 32
Functional Assessment Methods A Functional Analysis focuses on specific threats and protections. • A threat model -- a list of system vulnerabilities, and the likelihood of
successful threats against those vulnerabilities -- is weighed against organizational objectives, assets, and protections -- and the likelihood that available protections will successfully defend those assets against specified threats.
• Temporal Assessment methods, such as statistical modeling; and Comparative Assessment techniques, such as expert systems, are often employed jointly.
• The key strength of a Functional Assessment is its ability to specifically consider a wide range of threats, vulnerabilities, assets and countermeasures.
33 33
Failure Mode & Effects Analysis (FMEA) FMEA identifies where & how failures can occur within processes and measures the impact of those failures. • The FMEA Process has 4 basic steps:
1. Determine the failure modes of specific process elements – that is, what (specifically) could go wrong;
2. Analyze the likely effects of those failures on other elements and the overall system;
3. Rank criticality; and, 4. Identify existing and potential controls and their probable impact.
• FMEA is particularly useful for evaluating critical risks in very complex systems.
FMEA Thought Process
FMEA Path Model Example
36 36
Sample FMEA Template A c tion R es ults
Item / F unc tionP o tential F ailure
Mode(s )
P o tential E ffec t(s ) o f F ailure
Sev
P o tential C aus e(s )/
Mechanism(s ) o f F ailure
Prob
C urrent D es ign C ontro ls
Det
RPN
R ecommended A c tion(s )
R es pons ibility & T arget
C ompletion D ateA c tions T aken
New
Sev
New
Occ
New
Det
New
RPN
C oo lant containment. H o s e connec tion. C oo lant fill. M
C rack/break. B urs t. S ide wall flex. B ad s eal. P oo r ho s e rete
Leak 8 Over pres s ure 8 B urs t, validation pres s ure c yc le.
1 64 T es t inc luded in pro to type and produc tion validation tes ting.
J .P . A guire 11/1/95 E . E glin 8/1/96
Response Plans and Tracking
Risk Priority Number - The combined weighting of Severity, Likelihood, and Detectability.RPN = Sev X Occ X Det
Likelihood - Write down the potential cause(s), and on a scale of 1-10, rate the Likelihood of each failure (10= most likely). See
Severity - On a scale of 1-10, rate the Severity of each failure (10= most severe). See Severity
Detectability - Examine the current design, then, on a scale of 1-10, rate the Detectability of each failure(10 = least detectable). See Detectability sheet.
Write down each failure mode and potential consequence(s) of that
37 37
FMEA Technique: Fault Tree Analysis • A Fault Tree is a logical diagram that starts with an actual or
predicted failure and works backward to identify all of the possible causes or origins of that failure.
• Made up of branches connected by AND nodes and OR nodes; each node is assigned a pre-determined probability. – ALL of the branches below an AND node must occur for the
event above the node to occur. – Only ONE of the branches below an OR node needs to occur for
the event above the node to occur
38 38
Fault Tree Example
Identified “Fault”
Both Required
Any of These
Required
39 39
FMEA Technique: Event Tree Analysis • An Event Tree is a logical diagram that starts with an actual or
potential event and works forward to identify all of the failures -- and possible corrective actions -- that could result.
• Essentially the reverse of a Fault Tree; in an analysis, one Event Tree may lead to multiple Fault Trees and vice-versa.
• Originally developed by engineers to find and fix vulnerabilities in nuclear power plants; it is applicable, and has been applied, to assess many complex processes.
.302
.034
.084
.180
.400
41 41
Summary – Strategic Risk Assessment Various quantitative risk assessment methods view the landscape from different heights, so to speak -- altitude is a tradeoff between scope and detail. • The more abstract the method, the greater the scope but the
coarser the detail; the more concrete the method, the smaller the scope and the finer the detail.
• Different objectives, systems, threats, perils, hazards, controls, etc. dictate the use of different assessment tools and methods.
• Identifying the appropriate technique should be the first – and most important – step in risk assessment.
42 42
And, Don’t Forget – the Real Objective is to Manage Risk • Don’t waste time with complex analysis when you
KNOW what to DO – and everyone AGREES. • The methods examined in this discussion, and many
other related techniques, are best used when there is considerable uncertainty about exposures, risks, perils, outcomes, hazards, etc. – and about the best techniques to eliminate, mitigate,
control, or manage critical risks.
QUESTIONS?
Thank you very much for listening!
Backup
45 45
Categorizing Risk Assessment Techniques
Three basic categories of assessment tools are: 1. Comparative methods; 2. Temporal methods; and, 3. Functional methods.
Assessment techniques and tools can be classified on three axes: 1. by their level of formality on a continuum from abstract to
concrete; 2. the type of analysis performed; and 3. the threats they are attempting to find and address.
46 46
Types of Comparative Assessment Methods
• A Principles Method type, like all of the Comparative types, is a list. This type asks the user to apply agreed-upon principles to their system.
• A Best Practices list consists of directives: Do this, Don’t do that. This method asks the user to compare what they do—their current practice—with the best practice list: the list of differences represents the “Gaps” between actual practices and ideal.
• An Audit is based on an explicit standard, such as a Best Practice list, a requirements list or a Principles list. This type asks the user to evaluate the effectiveness of the controls in place in fulfilling each item in the standard.
47 47
Types of Temporal Assessment Methods
• An Engagement consists of experts looking for any way, within given bounds, to compromise assets.
• An Exercise links experts and owners together in order to test the protection of assets particular to a specific system.
• Compliance Testing includes methods that the owner can execute him/herself without the aid of an expert.
48 48
Types of Functional Assessment Methods • Sequence Methods are the epitome of abstract methods. A simple
sequence method asks the questions: 1. What can happen? (i.e., What can go wrong?) 2. How likely is [it] that that will happen?
3. If it does happen, what are the consequences?
• An Assistant Method type keeps track of details. Best illustrations of this type “walk” the user through a process, prompting for inputs needed to populate and rank lists of threats, vulnerabilities and remedial actions.
• A Matrix Method asks the user to select ranges for n dimensions – assets, threats, vulnerabilities and protections. The information in the cells of the corresponding n-dimensional subspace is the result of analysis.
– An Expert System is one implementation that is representative of the functional approach.