Module 1

KSC_CRM_SHORT_MODULE_01THRU10 REV BASIC, 1/021

Module 1

Welcome


Overview

•Introductions

•Administrative

•Course Objectives

•Course Schedule

•Style of Course

•Course Materials


Administrative

•Breaks

•Sign-In Sheet


Targeted Audience

•Mix of project personnel with variable levels of experience in KSC development projects

•Prerequisites:

• Project management or systems engineering experience (at least one year)

•Assumptions:

• Prior knowledge of risk or risk management is not required


Course Objectives

•Understand the concepts and principles of Continuous Risk Management and how to apply them

•Develop basic risk management skills for each component of Continuous Risk Management

•Be aware of key methods and tools

•Understand how CRM could be tailored to a project

•Be able to differentiate between Risks and Problems


Module 2

Introduction


Overview

•Agency Risk Management (RM) Requirements

•Risk Definitions

•RM/Project Management Relationship

•Risk Management Benefits

•Continuous Risk Management (CRM) Process

•CRM Process Application

•Risk Management Planning/Documentation

•Who Performs Continuous Risk Management?


Agency Risk Management Requirements

•Risk Management Documentation

• NPD 7120.4, Program/Project Management, describes the management systems for program/project formulation, implementation, and evaluation

• NPG 7120.5, NASA Program and Project Management Processes and Requirements, dictates basic risk management requirements

• NPG 8000.4, Risk Management Procedures and Guidelines, provides additional information for applying risk management to programs and projects as required by NPG 7120.5

• Procurement Notice (PN) 97-58, Risk Management



•Fundamental Risk Management Requirements

• Program and project decisions shall be made on the basis of an orderly risk management effort

• Risk management includes identification, assessment, mitigation, and disposition of risk throughout the project formulation, approval, implementation, and disposal phases

• Project/Program Manager (PM) has the overall responsibility for the implementation of risk management, ensuring an integrated, coherent risk management approach throughout the project




• Risk management planning will be developed during the project/program formulation phase, included in project/program plans, and executed during the implementation phase

• Programs and projects will develop and maintain prioritized risk lists

• Programs and projects must develop and clearly communicate “success criteria” to all levels of the program and project to define the scope of the effort and to guide risk decisions




• Programs and projects must define, within the constraints imposed upon them (e.g., budget, schedule), what level of risk (i.e., “acceptable risk”) is reasonable for the program/ project to accept while still achieving mission success

• Primary risks (i.e., risks having high probability and high impact/severity) must be classified, with acceptance rationale documented and concurred with by the Governing Program Management Council (GPMC)


Risk Definitions

•Risk is the combination of the probability (qualitative or quantitative) that a program or project will experience an undesired event (cost overrun, schedule slip, safety mishap, security compromise) and the consequences (impact) of the undesired event, were it to occur.

• NPG 8000.4


Risk Definitions

Risk involves the impact of the event should it occur.

Risk involves the probability that an undesired event will occur.

Qualitative orQuantitative


Risk Exposure = Probability x Impact


Some Perspectives on Risk

•Risk will always be present in programs and projects

•Not all risk can be avoided

•Management and stakeholders must be active participants in the mission risk acceptance process

•Risks are different from problems


Goal of Risk Management

•Achieving Mission Success

•Provide program/project managers principles and practices for decision making• Search out, identify, and manage risk

• Keep safety paramount

• Deliver a quality product on time and within cost


•Program/project teams must develop clear “Success Criteria” during the formulation phase

•Success criteria must be clearly communicated to all levels of the program and project to define scope of the effort and to guide risk decisions

•Success criteria need to be developed at system, subsystem, and component level to define trade space and support risk decisions

•Success criteria will continue to evolve throughout the life cycle of the project

Success Criteria Emphasis


Risk Management/Project Management Relationship

Project Management

RiskManagement

Budget

Schedule Performance

People

Quality

ConfigurationManagement

Safety


Risk Management Benefits

•Early identification of potential risks

•Facilitates setting priorities

•Increases chance of project success

•Enables more efficient use of resources

•Promote teamwork by involving personnel in all levels of the project

•Information for tradeoffs is based on priorities and quantified assessments

•Identify hidden risks


Everybody Wants to Understand Risk

Dilbert Scott Adams


Continuous Risk Management Process

•Continuous Risk Management (CRM) is a structured, formalized management practice with processes, methods, and tools for managing risks in a project

•It provides a disciplined environment for proactive decision making to:• Assessment (continual) of what could go wrong (risks)• Determination of which risks are most important to deal with• Implementation of mitigation strategies to deal with those risks• Assured, measured effectiveness of the implemented mitigation

strategies




CRM Process Components

•Identify• Search for and locate risks before they become problems

•Analyze• Convert risk data into useable information for determining

priorities and making decisions

•Plan• Translate risk information into planning decisions and

mitigating actions (both present and future), and implement those actions


CRM Process Components

•Track• Monitor risk indicators and mitigation actions

•Control • Correct risk mitigation plans deviations and decide on

future actions

•Communicate and Document• Provide information to project on risk activities and

current/future risks, and emerging risks


Relationship Among CRM Functions

•Throughout the project life cycle, risk components evolve

• Continuously

• Concurrently

• Iteratively


Risk Management Data Flow

Projectdata

Track

Identify Analyze Plan

Control

List of risks

Statements of risk

Context

Statement of risk

ContextImpactProbabilityTimeframeClassificationRankPlan Approach

Statements of risk

ContextImpactProbabilityTimeframeClassificationRank

Statements of risk

ContextImpactProbabilityTimeframeClassificationRankPlan ApproachStatus

Statements of Risk

ContextImpactProbabilityTimeframeClassificationRankPlan ApproachStatusControl Decision

Master listof risks

TopN

Resources

Resources

Projectdata

Project goalsand constraints

Group/teamuncertainties

Individualuncertainties

Action plans

Risk & mitigationplan measure

Projectdata

Decisions

• replan• close• invoke

contingency• continue

tracking

Status reports

• risks• mitigation

plans

Risk Risk

Risk

Risk

Risk Risk

Class 3

Risk

Classification

Class 1 Class 2

Plan


Continuous Risk Management Application

Testing

Testing

Integration

Fabrication

Code &Debug

DetailedDesign

DetailedDesign

SystemRequirements

Analysis

SystemDesign

PreliminaryDesign

PreliminaryDesign

HardwareRequirements

Analysis

SoftwareRequirements

Analysis

SystemIntegration

& Test

Hardware Development

Software Development

Identify

Analy

ze

Plan

Trac

k

Control

Communicate &Document

Identify

Analy

ze

Plan

Track

Control


Identify

Analy

ze

Plan

Trac

k

Control


And beyond...

FlightOperations

Formulation

Acquisition

Disposal


When Should CRM be done?

Formulation ImplementationApproval

EvaluationEvaluation

Phase APhase A

Preliminary Preliminary AnalysisAnalysis

Phase BPhase B

Definitions Definitions

Phase CPhase C

DesignDesign

Phase DPhase D

DevelopmentDevelopment

Phase EPhase E

Operations Operations

ReviewsReviews

NARNAR PDRPDR CDRCDR SARSAR ORRORR OAROARFRRFRR DRDRSRRSRR


RM Planning/Documentation

•Risk Management planning early in the project life cycle (i.e., formulation) is required per NPG 7120.5 (Section 4.3.2a); NPG 8000.4

•Options

• Stand-Alone Risk Management Plan (Medium-to-Large Projects)

• Risk Management Section in Project Plan (Smaller Projects)


Risk Management Plan Details

•Purpose• Documents the risk management practice (processes, methods,

and tools) to be used for a specific project

•Contents• Introduction/Overview• Project organization, roles, responsibilities• Practice details (e.g., how risks are identified)• Risk management milestones (e.g., quarterly risk list reviews)• Risk information documentation (e.g., database)• De-scope options

•Available Information• SE&T/YA-B, conjunction with SH&IA/QA-C, has established risk

management and project plan templates, and can offer consulting and guidance during plan development


Relationship to Everyday Practice

Learning Continuous Risk Management

is similar to incorporating any new habit

into your daily life.


Core Risk Management Team

Program/ProjectManagement

System Engineering

Safety & MissionAssurance

Risk Management

Board


Who Performs Continuous Risk Management?

•Everyone!


Module 3

IdentifyIdentify

Analy

ze

Plan

Track

Control



Overview

•Identification activities• Capturing statements of risk• Capturing the context of a risk

•Identification methods and tools• Brainstorming• Questionnaires and checklists• Personal knowledge/experience• RM/S&MA analysis tools (FMEA, FTA, PRA)• Lessons Learned


Recording Data on the Risk Information Sheet

•Risk Information Sheet

•Fields to be Completed in Identification Phase:• ID• Date Identified• Risk statement• Origin• Risk Context

ID Risk Information Sheet Date Identified:

Priority/RAC Probability Impact

Risk Statement

Timeframe Originator Classification Assigned to:

Context Approach: Research / Accept / Watch / Mitigate Contingency Plan and Trigger Status Status Date Lessons Learned Approval Closing Date Closing Rationale


Capturing Statements of Risk

•Purpose:• Arrive at a concise description of risk, which can be understood

by everyone and acted upon

•Description:• Involves considering and recording the condition that is causing

concern for a potential loss to the project, followed by a brief description of the potential consequences of this condition


Components of a Risk Statement

•Condition: A single phrase that identifies possible future problems, and describes current key circumstances and situations that are causing concern, doubt, anxiety, or uncertainty

•Consequence: A single phrase or sentence that describes the key, negative outcome(s) of the current conditions

Condition Consequence

Risk Statement

there is a possibility thatGiven the will occur ;


Elements of a Good Risk Statement

•Consider these questions when looking at a risk statement:

• Is it clear and concise?• Will most project members understand it?• Is there a clear condition or source of concern?• If a consequence is provided, is it clear?• Is there only ONE condition, followed by one (or more)

consequence?


Example Risk Statements

Good or bad risk statements?

1. Object Oriented Development (OOD)!

2. The staff will need time and training to learn object oriented development.

3. This is the first time that the software staff will use OOD; the staff may have a lower than expected productivity rate and schedules may slip because of the associated learning curve.


Capturing the Context of a Risk

•Purpose:• Provide enough additional information about the risk to ensure

that the original intent of the risk can be understood by other personnel, particularly after time has passed

•Description:• Capture additional information regarding the circumstances,

events, and interrelationships not described in the statement of risk


Context of a Risk Statement

Risk Statement

Condition ; Consequence

Contributing factors

Risk source

Circumstances

Interrelationships

Context

An effective context captures the what, when, where, how, and why of the risk by describing the circumstances, contributing factors, and related issues (background and additional information that are NOT in the risk statement)


Elements of Good Context

•Consider these questions when looking at the context

• Can you identify which risk statement this context is

associated with?• Is it clear what the source or cause of the risk is? • Is it clear what the impact might be?• Would you know who to assign the risk to for mitigation?

Would they (the person responsible for risk mitigation) know what to do?

• Would you be able to tell if the risk has gone away?


Example Context (#1)

Risk Statement:•This is the first time that the software staff will use Object Oriented Development (OOD); the staff may have a lower-than-expected productivity rate and schedules may slip because of the associated learning curve.

Risk context•It’s a typical NASA project – new concepts without training.

•Is this an example of good or bad context?


Example Context (#2)

Risk Statement•This is the first time that the software staff will use OOD; the staff may have a lower than expected productivity rate and schedules may slip because of the associated learning curve.

Risk Context:•Object oriented development is a very different approach that requires special training. There will be a learning curve until the staff is up to speed. The time and resources must be built in for this or the schedule and budget will overrun.


Risks are Not Problems (1)

•Risk:

• A future event• A potential problem• Has a level of uncertainty (>0% and <100% chance of

occurrence)

•Problem

• Is happening now• Must be dealt with immediately• No uncertainty (it’s occurring now)


Risks are Not Problems (2)

•Risks are anticipated problems• Example: Delivery of Class S parts on schedule is

questionable

•A Problem is a Risk that has occurred• Example: Class S parts have not been delivered

•Problems may create new risks• Change in design, increased testing

• Schedule slip, screening cost


Brainstorming

Purpose:• Group method for generating ideas

Description:• Participants verbally identify ideas as they think of them,

thus providing the opportunity for participants to build upon or spring off of ideas presented by others

BrainstormingListof

Risks

Creative Energy


Risk Statement Identification Tools --

Taxonomy-Based Questionnaire (TBQ)

•Taxonomy – The classification of something in an ordered system that indicates natural relationships; division into ordered groups or categories

•TBQs are questionnaires organized according to the taxonomy of a particular body of knowledge• TBQs provide a structured approach for identifying risks

associated with a project

•CRM Guidebook (pp. 471-493)


Additional Risk Identification

Methods

• Failure Modes and Effects Analysis

• Fault Tree Analysis

• Probabilistic Risk Assessment (PRA)

• Lessons Learned (http://llis.nasa.gov)

• Various Other Checklists


Risk Identification Data Flow

Statement of Risk

Risk Context

List of RisksGroup/TeamUncertainties

IndividualUncertainties

Project

Data

Identify

• Capture statementof risk

• Capture context ofrisk


Risk Information Sheet(After the Identification

Phase)

•Risk Information Sheet (RIS)after the CRM Identify phase

ID 11 Risk Information Sheet Identified: _11/_1/_95_

Priority/RAC Probability Impact

Statement It has recently been decided that the Infrared sensors will be developed in-house and how they will communicate and how sensor data will be processed will be based on assumptions until the detailed design is baselined; the accuracy and completeness of those assumptions will determine the magnitude of change in the IR-SIP Instrument Controller CI and Infrared Sensing Unit CI interface requirements - it could be minor or catastrophic.

Timeframe Origin K. Green

Class Assigned to: __________________

Context The AA program is in the Systems Preliminary Design Phase and the IR-SIP project software is in the Software Specification Phase. This is the first time these sensors will be used on a NASA mission. They will still be

under design and definition during the IR-SIP Controller’s software specification through implementation phases. Therefore, assumptions about the interface will have to be made in implementing the IR-SIP CSCI and if those assumptions are incorrect, then software rewrites will be necessary. We do have access to a reasonable set of assumptions and information from a contractor who has developed very similar sensors, but again, we don’t really feel 100% confident in those assumptions.

Problems were not anticipated in the current success-oriented schedule so there is no slack time if the impact of the changes is major. Schedule slips, cost overruns, and reduction in adequate testing time are all possible if the assumptions prove false.

System testing does not begin until very late in the development, so if problems are encountered there is usually no time to make changes in the hardware. Therefore, software must provide work-arounds for problems encountered.

Mitigation Strategy

Contingency Plan and Trigger

Status Status Date

Lessons Learned

Approval __________________________

Closing Date __/__/__

Closing Rationale


Identification Phase Summary

•Good risk statements:

• Contain ONLY one condition• Contain at least one consequence• Are clear and concise

•Good context:

• Provides further information not in the risk statement• Ensures that the original intent of the risk can be

understood by other personnel, even after time has passed

•Communication is an integral part of risk identification

Risk StatementCondition; Consequence


Module 4

AnalyzeIdentify

Analy

ze

Plan

Track

Control



Overview

•Analysis activities• Evaluating attributes of risk• Classifying risks• Prioritizing risks


Recording Data on theRisk Information Sheet


•Fields to be Completed in Analysis Phase:• Priority• Probability• Impact• Timeframe• Class

ID Risk Information Sheet Date Identified: Priority/RAC Probability Impact

Risk Statement




Evaluating Attributes of Risk

•Purpose• To gain a better understanding of the risk by determining

the expected impact, probability, and timeframe of the risk

•Description• Involves establishing values for:

• Impact: the loss or effect on the project if the risk occurs

• Probability: the likelihood the risk will occur

• Timeframe: the period when you must take action to mitigate the risk (NOT: when the risk will occur)


Levels of Analysis

Level Impact Probability Timeframe

binary level significantinsignificant

likelynot likely

nearfar

tri-level highmoderatelow

highmoderatelow

nearmidfar

5-level very highhighmoderatelowvery low

very highhighmoderatelowvery low

imminentnearmidfarvery far

n-level n levels ofimpact

n levels ofprobability

n levels oftimeframe


Tri-Level Attribute Evaluation Example

•Each attribute has one of three values• Impact: catastrophic, critical, marginal• Probability: very likely, probable, improbable• Timeframe: near-term, mid-term, far-term

•Risk Exposure

Impact

Probability

Improbable Probable Very Likely

Catastrophic

Critical

Marginal


Example: Probability Definitions

•A risk is very likely if there is a >70% probability that it will occur

•A risk is probable if there is a 30-70% probability that it will occur

•A risk is improbable if there is a <30% probability that it will occur


Example: NASA Safety Impact Definitions

•Catastrophic (Class I)• Loss of entire system• Loss of human life• Permanent human

disability

•Critical (Class II)• Major system damage• Severe injury• Temporary disability

•Marginal (Class III)• Minor system damage• Minor injury (e.g.,

scratch)

•Negligible (Class IV)• No system damage• No injury (possibly some

aggravation)


Example: Impact Definitions

Catastrophic Critical Marginal

ScheduleSlip

> 20% 10 – 20% 0 – 10%

CostOverrun

> 15% 5 – 15% 0 – 5%

Technical System islost

Majorfunction

lost

Data lost


Example: Timeframe Definitions

•A risk has a near-term timeframe if the project must take mitigation action in the next 90 days

•A risk has a mid-term timeframe if the project must take mitigation action in the next 90-180 days

•A risk has a far-term timeframe if the project does not need to take mitigation action within the next 180 days


Standardized Agency 5x5 Risk Matrix

MedHigh

Low

Criticality5

4

3

2

1

1 2 3 4 5

LIKELIHOOD

CONSEQUENCES

Very Likely

High

Moderate

Low

Very Low

Very Low Low

ModerateHigh

Very High

NOTE: Specific criteria for each of the Likelihood and Consequence categories are to be defined by each Enterprise or Program. Criteria may be different for manned missions, expendable launch vehicle missions, robotic missions, R&T programs, etc.

Primary Risks


Definition of a Primary Risk

•NPG 8000.4 defined a Primary Risk as those undesirable events (risks) having both high probability and high impact/severity

•Characterization of a Primary Risk as “Acceptable” shall be supported by rationale, with concurrence of the Governing Program Management Council (GPMC), that all reasonable mitigation options (within cost, schedule, and technical constraints) have been instituted


Risk Matrix Application

LEVEL LIKELIHOOD APPROACH

5

4

3

2

1

Very High,

High

Moderate

Low

Very Low

Nearly certain to occur, requires immediate management attention

Highly likely to occur, most cases require management attention

May occur, management required in some cases

Not likely to occur, management not required in all cases

Very unlikely to occur, management not required in most cases

What is the likelihood the situation or circumstances will occur?

LEVEL TECHNICAL PERFORMANCE SCHEDULE IMPACT COST (MILLIONS)

Very Low

Low

Moderate

High

Very High

Minimal impact, overall systemperformance unaffected

Slight impact, overall system performance below goal but acceptable

Moderate impact, system performancebelow goal and unacceptable

High impact, overall system performance below acceptable limits but manageable

Very high impact, system performanceunacceptable, loss of system likely

Minimal, schedule slip

Slight, additionalresources required. Ableto meet dates

Moderate, will miss needdate, crit path unaffected

Moderate impact, budgetincrease btwn x and y *

Major schedule slip,critical path affected

Critical schedule slip,major milestone in jeopardy

Minimal, no significantcost increase

Slight, budget increase

between x and y *

Significant cost impact,budget increase between x and y *

Major cost impact, budget increase between x and y *

If the Risk is realized, what would be the magnitude of the impact?

CO

NS

EQ

UE

NC

E

Lik

elih

oo

d

Consequence

2 3 4 51

23

45

1

HIGH/PRIMARY RISKS

LIK

EL

IHO

OD

LOW LEVEL RISKS

MODERATE RISKS


Example Impact Level Definitions

Impact

RatingSafety

Technical/

PerformanceCost Schedule Programmatic

5 Catastrophic, may

cause death or

permanently disabling

injury

Cannot meet minimum

success criteria

>15%

Over Run

Unrecoverable Project

delay

Forces project

cancellation review

4 Critical, may cause

severe injury or

occupational illness

Major impact to full

mission success

>10%

Over run

Major slip in key

milestone

Major impact to

budget, schedule, or

mission success

3 Moderate, may cause

injury or occupational

illness

Loss of system,

With work-arounds,

moderate impact on

full mission success

>5%

Over run

Minor slip in need

date

Moderate impact to

budget, schedule, or

technical success of

mission

2 Negligible, no adverse

affect to personal

safety or health

Loss of redundancy or

functional

degradation, Minor

impact to full mission

success

Reserves

eroded

Meet need date

with no margin

Can be covered by

reserves, leaves no

contingency for other

Risks, minor impact to

technical success

1 Meets safety

requirements

Component degrades,

minor impact to full

mission success

Minimal -

Begins to erode

reserves

Minimal -

Begins to erode

margins

Minimal budget,

schedule, or technical

impact to project


Example Probability Rating Definitions

Probability Rating

Probability of Occurrence

(cost & schedule )

Probability of Occurrence (performance)

Safety Probability of Occurrence

5 81% – >99% 51% – >99% 10-6 >_ P

4 61% – 80% 31% – 50% 10-3 >_ P >_ 10-6

3 41% – 60% 15% – 30% 10-2 >_ P >_ 10-3

2 21% – 40% 5% – 14% 10-1 >_ P >_ 10-2

1 0< – 20% 0< – 4% 10-6 P > 10-1 >_ P

5

4

3

2

1

Pro

bab

ilit

y

Impact1 2 3 4 5


Classifying Risks

•Purpose:• Look at a set of risks and how those risks relate to each

other within a given structure• Efficiently sort through large amounts of data

•Description:• Involves grouping risks based on shared characteristics.

The groups or classes show relationships among the risks

•Example classificationsSafety ManagementCost EnvironmentalSchedule Security (plus IT Security)Technical Performance Political


Prioritizing Risks

•Purpose:• Sort through a large amount of risks and determine which

are most important• Separate out which risks should be dealt with first (the vital

few risks) when allocating resources

•Description:• Involves partitioning risks or groups of risks based on the

Pareto “vital few” sense and ranking risks or sets of risks based upon a criterion or set of criteria

• Prioritization should utilize the project’s “success criteria”• Prioritization should yield the project’s “Primary Risks”

Master list of risks

TopN


Two Step Risk Prioritization

List of risks*

Prioritized & Ordered Master List of Top N RISKS Select the top %

or N risks

Master listof risks

Top 10%

Top 20%

Order the Top N risks


Risk Prioritization Using Multivoting

•Multivoting is a technique for prioritizing a subset of items from a larger set (usually one-third of the items on a list)

•Each evaluator in a group gets a certain number of votes (e.g., 3) to use for risk prioritization

•Each evaluator then assigns a number between “1” and “3” (i.e., the number of votes they have) to the risks they feel are most important (e.g., a “3” is a higher priority risk than a “1”)

•Totals for each risk are tallied – highest priority risks are those with the most points


Multivoting Example

Example: Risk order of criticality: 5 participants H B A C J 12 risks 3 weighted votes (1 2 3)

Risk:Evaluator A B C D E F G H I J K L

Eval1 3 1 2 Eval2 3 2 1Eval3 2 1 3 Eval4 1 2 3Eval5 2 1 3

TOTAL 6 7 3 13 1


Risk Prioritization Using a Risk Assessment Code (RAC)

•The use of Risk Assessment Code (RAC) is an additional, qualitative method for prioritizing risks

•The RAC is a tool used to express comparative risks in all categories by evaluating both the potential severity of a condition and the probability of its occurrence

•RACs can be utilized to determine a project’s “Primary Risks” (i.e., those risks with both a high probability and a high impact) [e.g., those risks with a RAC of 1 or 2]

•RACs can be assigned in a tailored manner to meet the needs or complexity of a program or project


Risk Assessment Code (RAC)[Project Example]

Very Low Low Moderate High Very HighVery High 5 10 15 20 25High 4 8 12 16 20Moderate 3 6 9 12 15Low 2 4 6 8 10Very Low 1 2 3 4 5

Likelihood Estimate

Impact/Severity

High risk Medium Low

•RACs are assigned a number based on the risk exposure (product of probability times impact), with each attribute level having a score from 1 to 5 in this risk matrix

•For this example, the RED (High Risk) risks having a value of 15-25 would be classified as Primary Risks


Risk Analysis Data Flow (1)

Risk I P T

Risk a M M F

Risk b M L N

Risk c L H N

. . .

Risk I P T

Risk set A H M F ----- -----

Risk b M L N

Risk c L H N

. . .

Risk I P T

Risk n H H N

Risk s H M N

Risk set A H M F ----- -----

Risk c L H N

Top N1.2.3.. . .

Evaluate:•Impact (I)•Probability (P)•Timeframe (T)

Classify:•Identify duplicates•Consolidate risks to sets

Prioritize:•Identify Pareto top N•Rank top N•Determine RACConsolidate

risks

Sort by evaluation results

Rank orderthe Paretotop N

Pa

reto

to

p N

RAC


Risk Analysis Data Flow (2)

Master list of risks

TopN

Risk Risk

Risk

Risk

Risk Risk

Class 3

Risk

Classification

Class 1 Class 2

Statement of risk

ContextImpactProbabilityTimeframeClassificationRank

List of risks

Statement of risk

Context

Analyze• evaluate

• classify

• prioritize


Sample Risk Management Data Flow

SafetyRisk

Injury to Personnel or Damage to Equipment

Ranked Risk List and Matrix

MissionSuccess

Technical Performance

RiskProbabilistic Risk

Assessment

Ranked Risk List and Matrix Programmatic

Implementation Risk

Product Quality, Schedule, & Cost


RiskManagement

BoardRanked Risk List

and MatrixResourceAllocation

HighMedium

Low

Safety Hazard AnalysisMonthly Reports

What Can Go Wrong ThinkingProblem Reporting

Fault Tree Analysis (Top Down)FMEA (Bottom Up)

Reliability Block Diagram (Predication)What Can Go Wrong Thinking

Problem Reporting

Monthly ReportsTelecons and Status Reports

Schedule & Pert Chart AssessmentCost vs. Schedule Assessment


MASTER RISK LIST


Risk Information Sheet after Analyze Phase

•Risk Information Sheet (RIS)after the CRM Analyze phase

ID 11 Risk Information Sheet Identified: _11/_1/_95_

PriorityRAC Probability Impact

10/2 M H


Timeframe N Origin K. Green

Class Requirements

Assigned to: __________________





Mitigation Strategy

Contingency Plan and Trigger

Status Status Date

Lessons Learned

Approval __________________________


Closing Rationale


Analyze Phase Summary (1)

•Evaluate risks at a level that is sufficient to determine the relative importance

•Select attribute definitions (e.g., catastrophic impact, RAC 1) that make sense for your project – document these in the project Risk Management Plan

•Classify risks to help the project understand the risks

•Group related risks into sets to help build more cost-effective mitigation plans


Analyze Phase Summary (2)

•Prioritize to determine which risks should be dealt with first when allocating resources

•Prioritize the risks based on the criteria for what is most important to the project

•Communication is central to• Defining project evaluation definitions• Evaluating risks• Selecting a project classification scheme• Classifying risks• Defining prioritization criteria• Identifying and prioritizing the Top N risks


Module 5

PlanIdentify

Analy

ze

Plan

Track

Control



Overview

•Risk Planning activities• Assigning responsibility• Determining risk mitigation approach• Defining scope and actions

•Mitigating a set of related risks


What Is Risk Planning?

•Risk planning is the function of deciding what, if anything, should be done with a risk

•Risk planning answers the questions:• Who does planning?• Is it my risk? (responsibility)• What can I do? (approach)• How much and what should I do? (scope and actions)




•Fields to be Completed in Risk Planning Function:• Assigned to• Mitigation Strategy• Contingency Plan and Trigger


Risk Statement




Risk Planning Decision Flowchart

Statement of risk

Review Risks

Is it mytask to deal

with the Risk?

Do I knowenough

about thisRisk?

Transfer Keep

YesNo

ResearchNo

Can Ilive withthis Risk?

Can Iact on

this risk?

Is an actionItem listEnough?

Yes

Yes

No

Accept

Watch

No

Mitigate

ScheduleWBS

ScheduleWBS

Task PlanResponsibility

GoalsTasks

Task PlanResponsibility

GoalsTasksRisk Action Item List

Item 1-do xxxxItem 3-do yyyyItem 7-do zzzz

Risk Action Item ListItem 1-do xxxxItem 3-do yyyyItem 7-do zzzz

NoYes

Yes

Responsibility:Is it my risk?

Approach: Can I do Anything?

Scope and actions:What should I do?


Project Considerations Regarding Risk Planning

•What are the risk attributes? • Is it a Primary Risk?•What is currently important to the project, management, customer, or user?

•Are there critical milestones the project is currently is facing?

•What limits or constraints do the project, organization, or manager have?

•What resources are available for mitigation?•How does the risk fit into the overall project issues and concerns? When is the best time to address or mitigate a risk?


Assigning Responsibility for Risk Planning

•Purpose:• Ensure that no risks are ignored• Make effective use of expertise and knowledge within the

project when planning for risk mitigation• Ensure that risks are being managed by those with the

appropriate abilities, knowledge, and authority to commit resources for mitigation

•Description:• Involves reviewing the risk(s) and determining who is best

able to deal with the risk(s)


Determining Risk Planning Approach

•Purpose:• Ensure you know enough to make an informed decision• Pick an appropriate approach for effective management of the

risk(s)• Establish measurable mitigation goals that provide a target for

evaluating success and direction during the development of action plans

•Description:• Involves reviewing the risk(s) and determining the best

approach to take


Risk Planning Approaches

Risk Planning(Approaches/types)

Options:

Notes:(1) Documented on Risk Information Sheet (PREFERRED APPROACH)

(2) Separate Plan – Used for significant, complex risks with multiple mitigations (OPTIONAL APROACH)

Research Research Planning

Accept Watch

TrackingRequirements

Acceptance Rationale

Mitigation Planning

Mitigate

Strategy/Actions (Note 1)

MitigationTask Plan (Note 2)


Contingency Planning

•Not all risk mitigation strategies can or should be carried out immediately, for example:• There may not be sufficient funding at this time• Other circumstance (available personnel) may not be right• Risk may be low probability, catastrophic impact with an

expensive mitigation strategy• Current strategy might not be working

•May need to develop a contingency risk mitigation strategy (to be used as Plan B if Plan A fails)

•Contingency risk mitigation strategy plans are held in reserve until specific trigger conditions are true or certain events occur• Watch for the conditions and events!


Defining Scope and Actions for Risk Mitigation Strategies

•Purpose:• Take a balanced approach in developing effective actions to

mitigate risks

•Description:• Involves reviewing the risk(s) and determining the appropriate

level of mitigation to take and the goal of the mitigation


Determining Risk Mitigation Goals and Success Measures

•There is a need to set realistic, measurable (or verifiable) goals for mitigating individual risks:

• Avoid changes to scheduled milestones• Eliminate change requests unsupported by funding to

implement the change

•Define mitigation success criteria – it is important to know when you’ve succeeded or failed in mitigating risks

• All current change requests implemented by DD/MM/YY, with no change to scheduled milestones


Discussion -- Risk Mitigation Goals and Success Measures

Risk 7•Science requirements have substantial TBDs; late completion of TBDs likely, with reduction in adequate testing time, possible science application software failure, incorrect science data being captured, hardware damage if incorrect safety limits were provided, extensive rework and substantial cost overruns, mission failure if problems not found before system is in operation

•What risk mitigation goals and success measures would you look for?


Risk Planning Data Flow

Risk Risk

Risk

Risk

Risk Risk

Class 3

Risk

Classification

Class 1 Class 2StrategiesActionsTriggers

Statement of risk


Project goalsand constraints

Resources

Master listof risks

TopN

Statement of risk *ContextImpactProbabilityTimeframeClassificationRank

Consequences may be addedto the risk statement if not already documented

*

Plan

• Assign responsibility

• Determine approach

• Define scope and actions


Risk Information Sheet(After the Risk Planning

Phase)

•Risk Information Sheet (RIS)after the CRM Risk Planning phase

ID 11 Risk Information Sheet Identified:

_11/_1/_95_ Priority Probability Impact

7 M M


Timeframe N Originator K. Green

Class Requirements

Assigned to: J. Johnstone





Approach: Research / Accept / Watch / Mitigate [Mitigation goal/success measures: Reduce the probability and impact of incorrect interface assumptions to a minimum: estimated low probability and low impact. Ideally, completion of prototype tests will show that assumptions we got from EasySensor were correct and there is no impact at all.]

1. Build prototypes of the IR-SIP CSCI software primitives needed to control the interface with the Infrared Sensing Unit early in the software requirements phase.

Start by 1/10/96. Prototype should contain all the functionality defined by that date for the configuration of the Infrared Sensing Unit. Complete by 1/30/96.

2. Have early interface tests with the Infrared Sensor Unit to confirm functionality and control issues. Allocate enough time for software work-arounds to be developed if problems arise.

Mitigation Strategy (cont.)

Test of the interface between the two subsystems will be completed by 2/3/96.


Risk Information Sheet(After the Risk Planning

Phase)

•Risk Information Sheet (RIS)after the CRM Risk Planning phase

Mitigation Strategy (cont.)

Test of the interface between the two subsystems will be completed by 2/3/96.

Second prototype to command the transmission of sensor data from the Unit to the IR-SIP CSCI will be started by 2/12/96 and completed by 2/20/96.

All subsequent interface tests will be performed by 2/28/96.

1. Feed information from the two prototype tests into updates to the Interface Requirements Specification and the associated sections of the schedule by 3/2/96.

2. Determine the impact of the revised requirements by 3/6/96. Contingency Plan and Trigger

Trigger: If the 2/12/96 or 2/28/96 dates cannot be met, put the contingency plan in place.

Contingency Plan: Elevate this as one of the top 10 project risks and request that project reserves be used to pay for additional contract support to get the two sets of requirements firmed up (i.e., configuration and data transfer). If additional contract resources are not available, slip the schedule for completion of the prototypes to be done by March 20, and request that project reserves be used to pay for additional resources to be added to the software design and implementation to make up the schedule slip.

Status Status Date

Approval __________________________


Closing Rationale


Risk Planning Phase Summary (1)

•The result of risk planning is a documented decision about what should be done with each risk

•The risk planning approach, as well as the mitigation strategy-related details are documented on the Risk Information Sheet. The risk planning approaches and their related risk planning documentation are:

• Research (Research Planning)• Accept (Acceptance Rationale)• Watch (Tracking Requirements)• Mitigate (Mitigation Strategy, Actions, Completion Dates,

Triggers, Contingency Strategies)


Risk Planning Phase Summary (2)

•Mitigate unacceptable risks to the project

•You can’t mitigate all risks – but you need to understand which risks you are taking

•Watch the risks that you can’t currently mitigate and don’t want to accept

•Unassigned tasks tend to fall through the cracks

•Don’t over plan – documentation of mitigation strategy actions on the Risk Information Sheet is sufficient for almost all identified project risks


Module 6

TrackIdentify

Analy

ze

Plan

Track

Control



Overview

•Tracking activities• Acquiring Data• Compiling and Evaluating Data• Reporting Status (planning, risks)


What do We Mean by Tracking?

•Tracking• A process for watched and mitigated risks where related

data are acquired, compiled, analyzed, and reported

•Risks can be tracked individually or in sets




•Fields to be Completed or Updated in the Tracking Function:• Priority• Probability• Impact• Timeframe• Status• Status Date


Risk Statement




Tracking Risks and Mitigation Strategies

•Tracking risk mitigation strategies will indicate• Whether the mitigation strategy is being executed correctly• If risk mitigation is on schedule (including action items)

•Tracking individual risk attributes will indicate• Mitigation strategy effectiveness• Is the impact/probability reduced?


Risk Metrics

•Risk Metrics are used to:• Measure attributes of a risk

- Impact, probability, and timeframe- Other risk- or project-specific attributes

• Provide meaningful information to enable more informed control decisions

• Assess the impact or success of a mitigation strategy• Identify new risks (if indicated by the risk metrics)


Acquiring Data

•Purpose• To collect tracking data for a given risk

•Description• A process that includes all of the steps associated with

collecting information about and updating the values of risk measures and status indicators for watched and mitigated risks


Considerations When Acquiring Data

•Status information is only as good as its accuracy and timeliness

•Stale data are more dangerous to decision makers than no data at all

•When a group of indicators is required, all of the data must be acquired from the same time period

•Collect just the data needed to track the project’s risks. Collect only what you need and use what you collect


Data Acquisition (Metrics Examples)

•Requirements• Ambiguity = Weak Phrases and/or Optional Language• Measure of Completeness = TBD + TBA + TBR

•Design and Implementation• Structure/Architecture = Complexity and Size

•Testing• Problem Reporting Tracking = Open, Closed, Severity• Defect Density

•Process• Schedule = Effort, Completion Rates• Budget


Compiling and Evaluating Data

•Purpose• Organize and understand the relevant tracking data for a

given risk

•Description• A process in which data for a given risk is combined,

calculated, organized, and interpreted for the tracking of a risk and its associated mitigation strategy


Triggers/Thresholds

•A value of an indicator that specifies the level at which an action, such as implementing a contingency plan, may need to be taken

•Triggers and thresholds are generally used to• Provide early warning of an impending critical event• Indicate the need to implement a contingency plan to

preempt a problem• Request immediate attention for a risk

•Triggers and thresholds are effective if• They do not trip unnecessarily• They are easy to calculate and report


Example – Use of Triggers

Overbudget

Underbudget

acceptable

Within Budget

Risk 100:Project resources (personnel number and availability) and schedules were underestimated; schedule slips, cost overruns, reduction in adequacy of development processes (especially testing time adequacy) likely.

Percent within Budget

-15.0%

-10.0%

-5.0%

0.0%

5.0%

10.0%

15.0%

20.0%

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep


Process (Metrics Example)

•Risk # 6: Project software schedule and resources were underestimated; schedule slips, reduction in adequate testing time

•Data to be collected• Effort per activity

•Trigger• Exceeds expected percentages


Process Metrics Example(Effort per Life Cycle Phase)

Req/Design

30%Test

33%

Development37%

Actual/Projected Effort

Req/Design

34%

Development48%

Test

18%

Risk - Decrease in Testing projected

Standard

Current Status


Reporting Data

•Purpose• Communicate risk status reports to support effective

decision making (inputs for the “Control” function)

•Description• A process in which status information about risks and

mitigation strategies is communicated to decision makers and team members


Reporting Considerations

•What information needs to be reported?

•What presentation formats best present the analyzed data?

•Does the information and the format of the report provide the basis needed by decision makers?


Sample Risk Management Data Flow

SafetyRisk

Injury to Personnel or Damage to Equipment


MissionSuccess

Technical Performance

RiskProbabilistic Risk

Assessment

Ranked Risk List and Matrix Programmatic

Implementation Risk

Product Quality, Schedule, & Cost


RiskManagement

BoardRanked Risk List

and MatrixResourceAllocation

HighMedium

Low

Safety Hazard AnalysisMonthly Reports


Fault Tree Analysis (Top Down)FMEA (Bottom Up)

Reliability Block Diagram (Predication)What Can Go Wrong Thinking

Problem Reporting

Monthly ReportsTelecons and Status Reports

Schedule & Pert Chart AssessmentCost vs. Schedule Assessment


MASTER RISK LIST


Standardized Agency 5x5 Risk Matrix

MedHigh

Low

Criticality5

4

3

2

1

1 2 3 4 5

LIKELIHOOD

CONSEQUENCES

Very Likely

High

Moderate

Low

Very Low

Very Low Low

ModerateHigh

Very High

NOTE: Specific criteria for each of the Likelihood and Consequence categories are to be defined by each Enterprise or Program. Criteria may be different for manned missions, expendable launch vehicle missions, robotic missions, R&T programs, etc.

Primary Risks


Top Risk List and Risk Matrix Example

Top Risk List & Risk Matrix (Example)

Approach

M - Mitigate

W - Watch

A - Accept

R - Research

5

4

3

2

1

1 2 3 4 5

LIKELIHOOD

CONSEQUENCES

Program/Project Name from date last review to date current review

Med

High

Low

Criticality L x C Trend

Decreasing (Improving)

Increasing (Worsening)

Unchanged

New Since Last PeriodNew

Unachievable Thruster Technology

MEngine-15

10New

Reboost Engine/Thruster Lives

MEngine-07

9New

12 Year Life CertificationWEngine-09

8

Returnability RequirementREngine-04

7

Design Launch Weight Exceeds the Shuttle Capacity

MEngine-01

6

Hot Fire Test Schedule SlipMEngine-05

5New

Government Furnished Property (GFP)

MEngine-08

4

Aggressive ScheduleMEngine-03

3

On-Orbit Propellant TransferWEngine-06

2

Thermal Vacuum/Acoustic Test

MEngine-02

1

Risk TitleApproach

RiskID

Ran

k

LxCTrend

1 2

3

4 5

6

7 8

9

10


Criticality

Unspecified

Planned Closure

02/02/06

Planned ClosureCriticality

On-Orbit Propellant Transfer

• Given NASA has never transferred propellant on orbit, there is a risk that technical surprises could lead to significant operational, cost, and schedule impacts.

Thermal Vacuum/Acoustic Test

• Given the lack of thermal vacuum and acoustic element level testing, there is a possibility that the PM may not perform on-orbit as designed

Risk Statement

(Title & Detailed Description)

Astronaut Office has requested to be kept abreast of potential risk.

Watch

• If design utilizes change-out of tanks on orbit initiate mitigation plan to limit risk.

Engine-062

NoneMitigate

• Perform trade study to investigate feasibility of additional thermal vac testing

Engine-021

CommentsApproach & Plan

Risk

IDRank

Risk Focus Chart Example

Risk Criticality

Program/Project Name as of date current review

M

H

L

H

H

Risk Focus Chart Example


Stoplight / Fever Chart

Condition/Change

RiskID

RiskStatement

AssignedTo

PlanningApproach

Remaining Milestones

Comments

Yellow

Green

Red

14

7

6

Contracting different test facility; insufficient testing, damage.

Science reqt substantial TBDs; late completion, incomplete testing, wrong data.

SW schedule and resources under estimated; schedule slips, cost overruns.

$$$


Reporting Schedule

•Reports are generally delivered as part of routine project management activities:

• Weekly status meetings• Monthly project meetings

•The frequency of reporting depends on:• The reporting requirements for each risk or risk set• The manner in which the report will be used

•Exception reporting may be necessary


Risk Tracking Data Flow

Resources

Action Plans

Statement of Risk


Project

Data

Risk & MitigationPlan Measure

Status Reports

• Risks• Mitigation

Plans

Statement of Risk

ContextImpactProbabilityTimeframeClassificationRankPlan ApproachStatusMetrics

Acquire Compile Report

Track


Tracking Phase Summary

•Tracking reports communicate information required for effective control decisions

•Tracking information and reports can include quantitative indicator data as well as more subjective information (e.g., recommendations)

•Tracking information is not limited to formal reporting mechanisms

•Informal reporting of risk-related information by all project personnel can aid decision making

•Risk tracking should be integrated with standard management practices – risk management should be tailored for a project


Module 7

ControlIdentify

Analy

ze

Plan

Track

Control



Control Overview

•Control activities• Evaluate tracking results• Decide on course of action• Execute control actions


What is Control?

•Control• A process in which decisions are made based on the data

presented in the tracking reports

•Risks can be controlled individually or in sets


What Is Effective Control?

•Monitoring the quality of risk mitigation execution

•Assessing the effectiveness of mitigation strategies

•Assessing significant changes in risks and trends

•Determining appropriate responses

•Executing the plan of attack

•Communicating the above information




•Fields to be Completed in Risk Control Function:• Approval• Closing Date• Closing Rationale• Status


Risk Statement




Evaluate Tracking Results

•Purpose• Allows decision makers to identify significant changes in

risks, to assess the effectiveness of mitigation strategies, and to accurately determine the best courses of action

•Description• Uses tracking data to reassess project risks for trends,

deviations, and anomalies


Metric Trend Analysis

•The Risk Management Plan can document which project metrics to track

•Trend and data analysis of project metrics can be used to identify new risks

•Trends can be observed through the evaluation of successive reports

• Persistent lateness in taking action• Oscillating priority values• Significant changes in the number of high impact risks or

risks of a particular type


Trending Metrics Example

•Given concerns about unstable or incomplete requirements, which metrics might be useful in controlling this risk area?

•Risk #7 – Science requirements have substantial TBDs; late completion of TBDs likely, with reduction in adequate testing time, possible science application software failure, incorrect science data being captured, hardware damage if incorrect safety limits were provided, extensive rework and substantial cost overruns, mission failure if problems not found before system is operational

•What data would you collect? What trends would you expect to see evolve?


Requirements Metrics (Sample Solution)

Completeness and Volatility Analysis

CRR Looks Good!

(Stable)

CRR Excessive Changes!

NOT Stable

Modifications to Requirements

0

50

100

150

200

250

300

350

400

450

1Q94 2Q94 3Q94 4Q94 1Q95 2Q95 3Q95

Calendar Quarter

Qu

anti

ty

New

Modified

Deleted

Total Number of Requirements

0

100

200

300

400

500

600

700

800

900

1000

1Q94 2Q94 3Q94 4Q94 1Q95 2Q95 3Q95

Calendar Quarter

Qu

anti

ty

Combination of BOTH views indicates risk area - requirements are NOT YET stable


Decide on Course of Action

•Purpose• Ensure that project risks continue to be managed

effectively

•Description• Uses tracking data to determine how to proceed with

project risks- Close the risk- Continue tracking and executing the current mitigation

strategies- Re-plan risk mitigation- Invoke an alternate mitigation strategy (e.g., use a

contingency plan)


Execute Control Actions

•Purpose• Implement both the decision made about a risk and

mitigation strategy as well as to ensure that all decisions are appropriately documented for future reference and historical record maintenance

• Ensure approval and resources are allocated

•Description• The process where control decisions are implemented


Risk Control Data Flow


Statement of Risk

Decisions Replan Close Invoke

contingency Continue

tracking

Status Reports Risks Mitigation plans

Project Data


Statement of Risk

Control• evaluate

• decide

• execute


Risk Information Sheet (After Tracking and Control

Phase)ID 11 Risk Information Sheet Date Identified: Priority/RAC 7/3 Probability M Impact M

Risk Statement It has recently been decided that the Infrared sensors will be developed in-house and how they will communicate and how sensor data will be processed will be based on assumptions until the detailed design is baselined; the accuracy and completeness of those assumptions will determine the magnitude of change in the IR-SIP Instrument Controller CI and Infrared Sensing Unit CI interface requirements - it could be minor or catastropic.

Timeframe N Originator K. Green

Classification Requirements

Assigned to: J. Johnstone

Context The AA program is in the Systems Preliminary Design Phase and the IR-SIP project software is in the Software Specification Phase. This is the first time these sensors will be used on a NASA mission. They will still be under

design and definition during the IR-SIP Controller's software specification through implementation phases. Therefore, assumptions about the interface will have to be made in implementing the IR-SIP CSCI and if those assumptions are incorrect, then software rewrites will be necessary. We do have access to a reasonable set assumptions and information from a contractor who has developed very similar sensors, but again, we don't really feel 100% confident in those assumptions.


System testing does not begin until very late in the development, so if problems are encounted there is usually no time to make changes in the hardware. Therefore, software must provide work-arounds for problems encountered.

Approach: Research / Accept / Watch / Mitigate Mitigation goal/success measures: Reduce the probability and impact of incorrect interface assumptions to a minimum: estimated low probability and low impact. Ideally, completion of prototype tests will show that assumptions we got from EasySensor were correct and there is no impact at all. Mitigation Strategy 1. Build prototypes of the IR-SIP CSCI software primitives needed to control the interface with the Infrared Sensing Unit early in the software requirements phase. Start by 1/10/96. Prototype should contain all of the funtionality defined by that date for the

configuration of the Infrared Sensing Unit. Complete by 1/30/96. 2. Have early interface tests with the Infrared Sensor Unit to confirm functionality and control issues. Allocate enough time for software work-arounds to be developed if problems arise. Test of the interface between the two subsystems will be completed by 2/3/96. Second prototype to command the transmission of sensor data from the Unit to the IR-SIP

CSCI will be started by 2/12/96 and completed by 3/2/96. All subsequent interface tests will be performed by 2/28/96.

3. Feed information from the two prototype tests into updates to the Interface Requirements Specification and the associated sections of the schedule by 3/2/96. 4. Determine the impact of the revised requirements by 3/6/96. Contingency Plan and Trigger Trigger: If the 2/12/96 or 2/28/96 dates cannot be met, put the contingency plan in place. Contingency Plan: Elevate this as one of the top 10 project risks and request that project reserves be used to pay for additional contract support to get the two sets of requirements firmed up (i.e. configuration and data transfer). If additional contract resources are not available, slip the schedule for completion of the prototypes to be done by March 20, and request that project reserves be used to pay for additional resources to be added to the software design and implementation to make up the schedule slip. Status Status Date Interface tests in progress - no significant difficulties found so far. 2/20/96 Expected completion of tests on 2/26. Second prototype complete 2/7/96 Testing of the interface complete, ran a bit late but no significant 2/4/96 difficulties found with the interface First prototype complete 2/1/96 Lessons Learned Approval Closing Date Closing Rationale


Risk Control Phase Summary

• Control Decisions are based on current information as well as experience, and are required to respond to changing conditions in watched and mitigated risks

• Risk tracking and control should be integrated with standard project management practices – risk management should be tailored for a project


Module 8

Communicate & Document

Analy

ze

Plan

Track

Control

CommunicateDocument

Identify


Overview

•What is communication?

•Relationship to other CRM Process functions

•Enablers to communication

•Barriers to communication

•Documentation of risks


Relationship To Other CRM Process Functions


Why Communicate Risks?

•Makes risks, plans, actions, concerns, exchanges, forecast, and progress known

•Ensures the visibility of risk information•To enable all project members to participate in defining and managing risks

•Ensures understanding of risks and mitigation plans

•Establishes an effective, ongoing dialog between the manager and the project team

•Ensures appropriate attention is focused on issues and concerns


Why Document Risks?

•In order to track and collect risk information internally and externally

•Risks have a life cycle and eventually all risks go away• Probability or impact goes to zero• Risk becomes a problem

•Documenting the life cycle of risks• Helps you learn what worked and didn’t work• Should help you avoid similar difficulties• Provides the opportunity to help other projects learn from

your experience (Input to Lessons Learned Database)


Risk Documentation Types

•Risk Management Plan (*NPG 7120.5B, NPG 8000.4)•Risk Implementation Plan•Risk Information Sheets (*)•Prioritized Risk List (*NPG 7120.5B)•Risk Profile•Risk Analysis Reports•Risk Mitigation Status Reports•Risk Databases (*)•Spreadsheet Risk Tracking Logs

(*) Recommended for KSC risk management activities


Module 9

Implementing Continuous Risk Management

Identify

Analy

ze

Plan

Track

Control



Overview

•Frequently asked CRM implementation questions• When do I start?• Who’s involved?• What do I need?• What should I choose?• What actions should I take?

•Hints and Tips

•Things to watch out for


When do I Start CRM?

Opportunity Actions

Pre-contract activity Include risk management provisions in the solicitation and statement of work.

Major project milestones(e.g., contract award)

Prepare for a major project decision point,and the need to increase knowledge aboutrisks for improved strategic planning.

Major project review Prepare for standard reviews, such as design reviews, functional tests.

Best time to start is at the beginning!


Who’s Involved? (1)

Role/Description Responsibilities and Tasks

Change agents(plan and implement changes in organizations and projects

• Assist with recommendations of plans• Evaluate existing and new tools

Sponsor(e.g., senior mgr., VP, division chief)

• Provide visible support and encouragement• Reward effective management of risks

Project manager(responsible for ultimate success of project)

• Provide resources and funding• Reward effective management of risks• Monitor progress

Champion(advocates new technology or process within the project)

• Publicize and promote CRM• Coordinate changes and improvements on the project


Who’s Involved? (2)

Role/Description Responsibilities and Tasks

Project personnel(e.g., software or hardware engineers, testers, etc.)

• Add CRM activities to day-to-day operations• Maintain open communication about risks

Facilitators(trained in meeting skills, conflict resolution, tools, etc., - act individually or as a team)

• Conduct training sessions• Provide CRM expertise• Provide consulting during evaluation of progress

Technical managers (e.g., team or functional leads, such as software/hardwaremanager, test mgr., etc.)

• Encourage and support use of CRM within their teams• Report risk information to project manager• Evaluate progress within their teams


Core Risk Management Team

Program/ProjectManagement

System Engineering

Safety & MissionAssurance

Risk Management

Board


Core RM Team Key Responsibilities (1)

• Reviews current and previous risk issues to validate, classify and group risks at the project level

• Determines risk attributes (probability, impact, and timeframe) and reprioritizes risks as needed

• Determines if addition information is needed (trade-studies or research)

• Implements mitigation options (contingency plans, descope plans). Determines who is involved and assigns risk owner. Reassigns risks when necessary

• Adjusts action planning (mitigation plans, mitigation time frames, etc)


Core RM Team Key Responsibilities (2)

• Tracks, communicates and controls risks• Makes control decision recommendations

(analyze, decide, execute) to PM for all risks• Makes recommendations to PM regarding

CRM policy and the communication of risks • Makes recommendations to PM regarding

risk closure and/or acceptance• Make recommendations to PM concerning

allocations of resources to mitigate risks


You need . . . Organization Structure, Internal/External

Communication

EngineersTesters

Software

manager

Hardware

manager

ProjectManager

Integration/

test manager

Software

engineers

Configuration

management

lead

Qualityassurancemanager

Systemengineermanager

Projectmanager

Technicalleads

Individuals/team members

Identify Track

Risks

Top Nrisks

Status/forecast

Status/trends

Assignresponsibility

Control

- review- reprioritize

- integrateacross teams

Requiredindicators

Analyze

- evaluate

- classify

- review- prioritize

Plan

- approve plans- recommend- accomplish

DecisionsProject

Top N

Multi-project Integration

Senior Managers

Project

Awareness

Issue

Customer

resolution

Awareness

Risk

Suppliers

mitigation

Selected

Top N

Decisions,

Agreements

Selected

Top N

Mitigation plans,

Status reports

Organization Structure

ExternalCommunication

InternalCommunication


Risk Management Planning Documentation Options

•Risk Management Plan • Guides project personnel through the tailored process,

methods, and tools for managing their project risks• Recommended and preferred risk management planning

documentation approach for KSC projects

•Risk Management Implementation Plan• Guides project personnel (in the pre-formulation phase)

regarding how they intend to bring risk management into the project’s infrastructure and processes

• Utilized primarily for extremely large programs/projects, where there is a need to establish sponsorship, resources, and infrastructure for the overall risk management approach

• Development of a Risk Management Implementation Plan is not envisioned for KSC projects (Refer to Tab L [Module 11] for an example plan)


Risk Management Plan Elements

(NPG 8000.4, Section 2.7.4)•Introduction/Overview of Risk Management process

•Project Organization and Responsibilities•Risk Management activities and practices in detail

•Budget, resources, and milestones for risk management activities and risk mitigation

•Procedure for documenting risks•Assumptions•Technical Considerations•Constraints•De-Scope Options

Overview

Budget

Procedures

Goals

Milestones


Sample Risk Management Plan

•KSC’s Advanced Technology Development Center (ATDC) Risk Management Plan can serve as an example for your project to use

•Take a few minutes to read the ATDC Risk Management Plan (After Module 11)

RiskManagementPlan


You need to . . . Utilize Established Project Meetings

•Weekly Team Meetings• Establish priority of team’s risks• Assign responsibility for new risks• Review and approve mitigation strategies

•Monthly Project Meetings• Presentation of the team’s Top N risks (and mitigation

strategies)• Decisions of appropriate risk mitigation actions• Determination regarding allocation of resources to

implement risk mitigation strategies

•NOTE: The above are examples – your project team should utilize existing meetings, NOT separate Risk Management meetings


You need a. . . Defined Process

and Data Flow

Legend:

Processactivity

Outputor data

Decision

Individual activities Periodic meetings

DecisionClose riskTake planned actionContinue trackingReplan

Plan:Define mitigationapproachDeterminemitigation plan

Riskstatement& context

Riskmitigationplans

Risk class,probability,impact, &timeframe

Statusreports

Prioritizedlist of risks

Assignments,approved plans

Analyze:Prioritize risks

Analyze:

Classify risksEvaluate risks

Plan:AssignresponsibilityApprove plans

Control risksTrack risks

Individual activities Periodic meetings

Identify risks

Example:


• Baseline Planning• Planning Decision Flowchart• Planning Worksheet• Problem-Solving Planning

- Affinity Grouping - Brainstorming- Cause and Effect Analysis- Cost-Benefit Analysis- Gantt Charts- Goal-Question-Measure- Interrelationship Digraph- List Reduction- Multivoting- PERT Chart- Work Breakdown Structure

• Risk Information Sheet

• Action Item List

Control• Cause and Effect Analysis• Closing a Risk• Cost-Benefit Analysis • List Reduction• Mitigation Status Report• Multivoting• PERT Chart• Problem-Solving Planning• Risk Information Sheet• Spreadsheet Risk Tracking • Stoplight Chart

Analyze• Affinity Grouping• Baseline Identification and Analysis• Binary Attribute Evaluation• Comparison Risk Ranking• Multivoting• Pareto Top N • Potential Top N

• Risk Information Sheet• Taxonomy Classification

• Top 5• Tri-level Attribute Evaluation• FMEA• FTA

Plan

Track• Bar Graph• Mitigation Status Report• Risk Information Sheet• Spreadsheet Risk Tracking• Stoplight Chart• Time Correlation Chart• Time Graph

Risk Management PlanA Risk Management Plan documentshow risks will be managed on aproject: the process, activities,milestones, and responsibilities

associated with risk management. It is a subset of theproject plan and is written before the project begins.

Identify• Baseline Identification and Analysis• Brainstorming• Periodic Risk Reporting• Project Profile Questions• Risk Information Sheet• Short TBQ

• Taxonomy-Based Questionnaire (TBQ)• TBQ Interviews • Voluntary Reporting• Project Metrics• Failure Modes & Effect Analysis (FMEA)• Fault Tree Analysis (FTA)

• Project Metrics• Statistical Process Control (SPC)

• Project Metrics

Identify

Analy

ze

Plan

Track

Control

Communicate Document

You must choose your . . . Methods and Tools


•Purpose:• Make maximum use of existing, effective project

management processes and methods while integrating a set of proactive risk management activities

• Document the tailored processes, methods, and tools in a risk management plan

• Define a schedule for transitioning specific methods, tools, and activities into the project

•Description:• Tailors risk management processes, methods, and tools for

use on the project

You should carefully . . . Adapt CRM to Your Project


You should choose a . . . Risk Database

•A database is the simplest means to retain and keep risk information current

•Data entry forms and reports can be used as the risk information sheet, spreadsheet, and other templates

•A risk database enables documentation of lessons learned, trend analysis, and pattern analysis to support identifying common risks (and solutions) across projects


Sample Risk Tracking Database

(NASA GRC RM Database Tool)

•Developed at NASA Glenn Research Center (GRC) to help capture and track project risks

•Based on experience and the Continuous Risk Management Guidebook

•Contains most of the items found on the Risk Information Sheet (RIS)

•Further information available from website http://smo.gsfc.nasa.gov/riskman/relatedlinks.html

•Database can be downloaded from the Internet if you have Access 97 or greater


Sample Risk Tracking Database

(NASA GRC RM Database Tool)


Sample “Risk Radar” Tracking Database

(http://smo.gsfc.nasa.gov/riskman/relatedlinks.html)


CRM Implementation Tips and Hints

•Start simple; learn to “think risk”

•Never throw out or ignore any risk information; scan it once in a while for applicability

•Always ask for feedback on how things are going and what is working

•Use outside facilitators until you’re comfortable with the CRM process


CRM Implementation Summary

•Adapt Risk Management to your specific project’s complexity and needs

•Document your risk management practice and rationale in a Risk Management Plan

•Your CRM practice will evolve and improve as you gain experience and learn from others’ experiences

•Capture Lessons Learned RiskManagementPlan


Module 10

Course Summary

Identify

Analy

ze

Plan

Track

Control



Review of CRM Short Course Objectives

•Understand the concepts and principles of Continuous Risk Management and how to apply them

•Develop basic risk management skills for each function of Continuous Risk Management

•Become aware of key methods and tools

•Understand how CRM could be tailored to a project

•Be able to differentiate between Risks and Problems


Risk Definitions

•Risk is the combination of the probability (qualitative or quantitative) that a program or project will experience an undesired event (cost overrun, schedule slip, safety mishap, security compromise) and the consequences (impact) of the undesired event, were it to occur.

• NPG 8000.4


Risk Definitions

Risk involves the impact of the event should it occur.

Risk involves the probability that an undesired event will occur.



Risk Exposure = Probability x Impact


Risk Management/Project Management Relationship

Project Management

RiskManagement

Budget

Schedule Performance

People

Quality

ConfigurationManagement

Safety




•A Risk Management Plan (NPG 7120.5B, Section 4.3.2a) describes how the project will perform its tailored risk management process, methods, and tools

• Introduction• Risk Management practice overview• Project organization, roles, and responsibilities• Risk Management practice details• Risk management resources and milestones• Risk information documentation• De-Scope Options

Risk Management Planning


Risk Management Implementation

•Begin risk management effort as early as possible in the Project’s life cycle

•Adapt risk management to your specific project’s complexity and needs

•Establish a baseline set of risks before project start to identify major risks to be avoided

•Document your risk management practice and rationale in a Risk Management plan

•Identify and acquire any needed tools, training, and supporting infrastructure to support the project risk management effort


KSC Risk Management (RM) Resources

•RM consulting is available from KSC Risk Analysis Manager (John Branard, QA-C, 867-2268) • RM Policy/Requirements/Approach/Plans/Tools

• RM Training (Full CRM Course, Risk Baselining Workshop)

• Risk-Based Acquisition Management (R-BAM)

•Project RM/S&MA consulting is available from the SE&T S&MA Project Assessment Office (Ron Gillett, YA-B, 867-9135) • RM approach

• RM Plan development

• RM/S&MA analysis tool selection

• Risk tracking tools

• Project/Mission risk reporting techniques


Final questions?