Management Development Institute - ITP, a veteran-owned small business

Introduction to Risk Management in the DOE

Jack JekowskiInnovative Technology Partnerships, LLC

[email protected] 26, 2008

Management Development Institute

Management Development InstituteMarch 26, 2008

2

We Have Customized This Module to Your Needs

• Review of feedback from pilot MDI class in August of 2007

• Review of your backgrounds and interviews• Review of your response to questions on

knowledge and use of risk management terminology and tools

• Review of your specific feedback on what you would like to walk away with from this session


3

Instructional Goals(Terminal Performance Objectives)

Upon successful completion of this session participants will be able to:1. Understand why there is a growing need for risk

management in the DOE2. Converse in the “language” of risk management3. Locate DOE/NNSA, contractor, other federal agency

and other industry recognized risk management tools4. Experiment with basic risk management tools


4

Enabling ObjectivesInstructional Goal 1

Understand why there is a growing need for risk management in the DOE:

– Examine the historical events that have driven this growing need

– Review the current status of risk management policy and guidance in the DOE

– Discuss how the growing need for risk management has impacted the current contracting environment –in particular the development of the Contractor Assurance System (CAS) concept and oversight policy


5

Historical Perspective• The early decades - M&O Contract model

– “no-gain, no-loss”– AT&T - $1 a year – Exceptional Service in the

National Interest– University collaboration – LANL, LLNL, others– 52 M&O contracts by the early 1990’s

• The 80’s and 90’s– Security and Environmental issues – Oak Ridge and

Rocky Flats– End of the Cold War – refocus of national security– 1990 – GAO identification of Contractor Management

as “High Risk Area”– Challenges to M&O System – “At the Crossroads”


6

Historical - 1990• “In 1990 the GAO designated

DOE contract management as a high-risk area because of both inadequate management and oversight of contractors and failure to hold contractors accountable…while DOE is continuing its improvement efforts GAO found that performance problems still regularly occur on DOE’s major projects.”

GAO-07-310High-Risk Series, an Update

January 2007http://www.ig.energy.gov/reports.htm


7

Historical - 1993

• “In the words of John Gibbons in Holding the Edge (1989)…report on maintaining defense technology, the AEC and then DOE “sought independent, outside expertise from organizations unfettered by Federal regulations…The M&O contract system reflects a balance…of the need to maintain flexibility, the need for appropriate management controls, and the need for government oversight.”

“At the CrossroadsThe M&O Contract System”

August 10, 1993


8

Historical Perspective

• Mid to Late 1990’s– Galvin Commission Report– DOE Abolishment Act

• Early 2000’s– Formation of NNSA– New Contract Model– Competing of major M&O contracts– Defense Nuclear Facility Safety Board –

formal risk management questions


9

The Galvin Report: Risk Aversion

• From the Galvin Report* (September 1994):– One of the consequences of the troubles has been

the enhancement of a syndrome common to large bureaucracies: risk aversion…its symptoms are an unwillingness to alter familiar behavior patterns, to stick with unproductive or failing procedures, to enhance tendencies for excessive resource allocation and regulation, and to oppose innovation. It is an important element in sustaining unproductive patterns of work.”

* http://www.seab.energy.gov/sub/galvintsk.html


10

Current Status in DOE

• M&O Contracting today– Contractor Assurance System– Risk management requirements in RFPs– “At-risk” fee models– Office of Enforcement

http://www.hss.doe.gov/Enforce/• However, no overarching policy or guidance is in

place– Draft Policy on “Risk Assessment for Nuclear Safety”

under development in response to Defense Nuclear Facility Safety Board request


11

• “The Board’s review also revealed that DOE does not have mechanisms (such as standards and guides) to control the use of risk management tools nor does it have an internal organization assigned to maintain cognizance and ensure the adequacy and consistency of risk assessments.”

Defense Nuclear Facilities Safety BoardApril 5, 2004

• “…on the use of risk assessment…The Board is concerned…individual program elements and field entities continue to apply various approaches on an ad-hoc basis.

Defense Nuclear Facilities Safety BoardNovember 23, 2005Also, see: http://www.dnfsb.gov/

http://www.hss.energy.gov/deprep /



12

• Current DOE Policy Status– Draft Policy* has been submitted to DNFSB for

review• “Risk Assessment Policy for Nuclear Safety”• Would be applicable to other activities at nuclear

facilities• Joint committee formed among various DOE/NNSA

programs, NRC and NASA representatives• Guidance document to go with policy is now in draft

form (first draft provided to DNFSB in January 2006 –“Risk Management Planning and Execution Guidance”)

*http://www.hss.energy.gov/deprep/archive/chron/2007.aspJanuary 7, 2007 pdf link



13

• Current DOE Policy Status– Response by Defense Nuclear Facility Safety

Board*• “…the policy must contain more clearly defined roles

and responsibilities, as well as mechanisms to ensure quality and consistency in the conduct of risk assessment across the complex…”

• “Certain Department of Energy (DOE) activities may not be well suited to the use of probabilistic approaches.”

*http://www.hss.energy.gov/deprep/archive/chron/2007.aspMay 16, 2007 pdf link



14

– We concluded that the Department should first“establish an up-to-date, unified, risk-based security policy that flows throughout all elements of the Department. It is essential that this policy be applied consistently and that all aspects of security – physical, cyber, and personnel – be integrated to ensure a seamless system.”*

Gregory H. FriedmanDOE I.G.

January 30, 2007House Energy and Commerce Subcommittee on Oversight and Investigations

* http://www.ig.energy.gov/testimony.htm



15

• DOE Oversight Policy and Implementation – DOE P 226.1A (5-25-07) – DOE Oversight

Policy• Establishes a Department-wide oversight process

to protect the public, workers, environment, and national security assets effectively through continuous improvement

• Site assurance systems…will be tailored to…take into account hazards and risks

• DOE line management is responsible and accountable for understanding and accepting the hazards and risks associated with activities http://www.directives.doe.gov/directives/current.html



16

• DOE Oversight Policy and Implementation – DOE O 226.1A (7-31-07)

• Implementation of DOE Oversight Policy• The effectiveness of contractor assurance systems,

the hazards at the site/activity, and the degree of risk are factors in determining the scope and frequency of DOE line management assessments and operational awareness activities.

• Includes Contractor Requirements Document (CRD)– Contractors must ensure that a comprehensive, structured

issues management system is in place, including structured processes for determining the risk, significance, and priority of deficiencies

http://www.directives.doe.gov/directives/current.html



17

• DOE Oversight Policy and Implementation – DOE G 226.1 (12-21-07) – S&S Oversight and

Assessments Implementation Guide• This Guide is intended to identify acceptable

methods for implementing the safeguards and security provisions of DOE O 226.1A

• Issuance indicates that the 226.1 series will be influential in future federal and contractor oversight programs

http://www.directives.doe.gov/directives/current.html



18

• There are many references available by DOE facilities to help with assessing and analyzing risk

Some References in DOE


19

The Evolution of the Contractor Assurance System

• “Model Contract” with Sandia– 1993 with refinements added in 2003

• Contract Model request for input (2003)• Development of CAS with LANL RFP

– Refinement of CAS with LLNL RFP• “Section H” – Special Contract Requirements

– Redefining the federal/contractor relationship to improve management and performance

– Incorporated now into other RFPs • RFI for New Contract Strategy


20

Contractor Assurance System

• Implementation at LANL– “In December 2002, we announced a new approach to

oversight within the National Nuclear Security Administration (NNSA). We concluded that oversight of nuclear operations and of security would continue to be conducted on a transaction basis, but that all other oversight would shift to verifying whether the contractor was operating an adequate internal oversight process, beginning with the lowest risk activities and working up to the more complex ones…using the at-risk fee to further incentivize LANS to excel in implementing this new model.” [Letter from Administrator Linton Brooks]


21

But Congress Has a Different Perspective

• Oversight model for NNSA sites – “The NNSA implemented a new Federal oversight model called Streamlined Oversight as a pilot initiative at the Kansas City Plant and the Los Alamos National Laboratory with the goal of reducing the authority and responsibility of the Federal personnel at the sites because a perception that the heavy hand of federal oversight was causing “excessive risk aversion” in achieving programmatic missions. The Committee notes with interest the NNSA implementation memorandum attributed the concern over “excessive risk aversion” to observations by outside groups.”

FY08 House version of the Energy and Water Bill


22

But Congress Has a Different Perspective (continued)

• Oversight model for NNSA sites –”The Committee is troubled by the federal senior management's decision-process that delegates the management model for an inherently governmental responsibility such as overseeing the contractors running the nuclear weapons complex to a nongovernmental outside group. …the Committee supports a stronger role by the federal program managers in improving safety and security and controlling costs and achieving program objectives.”

FY08 House version of the Energy and Water Bill


23

Contractor Assurance System

• DOE Order 226.1A:– “Perform periodic reviews of contractor

assurance system programs and processes for consistency across the complex and ensure that they reflect industry best practices.”

– “Assurance systems” encompass all aspects of the processes and activities designed to identify deficiencies and opportunities for improvement, report deficiencies to the responsible managers, complete corrective actions, and share in lessons learned effectively across all aspects of operation.”


24

New Contract Strategy RFI

• “The NNSA is planning to develop and implement a contracting strategy for its Management and Operating (M&O) Contracts that will promote more effective and efficient technical and business operations in support of a more responsive and affordable Nuclear Weapons Complex (NWC).”

http://www.doeal.gov/MOContracts/Default.aspx


25

Class Activity

• What are some examples of risk in your mission and contractor environment?

• From your work perspective, what are the reasons you should be managing risk?

• What do you worry most about?– Public perception?– Congressional response?– Mission accomplishment?


26

Example – NAP-18• Some qualitative impact criteria from NNSA Policy Letter

NAP-18:– Substantially impairs the organization’s mission– Could constitute a violation of significant statutory or

regulatory requirements– Substantially weakens safeguards against waste, loss,

unauthorized use, and misappropriation of funds or other assets

– Results in conflict of interest– Creates adverse publicity that affects and organization’s

credibility– Merits the attention of senior DOE management, the

Secretary, Congressional Committees, or the Executive Office of the President

http://www.nnsa.doe.gov/docs/policyletters/NAP-18.pdf


27

Example – NAP-18• Some qualitative impact criteria from NNSA Policy Letter

NAP-18:– Exists in a majority of programs, administrative functions,

and/or organizations and can cause harm, even though minor individually, because the aggregate is significant

– Risks or results in the actual loss of either $10 million or five percent of the resources of a budget line item

– Could reflect adversely on management integrity if not reported

– Endangers national security– Has received significant adverse audit coverage– Significantly impairs the Department’s ability to meet financial

management systems requirements



28

Secretary Chertoff Example• “Let me give you a simple example.

The perfect way to avoid the risk of a car accident is never to leave your house. But very few people pursue this kind of perfect security because we understand that it is self-defeating. We all have to live with a certain amount of risk if we don’t want to become prisoners in our own homes. When we get into our cars, we take reasonable precautions, but we also go about our lives: We go to work; we drive our children to school; we visit friends. We are managing risk.”

How do you reduce the risk of a car

accident?


29


Understand why there is a growing need for risk management in the DOE:

– Examine the historical events that have driven this growing need

– Review the current status of risk management policy and guidance in the DOE

– Discuss how the growing need for risk management has impacted the current contracting environment –in particular the development of the Contractor Assurance System (CAS) concept and oversight policy


30

Break


31


Converse in the “language” of risk management – Cite various published definitions for risk

management and related “terms of art”– Discuss examples of various risk management

methodologies including graphical tools used to assist managers in assessing risk

– Review various terms and concepts that are used in risk management and how different discipline areas in DOE/NNSA have their own unique terminology


32

What is it?• From DOE/EM Risk

Excellence Web Site:– Webster's New World

Dictionary of the American Language (1979, Simon & Schuster, New York, NY) defines risk as the chance of injury, damage, or loss. Therefore, to put oneself "at risk" means to participate either voluntarily or involuntarily in an activity or activities that could lead to injury, damage, or loss.

http://web.ead.anl.gov/whatisrisk/


33

What is it?• Society for Risk Analysis: “Risk analysis

is broadly defined to include risk assessment, risk characterization, risk communication, risk management, and policy relating to risk….”

http://www.sra.org/


34

What is it?

• From the APQC:– Risk - Inherent in any business

venture, risk can never be eradicated. It is an opportunity for financial gain, as well as a hindrance to achieving business goals.

– Risk Management – In some minds, risk management means insurance, but it is a much broader concept. Risks can be categorized as strategic, operational, compliance, or reporting. Risk management is an organization’s strategic response to risk.

http://www.apqc.org


35

What is it?

• From DOE (Software Risk Management Guide SQAS21.01.00-1999):

– Risk is the possibility of loss. It is a function of both the probability of an adverse event occurring and its impact; the impact manifests itself in a combination of financial loss, time delay, and loss of performance. A risk is a precursor to a problem.


36

What is it?

• From the National Infrastructure Protection Program:

– Risk is the expected magnitude of loss (e.g., deaths, injuries, economic damage, loss of public confidence, or government capability) due to a terrorist attack, natural disaster, or other incident, along with the likelihood of such an even occurring and causing that loss.


37

What is it?

• From the CTA-142 Course “Introduction to Risk Management”:

– Risk management — the process of selecting and implementing security countermeasures to achieve an acceptable level of risk at an acceptable cost

R = I * T * V– Risk equals the product of the Impact (such as

cost), the Threat (type and character) and the vulnerability (potential for threat to cause impact)


38

Examples of Risk• Risk: Sometimes it is a

matter of economic impact:– “management

deficiencies by both contractors were a central contributing factor”

– See newly formed HSS Office of Enforcement http://www.hss.energy.gov/Enforce/


39


matter of life and death:– “Terrorist threats force U.S.

Diplomats abroad to stay confined to embassies and compounds.”

– “It’s always a matter of managing risk” – U.S. Diplomatic Service security in 28 nations worldwide is increased because of terrorist threats


40


matter of international politics:– “The decision to destroy the

American satellite does not look harmless as they try to claim, especially at a time when the U.S. has been evading negotiations on the limitation of an arms race in outer space.”

ITAR-Tass news agencyStatement of Russian Defense Ministry


41

Examples of Risk• Risk: Sometimes it’s a

matter of global discontinuities:– “In addition, significant

quantities of weapons-usable HEU and Pu are used in legitimate commercial, medical, and scientific endeavors…Many of these civilian nuclear facilities are lightly guarded and the risk of theft of these materials is significant.”*

* NNSA Office of Global Threat Reduction Strategic Plan, January, 2007http://www.nnsa.doe.gov/na-20/docs/GTRI_Strategic_Plan_2007.pdf


42

Examples of Risk• Adm. Mike McConnell, Director of

National Intelligence: “…the most serious threat is that the plotters that are being observed will be successful in penetrating our defenses and conducting an attack that would result in mass casualties. Their intent is to effect an attack with mass casualties. A secondary attempt would be political or infrastructure targets to even include economic targets that would have long-lasting impact.”*

* http://www.msnbc.msn.com/id/19850951/


43

Class Exercise – Examples of Risk

• What examples of risk do you see in the work that you do or oversee?

• How are those risks currently being mitigated?• What risks do you see outside of your work within

the DOE that you think need to be analyzed and mitigation strategies developed?

• What risks are being mitigated, but don’t really need to be?


44

Break


45

Some Other Common Definitions• Voluntary risks

– Those risks associated with activities that we decide to undertake (e.g., driving a car, riding a motorcycle, smoking cigarettes).

• Involuntary risks– Those risks associated with activities that happen to

us without our prior consent or knowledge. Acts of nature such as being struck by lightning, fires, floods, tornados, etc., and exposure to environmental contaminants are examples of involuntary risks.

• Statistically verifiable risks– Those risks that have been determined from direct

observation. These risks can be compared to each other.


46

Some Definitions

• Statistically nonverifiable risks– Those risks from involuntary activities that are

based on limited data sets and mathematical equations. These risks can also be compared to each other, but no comparison should be made between verifiable and nonverifiable risks.

• The Risk Triplet*– “What can go wrong?”, “How likely is it?” and “What

are the consequences?”.

* From “Risk Management Planning and Execution Guidance” Draft DOE G 421.1-2


47

Some Definitions

• Deterministic Analysis*– Explicitly addresses two questions of the risk triplet

(“What can go wrong?” and “What are the consequences?”) – assumes that the adverse condition will exist.

• Probabilistic Analysis*– Explicitly addresses a broad spectrum of initiating

events and their event frequency. It then analyzes the consequences of those event scenarios and weights the consequences by the frequency, thus giving measure of risk.



48

Some Definitions

• Risk-Based Approach*– One in which decision making is solely based on

the numerical results of a risk assessment.• Risk-Informed Approach*

– Represents a philosophy whereby risk insights are considered together with other factors to establish requirements that better focus attention on design and operational issues commensurate with their importance to public and worker health and safety.



49

What is ERM?

• From the APQC:– ERM – Enterprise Risk Management

enables organizations to identify and manage all significant risks in an integrated way. ERM covers a broad portfolio of risk. Risk assessments are firmly rooted in an understanding of the business, its customers, and management’s strategic objectives.

http://www.apqc.org


50

How Risk is Expressed

• No matter how risks are defined or quantified, they are usually expressed as a probability of adverse effects associated with a particular activity. Risk is usually expressed as a fraction, without units, from 0 - 1.0, where at 1.0 there is absolute certainty that a risk will occur. Scientific notation is generally used to present quantitative risk information.

• However, many use whole numbers 1-5 or 1-10 or 1-100 to quantify relative levels of risk


51

The “Equation”

• The fundamental equation that is used to calculate “risk”:

Risk = Consequences X Likelihood– “Risk” is a number that can also more broadly be

categorized into “low”, “medium” or “high”– “Consequences” is the weight given to the impact if

the risk occurs – it may reflect issues of economic impact, security, and social consequences, including even life and death

– “Likelihood” is the probability that the risk will occur –ranging from highly unlikely to highly likely


52

Translating The “Equation”• For most people, creating a graphic such as this helps

to understand complex environments that have multiple risks as well as to prioritize risks relative to one another

Consequence of Risk

Probability of Risk

(Likelihood)

HIGH

MEDIUM

LOW


53

What is DOE Doing?• New Pilot Program to determine

fiscal and other impacts of Directives (NAP-18)– Risk Heat Map



54

Another Perspective• For most people, there is a very real aversion against catastrophic,

but rare events (e.g. the detonation of a nuclear weapons in a U.S. city) while there is a relatively high tolerance of risks that are less severe, but more likely to occur (e.g. fatal car accidents)

Consequence of Risk

Probability of Risk

Acceptance:• T - Tolerable• M – Marginal• U - Unacceptable

T

T

T

T

T

T

M

M

M

U

U

M

U

U

U

U


55

Another Perspective• Sandia’s Risk Management Process Guidance, RMPG-

001, offers a slightly different graphical interpretation:

See http://www.sandia.gov/E&E/ram.html


56

Another Perspective• Another method of categorization is shown in the DOE Project

Management Training Guide GPG-PM-007. This uses a “Risk Factor” equation:

RF = (P + C) – (P X C)

Consequence of Risk

Probability of Risk High

MediumLow

Thresholds:• Low < 0.3• Medium – 03. to 0.7• High > 0.7 .1 .2 .3 .4 .5 .6 .7 .8 .9

.9

.8

.7

.6

.5

.4

.3

.2

.1


57


Converse in the “language” of risk management – Cite various published definitions for risk

management and related “terms of art”– Discuss examples of various risk management

methodologies including graphical tools used to assist managers in assessing risk

– Review various terms and concepts that are used in risk management and how different discipline areas in DOE/NNSA have their own unique terminology


58


Locate DOE/NNSA, contractor, other federal agency, and industry recognized risk management tools

– Discuss examples of tools that are used – Describe how the federal and contractor

perspective on risk and risk mitigation priorities may differ

– Cite hyperlinked references and web sites in class handouts for later use by students


59

What is INMM* Doing?• Risk Management Workshop

– Held May 30-31, 2007 in Washington, D.C.– Second annual meeting held February 19-20, 2008– Part of an international effort to examine best practices – WINS

(World Institute for Nuclear Security) (http://inmm.org/best_practice/nmrm.cfm)

• “Risk management recognizes that not all variables in nuclear security can be addressed in an absolute manner.”

• Policy should be “risk-informed”• Risk acceptance decisions should be kept to the lowest level possible

– Introduction of Safeguards First Principles Initiative (SFPI)• Risk based model for MC&A program – moving toward “informed

accepted risk”• COMPASS – COMPrehensive Analysis of Safeguards Strategies

– Presentations from across the complex and some international papers * Institute for Nuclear Materials Management - http://www.inmm.org


60

What is INMM* Doing?

• Risk Management Workshop– Presentation by Michael A. Kilpatrick (HSS-1) – Risk

Management Policy within the DOE• Risk Acceptance:

– To determine the appropriate level of protection against risk, line management must consider the threat, the vulnerability of the potential target, and the potential consequences of an adversarial act.

– Risk Management is inherently a management function and always includes acceptance of some level of risk.

– Appropriate risk management decisions can only be made if managers are fully aware of the threat, the effectiveness of protection against those threats, and the costs (both fiscal andoperational) of achieving a given effectiveness level.

* Institute for Nuclear Materials Management - http://www.inmm.org


61


– Presentation by Ken Leifheit (NA-72) – Informed Decision Making in a Nuclear Security Environment (DBT)

* Institute for Nuclear Materials Management - http://www.inmm.org

DOE DBT/ACL

Site Planning and Scenario Development

Site VA Execution

Site Development of DBT Upgrades

VA Steering Committee Scoping

Agreement Visit (Peer Review)

DBT-IP Completed

Verification & Validation of

DBT-IP

Site DBT Implementation

Plan

DOE NNSA Risk Acceptance

Review

Sites builds facility characterization, target identification, mission planning and develops

scenarios

Site visit by NA-72-led team for data validation

and scenario concurrence

Site runs full suite of analysis, using standard tools,

develops results and recommendations

Options for upgrades

Senior Level review -approval for Moderate

or High Risk (if applicable)

Inclusion of funding needs into the

FYNSP

Federal Validation

Funding for DBT-IP

Formal IP with defined Scope, Schedule, Cost

Quarterly Reports

NNSA Approval of DBT

Upgrades


62


– Presentation by Bill Desmond (NA-70) – Transforming Security in the NNSA Weapons Complex

• NNSA used risk management to select the recommended upgrades

– Recently completed cost assessment study by SNL revealed significant cost avoidances tied to technology and physical upgrades for the 2003 DBT

• NNSA effectively pursued non-SPO alternatives in meeting the DBT Policy

• 1,457 additional security officers were not added to the payroll• $212M per year in additional protective force costs not incurred• $1,840M in added security not incurred over the next ten years

– Working with DOE and the field to develop a formal process for risk management

*Institute for Nuclear Materials Management - http://www.inmm.org


63

What is Government Doing?• Secretary Chertoff: “A nation as vital and thriving as ours

cannot become hermetically sealed. Even less can we afford to be overwhelmed by fear or paralyzed by the existence of threats.

That's why we need to adopt a risk-based approach in both our operations and our philosophy. Risk management is fundamental to managing the threat, while retaining our quality of life and living in freedom. Risk management must guide our decision-making as we examine how we can best organize to prevent, respond and recover from an attack. We all live with a certain amount of risk.”*

* George Washington University speech, March 16, 2005 http://www.dhs.gov/xnews/speeches/speech_0245.shtm


64

What is Government Doing?

• Secretary Chertoff: “That means that we tolerate that something bad can happen; we adjust our lives based on probability; and we take reasonable precautions. We must manage risk at the homeland security level…The most effective way, I believe, to apply this risk-based approach is by using the trio of threat, vulnerability and consequence as a general model for assessing risk and deciding on the protective measures we undertake.”*

* George Washington University speech, March 16, 2005 http://www.dhs.gov/xnews/speeches/speech_0245.shtm


65

Risk Management Process - DHS• From DHS National Infrastructure Protection Plan:

http://www.dhs.gov/nippChapter 3 - The Protection Program Strategy: Managing Risk


66

Risk Management Process - NASA• From NASA

Advanced Technology Development Center Risk Management Plan:


67

Risk Management Process - NASA• NASA – System Management Office*: “Risk can be

defined as the probability that a program/project will experience undesirable consequences.”

*http://kscsmo.ksc.nasa.gov/index.htm

Risk management is a daily occurrence within NASA – made more urgent by the Columbia Accident Investigation

http://caib.nasa.gov/


68

Risk Management Process - DOE

• From DOE (Software Risk Management :Guide SQAS21.01.00-1999):


69

Risk Management Process

• From DOE (Risk Analysis and Management Good Practices Guide GPG-FM-007):

– Can be used in combination with other tools, including “Project Risk Categories and Screening Questions”, Attachment 3 (see handouts)


70

Why Manage Risk?

• We plan to create a fully integrated, interdependent weapons complex with several uniform business enhancements. We will manage risk, rather than seek to eliminate it, by applying risk-analytical techniques to programmatic, safety, security, and environmental decisions..."

Tom D'AgostinoApril 5, 2006

House Armed Services Committee,Subcommittee on Strategic Forces


71

Two Differing Perspectives• Contractor view of Risk (DOE Office of

Management, Budget and Evaluation “Risk Management” Rev. E, June 2003, Section 5.4.1):– Contractors treat risk differently from the

Government because each views risk from a different perspective.

– Contractors typically divide risks into two basic types: business risks and project risks. Business risk, in the broadest sense, involves the inherent chance of making a profit or incurring a loss on any given contract. Project risk involves, among other things, technical, requirement, and design uncertainties.

– As a minimum, it is important that the PD writes the request for proposals asking the contractor to describe its risk management process, including its approach to managing any specific areas.


72

Class Exercise - Risk Aversion

• What examples of risk aversion have you seen?

• What were the consequences?• How can risk aversion be avoided?


73

Risk Aversion - Tom D’Agostino

• " By being too risk averse, we hurt productivity at our facilities without improving safety and security. Rather, by implementing methods to better manage risk, including analysis of the costs and benefits of the policies and procedures for ensuring safe and secure operations at our facilities, we will get the job done and do so safely and securely."

Tom D'AgostinoApril 5, 2006

House Armed Services Committee,Subcommittee on Strategic Forces


74

Break


75

An Approach by the Secretary

• Draft Guidance from Secretary*:– Proposes a set of principles to be applied to simplify

and clarify directives, reduce unnecessary burden, and ensure that directives support improved Departmental management and mission accomplishment

• “What vs. How:…it will be sometimes necessary to specify how requirements are met in directives that cover high risk functions such as safety and security..”

• Organizations developing directives will assess the level of risk or particular need for consistency and determine the level of prescription required.”

• “Unauthorized or “rogue” directives often have not had the benefit of being analyzed by the affected parties and risk being ignored or lost over time.”

* July 2007


76

What is Government Doing?

• OMB Draft Bulletin on Risk Management1– Issued January 9, 2006– Sought to “improve the quality of agency risk

assessments”– National Academy of Science (NAS) asked to review

the Draft Bulletin• Coordinated with the Society for Risk Analysis2

• NAS report3 issued on January 11, 2007– “bulletin…is ‘fundamentally flawed’ and should be withdrawn”– Agreed there was room for improvement in federal risk

assessments

3 http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=118112 http://www.ramas.com/omb.htm and http://www.sra.org

1 http://www.whitehouse.gov/omb/inforeg/proposed_risk_assessment_bulletin_010906.pdf


77

Risk Management – Nuclear Regulatory Commission

• NRC – Risk Management Technical Specifications -Since the mid-1980s, the NRC has been reviewing and granting improvements to technical specifications that are based, at least in part, on Probabilistic Risk Assessment (PRA). In August 1995, the NRC adopted a final policy statement on the use of PRA methods in nuclear regulatory activities that encourages greater use of PRA to improve safety decision-making and regulatory efficiency. Since that time, the industry and the NRC have been pursuing increased use of PRA in developing improvements to technical specifications.

http://www.nrc.gov/reactors/operating/licensing/techspecs/risk-management-tech-specifications.html


78

Risk Management – National Research Council

• National Research Council – DOE– In 2005, to enhance DOE’s risk

management efforts, the department asked the National Research Council to prepare a summary of the most effective practices used by leading owner organizations. The study’sprimary objective was to provide DOE project managers with a basic understanding of both the project owner’s risk management role and effective oversight of those risk management activities delegated to contractors.

http://books.nap.edu/catalog.php?record_id=11183

The National Research Council


79

What is DOE Doing?• From 1997-2002 – EM Risk Management

http://www.ead.anl.gov/inetapp/dsp_inetsum.cfm?appsumid=41


80

Quantifying the Risk• Using the formula:

Risk = Consequences X Likelihoodmultiply the two factors together to get a numerical result:

– One quantitative threshold method of categorizing:• High Risk > 0.7• Medium Risk – between 0.3 and 0.7• Low Risk < 0.3

– Often the data available only allows a qualitative evaluation of risk, and it is categorized as high, medium and low

– Remember, document, document, document


81

Determining Consequence• Usually assigned a value between 0 and 1, for

example:– .1 = minimal or no consequence (e.g., no cost or schedule

impact, or no danger of harming anyone)– .5 = some impact may occur (e.g. there may be a cost

overrun, or a person may be injured as a result)– .9 = high consequence (e.g. the success of the project may

be jeopardized, or it is possible that a life may be lost)• Usually, a team of knowledgeable individuals will

discuss and document the reasons for selecting the consequence of risks that are present

– Sometimes scenarios – “what if” stories can help the group in developing consensus on the consequence that might result

– Occasionally additional data will need to be collected to reach a consensus


82

Consequence of Risk• An example from Nevada Risk Management Plan (NNSA/NV-781):


83

Consequence of Risk• An example from Nevada Risk Management Plan (NNSA/NV-781):


84

Determining Likelihood• Usually assigned a value between 0 and 1, for

example, reflecting the probability that the risk will occur:

– .1 = very low probability that the risk may occur (e.g. a lightning strike)

– .5 = moderate probability that the risk may occur (e.g. a car accident)

– .9 = high probability that a risk may occur (e.g. heavy rain may occur during the monsoon season)

• Usually, a team of knowledgeable individuals will discuss and document the reasons for selecting the likelihood of risks occurring

– Likelihood assignments are usually accompanied by more data gathering than consequence – e.g. historical weather records, accident statistics, etc.


85

Likelihood of Risk• An example from Nevada Risk Management Plan (NNSA/NV-781):


86

The “Equation” – Other Variants• Other variants of this equation exist, depending upon the

actual application for which it is intended. Some examples taken from “technical” risk management papers:

Risk = PA x (1 – PE) x CWhere PA is the probability of attack, PE is the probability that the system

will be effective against attack, and C is the consequence of attack

Risk = PA x Ps x C x PMWhere PA is the probability of a terrorist attempt, PS is the probability of the

success of a terrorist attempt, and PM is the probability of mitigating a

successful attempt, and C is the consequence of attack


87

Dealing with Risks• Low Risks

– Apply sound management and control principles– Use a graded approach to oversight

• Medium Risks– Assess other factors that may raise the visibility of the risk

such as public response, Congressional interest, etc.– Follow low risk guidance

• High Risks– Develop a management plan with mitigation strategies– Apply increased oversight to determine if risk is being

addressed and mitigated– Monitor cost and disruption to mission to determine if the

mitigation strategies are appropriate– Document all actions taken and the results


88


Locate DOE/NNSA, contractor, other federal agency, and industry recognized risk management tools

– Discuss examples of tools that are used – Describe how the federal and contractor

perspective on risk and risk mitigation priorities may differ

– Cite hyperlinked references and web sites in class handouts for later use by students


89

Break


90


Experiment with a basic risk management tool– Class exercise

• Hypothetical example taken from today’s “headlines”

• Walk through a standard risk management approach– Risk identification– Risk assessment/analysis– Risk prioritization– Risk mitigation strategy development


91

Class Exercise

• Setting the stage:– It is early 2010, President Obama’s Administration is

now in its second year and the FY11 budget is about to be released. • Dramatic changes are in store for DOE:

– Yucca Mountain funding has been all but eliminated– The Nuclear Weapons budget has been cut by 30% with a

promise of another, similar cut the following year– With help from Senator Udall (D – NM), a new mission for Los

Alamos National Laboratory has been approved and funded:

The International Laboratory for development of the Next Generation nuclear power reactor, one that is proliferation resistant, can be literally “dropped into place”, and minimizes waste through new processing techniques.


92

Class Exercise

• Setting the stage:– This new mission will require the construction of a

model functional reactor in Los Alamos to demonstrate new technologies

– Also included in the new mission will be the need to accommodate an international collaborative team of scientists – approximately 3000

– The Secretary has directed the Site Office to identify risk mitigation strategies that will protect the DOE and the public, while at the same time giving more freedom to the Laboratory to conduct this research in a cost effective manner – eventually transferring the technology to the private sector


93

Class Exercise

• Setting the stage:– The LASO manager has decided to perform a top-level

risk analysis of this new mission, and is looking for your input to help identify the risks, analyze and prioritize them, and develop mitigation strategies that will ensure compliance, but not put undue burden on the researchers, including:• Nuclear operations for the new reactor• Significantly increased foreign interactions with a relaxed

security environment to facilitate collaborative efforts with other nations and the private sector

– You are on the Site Office team that has been gathered to evaluate the risk and oversight necessary for the new mission


94

Class Exercise

• What are the steps that you should pursue with your team?

– Identify the risks that will exist in this new environment• Identify the expertise you will need to add to your team• What regulatory issues will you have to deal with?• What other agencies or organizations might you partner with to

provide expertise and resources?• What industry standards do you think you can invoke?

– Analyze the risks– Prioritize the risks– Develop mitigating strategies to reduce the risk to the

Department


95

Some Helpful Tools

• From “Risk Analysis and Management”Guide (March 2006, GPG-FM-007)

– Use Project Risk Analysis Flow Chart as a guide for the steps necessary


96

Some Helpful Tools

• From “Risk Analysis and Management” Guide (March 2006, GPG-FM-007)

– Use the Screening Check List as a guide to identify risks – what other categories might be applicable for this new mission?


97

Some Helpful Tools

• From “Risk Analysis and Management” Guide (March 2006, GPG-FM-007)

– Also, use the Risk Identification and Analysis form to identify the top 3-5 risks that need to be addressed –what mitigation might be implemented?


98

Class Exercise

• Risk Identification–––––––


99

Class Exercise

• Risk Assessment–––––––


100

Class Exercise

• Risk Calculation and Prioritization–––––––


101

Class Exercise

• Other topics related to risk–––––––


102

Class Exercise

• Mitigation strategies–––––––


103

Class Exercise

• Cost and schedule–––––––


104

Review

Our objectives today were:1. Understand why there is a growing need for risk

management in the DOE– Changing status of M&O contract model– Congressional oversight and increasing costs– Complex transformation initiatives

2. Converse in the “language” of risk management– Basic risk equation – probability of occurrence and

impact– Probabilistic vs. Deterministic– Risk-based vs. Risk-informed decision making– Voluntary risks vs. Involuntary risks


105

Review

Our objectives today were:3. Locate DOE, contractor, other federal agency

and other industry recognized risk management tools

– Many different references available– You need to identify which ones pertain to your

discipline area – references on CD-ROM are a good starting point

4. Experiment with basic risk management tools– A new era in contractor oversight– Monitor the evolution of the LANL and LLNL contracts– New contract models will require increased use of risk

management tools


106

Summary

• Risk is around us every day• Whether we accept a risk, or work to reduce it,

it is a judgment call – the more data you have and the more formal your process, the more confident you can be in your decision

• DOE/NNSA mission will require more formal processes for identifying, analyzing, managing risk and documenting risk decisions

• We can expect to see more attention paid to the “process” and the development of policy


107

Your Challenge

When you go back to your work location, develop a list of those activities or projects

that have risk, identify where that risk is high and begin using the tools you have learned in

this class to manage the risk

Good luck!

Documents

Management Development Institute - ITP, a veteran-owned small business