Engineering Management Capstone Project

EM 697

Compare and C_mtrast

Risk Management Implementation at

NASA and the U.S. Army

L_

Mary Ann Brothers

December 12, 21JO1

https://ntrs.nasa.gov/search.jsp?R=20020048548 2018-08-17T11:21:15+00:00Z

Compare and Contrast

Risk Management Implementation at

NASA and The U.S. Army

Mary Ann Brotlners

EM69712 December 2001

2 of 27

ABSTRACT

NASA at Marshall Space Flight Center (MSFC) andthe U.S. Army at Redstone Av_+enal were analyzed to

determine whether they were successful inimplementing their risk mana_,ement program. Risk

management implementation surveys were distributedto aid in this analysis. The scope is limited to NASA

S&MA at MSFC, includirg applicable supportcontractors, and the U.S. Army Engineering

Directorate, including applicabh_ contractors, located atRedstone Arsenal.

NASA has moderately hi:_her risk managementimplementation survey scoles than the Army.

Accordingly, the implementation of the risk

management program at NA,_,;A is considered goodwhile only two of five of the survey categories

indicated that the risk management implementation isgood at the Army.

INTRODUCTION

The purpose of this project +_ to report tile surveyfindings of the Risk Management Implementation attwo government organizations. A survey developed by

the author, entitled Risk Ma_mgement, was used tosolicit this data. The first ¢_rganization is National

Aeronautics and Space Administration (NASA)Marshall Space Flight Center iMSFC), and the second

is the U.S, Army located at t,tedstone Arsenal. Both

organizations work through matrix support provided tovarious projects and thus each project would dictatespecific needs or requiremenls from the supporting

team. The author will compute and contrast the twoorganizations' implementation cl'lbrts.

ORGANIZATIONS EVALU _TED

This section introduces the two organizations to be

evaluated. It contains a description of the riskmanagement process utilized by both organizations.

Additionally, it defines the risk management categoriesthat will aid in evaluating the effectiveness of the risk

management implementation.

NASA. NASA was established in 1958 and has

accomplished many great scientific and technological

feats in air and space. NASA has also adapted

technology for many uses by the private sector. Thisstudy focuses on a field installation of the NationalAeror.autics and Space Administration, the Marshall

Space Flight Center, located in ttuntsville, AL. MSFCwas e_+tablished in 1960 and named in honor of General

Geor,t,e C. Marshall. General Marshall was the Army

Chief of Staff during World War If, Secretary of State,and Nobel Prize Winner for his world-renowned

"Mar,,.hall Plan." The survey focus at MSFC was theSafety and Mission Assurance (S&MA) team. A

sample of approximately 100 contractors and civilservants are considered to have been involved in risk

management implementation and thus were requested

to participate in providing the survey results.

NASA Risk Management. There are three

requfiements documents for risk management thatNASA considers interdependent:

• NPG 8705.XX (draft), Risk ManagementProcedures and Guidelines

• NPG 7120.5A, NASA Program and Project

Management Processes and Requirements

• NPD 8700.1, NASA Policy for Safety andMission Success

Within Risk Management Procedures andGuid,,lines is the risk management plan and risk lists.

Addilionally, it contains the Program/project manageracts us the integrator of risk management. Ultimately,

it pr,_vides additional information for applying riskmanagement as required by NPG 7120.5A.

The definition for risk management can be foundin NPG 7120.5A: "an organized, systematic decision-making process that efficiently identifies risks, assesses

or analyzes risks, and effectively reduces or eliminates

risks to achieving the program goals." Also found inNPG 7120.5A is the NASA risk management process:

Identify risk issues and concerns

Evaluate (impact/severity, probability,timeframe), classify, and prioritize risks

• Decidewhat,if anything,shouldbedoneaboutrisks

• Monitor risk metric_and verify/validatemitigationactions

• Decideto re-planmitigations,closerisks,invokecontingencypla1_s,orcontinuetotrackrisks

NASA'spolicycanbeli)u:_din theNASA Policy

for Safety and Mission Success. The policy states thatusing qualitative or quantity.live risk assessment

techniques will maximize the likelihood of missionsuccess. Additional evidence of NASA's commitment

and emphasis on risk manag,:ment is in a NASApresentation (Dr. Michael GreeJ_field, 1998) titled Risk

as a Resource. In his presentation, Dr. Greenfieldstates "effective project management depends on a

thorough understanding of the concept of risk, the

principles of risk management, and the establishmentof a disciplined risk management process." Dr.

Greenfield also wrote a paper fc,r NASA that addressesthe need for risk to be managec differently such as the

"knowledge-based" approach that NASA is moving to.NASA also conducts risk management training

classes for civil servants _s well as tbr their

contractors. The risk managem,znt class is presented bythe NASA Safety Training Center. The class

emphasizes that risk management and safety arecorrelated. The class teaches h.,w a risk is an attribute

of a hazard. Additionally, risk is an expression of thecombined severity and probability of loss. NASA uses

the convention for evaluating the severity of a risk for ahazard by working with the worst credible

consequence. When considerirg probability, operatingduration or number of trials/missions/operations isexamined. To assess risk, both must be evaluated. A

useful tool for assessing risl, is a risk assessmentmatrix. A risk assessmen_ matrix includes the

relationship of probability against the severity of the

consequence. Below is a simplified matrix.

Probabit!ty of Mishap

Severity ofConsequences

Cataslrophic

II Cdtical

III Marginal

IV Ne_li[lible

F E O C A

Impossible Improbable Remote Occaslonl Frequent

Some, but not all, of the NASA risk management toolsthat are in place include:

1. Fault Tree Analysis

2. Failure Mode and Effect Analysis

.

4.

5,

EM69712 December 2001

3 of 27

Probabilistic Risk Assessment

Reliability Block DiagramsRisk Assessment Matrix

Methods for establishing risk tolerance limits that areutilized by NASA include:

• Formal analysis

• Professional judgment

• Bootstrapping

Risk management roles and responsibilities are also amajor factor in effective implementation. For NASA,

perfolming risk management analysis is theresponsibility of the line organizations or the staff

specialists. However, the acceptance always falls on

mana),ement.

The U.S. Army. In 1941, congress approved funds forthe A_my to construct a chemical manufacturing and

storage facility, Huntsville Arsenal, to supplement theproduction of the chemical manufacturing plant at

Edgewood Arsenal. A facility, initially known asRedst_me Ordnance Plant, was built adjacent to the

chemical manufacturing installation. The plant was

designated Redstone Arsenal in February 1943.The U.S. Army Aviation and Missile Command

(AMCOM) Aviation & Missile Research,

Development, and Engineering Center (AMRDEC)

Aviation Engineering Directorate located at RedstoneArsenal in Huntsville, Alabama is the focus of this

proje_.t. The Director of Aviation Engineering is theAirworthiness authority for Army developed aircraft

and provides matrix support to their customers.

Aviation Engineering direct customers are the ProgramExecutive Officer Aviation Program/Project/ProductManagers (PMs) and the U.S. Army Aviation and

Missiie Command (AMCOM) Defense SystemsAcquisition PMs. Their ultimate customers are the

Army aircraft crew, passengers, and maintainers thatopera)e the Army aviation systems. The Engineering

Directorate is made up of approximately 660emph,yees. The survey was distributed to about 100contr_ctors and civil servants that were considered to

have applicable knowledge of the risk management

program implementation.

The U.S. Army Risk Management. For the Army,risk i_ a way of measuring the potential that an event

will result in a negative consequence, The Army has arisk management information system website thatcontains many useful tools and techniques utilized by

the Army. Additionally lessons learned as well assafety information can be obtained from this site.

Similar to the NASA philosophy, an Army ProgramManz_ger must consider the probability that an event

will occurandthe consequences should that event

occur when assessing risk. TL> ensure that DOD is

acquiring optimum systems that meet all requirements,Program Managers must manage risk and assess cost,

schedule, and performance. Olce a risk is assessed,Program Managers must determine how best to handle

it. Controlling risk, avoiding risk, assuming risk, andtransferring risk are four strategies used. The four

strategies can be use alone or in combination.Controlling the risk means lowe ing the chance that the

event will occur. Avoiding th,: risk means changing

the source that is subjecting the program to risk.Assuming the risk means Flanning for potential

consequences. Transferring the risk means havingsomeone else take accountability for the risk.

Similar to NASA, tile Army treats riskmanagement as a process for identifying and

controlling hazards to prot,:ct the force. Riskmanagement is a proven accid,:nt-prevention process.

According to BG James E. Sim_nons, director of ArmySafety and commanding genecal of the U.S. Army

Safety Center at Fort Rucker, AI,, accident rates acrossthe Army dropped following the adoption of risk

management as the principle accident-prevention

process. He also states that thc Army's most state-of-the-art safety weapon is ris_. management. Risk

Management is the Army's principle risk-reductionprocess to protect the force. The Chief of Staff states

the Army goal is "to make risk management a routinepart of planning and executin_ operational missions".

Another technique used by the Army is the five-steprisk management process. Acc,_rding to BG Simmons,

effectively applying the five-step risk managementprocess will help do the right training safely and will

also help execute operational missions safely. The

Army's Risk Management Card, which includes thefive-step risk management proc,_'ss, follows.

EM69712 December 2001

4 of 27

Risk Management

Matrix

H 8" '°.The basic principles that provide a framework

for imglementing the risk management process are:

• Integrating risk management into mission

planning, preparation, and execution.

• Making risk decisions at the appropriatelevel in the chain of command.

• Accepting no unnecessary risk.

Risk management integration strengthens risk

management by embedding it in all the Army does,both on and off duty, as organizations and as

individuals. Army risk management integration stepsare:

1, Identify risk management integrationopportunities.

2. Assess improvement opportunities.3. Develop integration procedures.4. Assist implementation of integration

procedures.

5. Measure and reassess the degree ofintegration and its results.

Some, but not all, of the Army risk

management tools that are in place include:

1. Safety Assessment Procedures2. Next Ground Accident Assessment

Individual

3. Leader Training Support Package4. Soldier Training Support Package

for

5. Small Unit Risk Man lgement Booklet

6. Risk Management Cald7. Protection (Safety) Readiness Checklist

from Center for Army Lessons

8. Risk Management W_,_rkSheet

9. CECOM System Stfety Lessons LearnedHandbook

10. Operation Risk Management Leader'sGuide

In addition to risk management tools, below is a

helpful listing of policy and doctrinal references related

to Army risk management.

• AR 70-1, Systems Acquisition Policy andProcedures, dtd 1997.

• AR 385-16, System Safety Engineering and

Management, dtd 3 M;_y 90.

• FM 100-14, Risk Management, dtd 23 April1998.

• HQDA Letter 5-97-1, Risk Management

Integration Responsibilities, dtd 1 May 97.

• MIL-STD-882C, System Safety ProgramRequirements, 19 Jan _)3,

• Center for Army Le_sons Learned (CALL)Newsletter 99-5, "b',_sk Management forBrigades and Battalions", dtd Apr 99.

• FM 101-5 Staff Orgat_zation and Operations,dtd 31 May 1997.

Risk management roles and responsibilities are a

little different for the Army than for NASA.Leadership at the appropriate h:vel of authority making

informed decisions to control hazards or accept risks is

the Army standard for risk management. It is theresponsibility and accountabil_ly of leaders to assess

their operation as a total system and to ensure thatplanning, risk management d:cisions, and executionproactively identifies hazards, assesses the associated

risks, and identifies control measures necessary toreduce the risks to the level commensurate with theircommander's intent.

The level of acceptar,,:e decision authority is

determined by the degree of r_sk. The risk issue mustbe elevated to the next higher command when

resources to control a high risk are not available. Thisprocess promotes that a c_mscious and informeddecision is made to commit the resources to control the

hazards or accept the risk.

EVALUATION.

This section of the report evaluates the results from

each organization individuall,r, Each organization is

EM69712 December 2001

5 of 27

measured against criteria established in the distributed

surveys.

The surveys provide data based on five riskmanagement categories. The risk management

categories, including a demographics section, are: RiskManagement Planning, Risk Identification, Qualitativeand Quantitative Risk Analysis, Risk Response

Planni_lg, and Risk Monitoring and Control. By

answering questions in each of these five categories,ranging from answers of strongly disagreeing to

strongly agreeing, the respondents indicated whether or

not their organization was successful in implementingrisk management. A range of six to eleven questions in

each of the five categories were answered and assigned

a value based on the employee's level of agreementwith five being considered the best score in terms ofsuccess. A score of three or below provided by the

employee indicates a lack of success in this category of

risk management implementation. A one was assignedfor each answer of don't know or not applicable. An

average of the questions was then calculated.

NASA Risk Management Implementation SurveyResults and Evaluation. NASA risk managementimple,nentation surveys were received from eighteen

gover_ment and fifteen support contractors. Of the

thirty-three surveys, results were received from ninemanagers, one support staff, and twenty-three technicalemph,yees. Of those, 39.39% have worked at or

supported NASA over seven years, 27.27% haveworked there between one and three years, 18.18%

have worked there between three and seven years, and

15.15% have been there for less than one year.

'lhe survey results indicated that NASA was most

successful in terms of Risk Management Planning,Risk Identification, and Risk Monitoring and Controlwith mean scores of 3.8. Qualitative And Quantitative

Risk Analysis was next with a mean score of 3.7. Risk

Resp_mse Planning barely ranked as a slightly positivescore with a mean of 3.5.

Although respondents indicated a successful scorefor Qualitative and Quantitative Risk Analysis, aweakness in the risk analysis process was in testing

identified project assumptions against the stability of

the assumption and against the impact on the project ifthe assumption is false.

Three weaknesses were identified in the Risk

Resp,)nse Planning category. One weakness was inchanging the project plan to eliminate the risk and

protecting the project objectives from the risk's impactto avoid specific known risks. The other twoweaknesses are in the risk response plan. The risk

resp_,nse plan does not allow for identification of

residualrisksand/orsecondalyrisksanddoesallowforidentificationofcontr;tctualagreements.

NASA Risk Management

"E

E

<

31,i,i

!!_!_i!!! .I

mt

Risk Management Category

not

EM69712 December 2001

6 of 27

assign personnel or other resources to projects with

different risk rankings, to make a benefit-cost analysisdecisitm about the project, and/or to support a

recommendation for project cancellation. A thirdweakness is that risk analysis is not used to provide a

prioritized list of quantified risks.

R_sk Response Planning received positive

indications, although having an overall unsuccessful

score, in the areas of the Army taking early action to

mitigate risks and developing contingency plans in casethe risk occurs.

For Risk Monitoring and Control, although anoverall unsuccessful score was indicated, a slightly

positive score was achieved for using projectperformance and/or risk reports to monitor and controlrisks.

Army Risk Management lraplementation SurveyResults and Evaluation. The Army risk management

implementation surveys were received from twenty-sixgovernment and five support cc,ntractors. Of the thirty-

one surveys, results were received from five managersand twenty-six technical employees. Of those, 54.84%

have worked at or supported NASA over seven years,22.58% have worked there between one and three

years, 16.13% have been there for less than one year,and 6.45% have worked there between three and seven

years.

The strongest area tbr the Army was RiskIdentification with a mean sco_c of 3.7 and Qualitative

and Quantitative Risk Analy_,is was next followingclose with a mean score of 3.6. The other three

categories indicate weak areas in risk managementimplementation for the Army. Risk ManagementPlanning had a mean score of 3.3 while Risk Response

Planning and Risk Monitoring and Control each had amean score of 3.2.

Although respondents indicated an unsuccessfulscore of 3.3 for Risk Management Planning, responses

to individual questions indiceled that the Army was

strong in that it has a project charter or equivalent andis strong in decision making that influences planning.

Qualitative and Quanttmtive Risk Analysis

received an overall positive score, however,respondents indicated three weaknesses in this area.

The respondents indicated that in the risk analysisprocess, identified project assumptions are not tested

against the stability of the assumption and against theimpact on the project if the assumption is false. A

second weakness is that an overall risk ranking for theproject is not provided by the risk analysis in order to

I::

<

Army Risk Management

4

.............. Iii_iiiilI_iiiiiiiii/i

- - I_ii_iii_i_ii:

272_Risk Management Category

COMPARE AND CONTRAST ORGANIZATIONS

This section will attempt to identify the similarities and

differences in risk management implementationbetween the two organizations.

Both are government organizations workingunde_ a matrix structure that provides their risk

management support. NASA had overall higher meanscores than the Army in each of the five categories.

NASA is considered successful in implementing theirrisk management program with an overall mean scoreof 3.',_7 while the Army is not deemed as having a

successful program with an overall mean score of 3.41.

E

NASA vs Army Risk Management

4

JS

2_

Risk Management Category

Respondents indicated that the strongest risk

management category for both organizations is RiskIdentification. No obvious weaknesses tor either

organization were apparent in this category. RiskManagement Planning ranked second strongest forNASA and ranked third f,_r the Army. Risk

Monitoring and Control ranked third for NASA and

fourth tbr the Army, and Quaiitative and QuantitativeRisk Analysis ranked fourth fet NASA and second for

the Army in terms of succe_,_. Both organizationsranked the weakest in their Risk Response Planning

category.

An area for improvement t,>r both organizations isin the Qualitative and Quantitative Risk Analysis

category. Both are weak in lcsting identified projectassumptions against the stability of the assumption and

against the impact on the project if the assumption isfalse. Additional individual weaknesses, as well as

noted strengths, are listed in the evaluation sections of

this report.

CONCLUSIONS

This section contains the _,ummary of the risk

management implementation _.urvey results for NASAand for the Army.

The implementation of the risk managementsystem at NASA is determined to be good. However,

the following areas indicate a need for improvement:

O Testing identified project assumptions against the

stability of the assumption and against the impacton the project if the assurr, ption is false.

EM69712 December 2001

7 of 27

o Tc_ avoid specific known risks by changing the

project plan to eliminate the risk and/or to protect

the project objectives from its impact.o NASA should develop the risk response plan to

allow for identification of residual risks and/or

secondary risks.

o NASA should develop the risk response plan toallow for identification of contractual agreements.

The implementation of the risk managementsystem at the Army is determined to be poor. Although

survey results in each risk management categoryindicated an overall weakness for the Army, the

follou'ing areas indicatefg'_he _trongest need forimprovement: _Q_.._____------

o q-esting identified project assumptions against the

stability of the assumption and against the impacton the project if the assumption is false.

o Lising risk analysis to provide an overall riskranking for the project to assign personnel or other

re'sources to projects with different risk rankings,tt_ make a benefit-cost analysis decision about the

project, and/or to support a recommendation for

project cancellation.o Using risk analysis to provide a prioritized list of

quantified risks.o Taking early action to mitigate the risk to reduce

the probability and/or impact of a risk to below ant_cceptable threshold.

o I)eveloping a contingency plan in case the riskc,ccurs once it is decided to accept the risk.

_oo

<

<z

L.

=

=

E

r_

<

o°_

N

.<

><

_o

_o

_o

CO

03cd

11 c_

x-. ¢_

.. <(

>., _3-

g _5

_ 5xII C

L7

co 1¢

o-I-a.<

o

UJ

,r- 00 T-- LoT--

'1--"

T'--

"r'-

,r-

v--

'r-

v=-

'r--

'r--

'r--

T--

,r--

v'-

o0

I-- c oc _ C)o E

•-- r" 1=_ o

.N > o(.9 cO

©

o_ cdco

°_

P,

(r_l0.)I

(I)I

C

cl 0JC.-- m

o

o E E,- E.__ _, ._

_ c-t-" c" e".o_ _ o

-i,=.,

0

o

>_ccE

C

C"_

c

o-,j0

x_

Ee--

m_.)

,,,=.

.,.=,

1:::o

C

.9ffl0

c-

C

EI-

E

e,A

L_ 0"_ ¢,D T-- CO

I °

---_ _-- cO l 0i

I

",r--

'r-

'r-

,r-

T'-

('4J

L_

CY

CY

CY

CY

CY

C_

CY

0J

C_

it')

Ec

"o

.7c

..o

r"

E

Eq_

E'_

_0i_-

cy

cy

cy

cy

c_

cy

cy

cy

cy

7_o

_E

o_

.c

6C 0"

q_

o.__

N

O

i

._C

0 m

c __ C

0 --

_m N

-'_ _

o_

edt_ig)

L._

,i

ts_

c

E

o

c

"o

x

-r- .c

t'.-ed

,,-(N

t_

r-

E _o_.c_._-,c

"go__.-- _ _

N_

o_

-_ o

_-_o

N_- _

_ co

.i

L._

,q-

0

9

E

e.._ (

E'_

"_-_1

(..q

ts")

',r--

.i

.r-

,r-

u')

7o

"o

c-

-o'6

o

L_ I._ L_

L_ _

k_ Z'_ II "_t L_ "q"

• l_ II1_ _ "q" ,_"ill _I) _[. II

c -e_l IJJI E_ •

I:=== ml _I

_ c! '--6 N

[,E ,-_._7 / o_

E _'-_1 / .--OE _)

__ _ __'q)l / __ _ _I __"°-_ I - _ _I _I

cn.__c -_I I_'" _- 0 "_i _ O) .._ _"0 _ ___1 I

t"q

o,.I

.T- o,.I

o'3

o4I'-.-o3

i..o

co

oo

o")

o")

o3

o'3

o4

._..ec

t=_

c _ c

._NO E

o •

o

t_

coo¢o0¢o

_,.

LE_

.,_

¢0

O0

L_

O3

O4

C

E

o

o

cc

_-_

e6 O3

0

._-

_._

OD

,,_

C_

.,_

,,_

.,_

4

O4

45o

cl.-_

o£

•e- C

C _

C •_1o o-ol

u')

ed

if)

._- ,_•"=-- I'¢'

o

.-_ _

cN

"-- C

I__ I_

7' ,_ II T"

(/') r't II _1

D tl_

tt ,,_r

t-

O

I.._70

b- '-'-C e)

U')

ts_

,/)

./)

.__ _:_- .9

Lf)

o

o

'E

.m_

X"E

if) C-=- _

I

o

edT--O

U') U'_

CO Or0 (DO

_0

E

c_

(.-q

t.t'3

c_

kt") O')a3t,-.. oO

t_

L_

t._

C'3

CO

C_

C")

Z'_l_ IIT-

r,/]a ll_

a ii _"_

[,/3 ,_ ii it)

r_n,-

.,r-- o

Lt'3

C_

C_

C_

C'3

C_

L_

C'3

L._

c-o'_

•._-, -o Q

_ (3 (/)

Emm

•r-- _ _

_Ecc_c

.-_ o •

.__ E_.m- t.- c

,,r,,,-

,,--- (xl

ed'--

U')

.i

U')

0

,,.,_

_-.__.

.--

_- 0

_ E .

_r-m

,o__m

r- ---_ ..=

o t.--t--- _

0"_ 0 t"q

er_Cr_

• .!-_

_Em'_ E

"_ E._

EgU

ig)

tc)

cN

u') t'_

r_ .,- o

_1-

¢o

co

,4-

v--

v--

v--

t- .• C

•_ .__

.__ 0

._ r--

c

go

<_

u')

O0 T-O

_t

O0

U')

O0

O_

o5

oo

LO

O0

04

o c_ o

c _

> e--

_'Ec _

_-._

o o:e'_ c_._

o cc- •

_ 0 _

eb

_E

m_

• o

c_.-Io o!e'_ c

• t-c- •

Z"_ IIT--

1:1 tt_

n,,

>

0

"';3"

":3"

It')

0

C

C

0 0_E

ET-C_I0000

03

'4"

LO

0'3

0'3

0'3

Lr)

04

LO

.C ":-C "7

C _ _

"._o

O3 O3

._-o

(]3 ._ -7

• - 0") 0L¢)

kt3

03

"4"

It')

Lt')

U'3

04

. ¢-

_0_.>

e_f'4 e-"E Nc30.

;__ o •

":5 c _

=_ _0

090

,r-co

co0o_0co

it)

,_-

03

Lt3

,_-

tt_

co

,_-

,_-

Lt3

co

u_

i

I""- _ t"-

<

L,

c_

=o

<

o

- >

==

°J.

= (/)mc_ IZ:

-< i

0

<

_0

II

(_

(_) GO

I

• C

c9 _

0

a.<

00

uJ

r-

x--

EE

o

o!

_o

r--

r-

r'-

r--

o

C

0

_D

o

co

C0.m

No_t-

O

r-

e

.o_

0

L_

Eo_

C

E

cy_

r-

s--

C

0_---7

_._

o

>o0

E

0.m

C

EI-

0"r-

o

O- oQ.,-

_0 0

r--. _ ['--

c.q o

.c,

c_

.c_CDE

I--

co L_ CO ,,--O CD _,-- r,..o_ o r,o coo.i co .,- c0•,- _ u,_ co

•,- o,.i _ID LC;

U_ I'--

.r-

T-

>_.,-j

C <D_

co

o

i

O_1',- r-..

,r-

>.c

<D >>.

r-. u_

co o

.-_-ICnl

._ul

.'_-I

CD>.,OCLEcD

w.-o

E

CO ",- _..O

.v'-

,,z-

l--

T-

ou_.r-

"13C

Lr_CM CMC C(_.C.i,-,

T--

o

L_

"OC 0

0Lr_ C

c _c

Z

ZZ

.JQ.

)-.Ziii

UJ(.9

Z!

v(/)

E

!

L_ Lt3

E

cq

o')

¢q

r--.

o_

o) ('y

o9

o,.i

coco

¢o

o,.i

o")

o,.i

0o

o_

co

co

o.i

¢'9LO

CO

L.O

0C

,,__. C 0

X:EN

r-E_o.._ oo m cD

m,--N

.___

_oo

,,.-o

o

o

o4

c_

(N

cN

T'--

t.--

__ .-_.__-ot- ..__ e- o "_

_._ __.•r" -_'*" _ 0m. r- .-

._._g _,-_

_D0

_DCO

0

T--

o_

o5

Lt_

09

CO

CO

CO

O9

CO

1.0

Lt_

0":-

_'_

• o

o5

c_u3

cooco

co

u3 u3

o3 03

o3 c_

co co

co c_

co co

co co

co co

co

•_ co

co

co

co "_

u3 t_

co co

u3 u3

o

.o o_

o _,g c-_

2e

O.._

,. mN

,,-O4

CO

0'3

O

0 "" •0_E -o_

_ o _u_ _-

_... _ __ '_- e-

_.,_ o)_ cN_C C_'--

C C _

_EoEo0 "7,__ "_ :_-_ 0 a_ t,O 0-

_'0,_ _ O.

_ _0

E_

o o

N _

N t,-- O

N ._ol

If

_ .-- if)

t_ 09 ocoO.o_

•_ _..__ m

,r-

cooo'_

T-- _r'- 04J

LO

OJ

o9

OJ

¢0

OJ

CO

o_

CO

o3

I.O

2co

c o

o_

Ec

u m

_..__.c_ -_

.__._ _

(N0

_6

(._,<

W

,<

.C

c_

T-O

_0

O_COCO

uDCO

05

_t

_t

co

04

co

co

co

_t

_t

co

,s--

I'_

o'_or-.ed

O3

_0 oo

o

-r- _N

_._

<o

,r"O

p_

CO

C0

N._-__

>,_

_No

.-_ "E

>,

g_

iT: _ "=

,.--o

u').q,._oocou_co

.,t-- T- .T--

u')

co

o,I

co

co

.q,.

co

co

co

.r--

co

co

LO

co

co

co

co

.-- _g.__

o

>,.__

-o._

.__ £

__

¢.D

0

'v- 0_1T--

UD

___.>

•._ .=_ o

"_ _ _

,_gs'm _'5

_cq o

,,.O

t"q

to

eo

to

I.LI

zzz

..Ia.

I,LlC/)Z

0a.c/)I,Llt_

m_a:

T-O

tO

coo4coo03o4

co

o4

to

to

w--

CO

T--

03

03

v--

v--

O3

CO

O9

03

w--

tO

co

oC b

C 4-,

"_ c:

_ _i -

*" 0

o_=: £ E

U3

t,DOCOLO

O3

v-- ._r-- ,r-

0'3

O3

CO

03

,r-

co

co

.._ o _

p,--N5__ c-_l

_o_ 03_

¢,_ °_

I-.- _ E E

coo_o

o5

_. 8 _

_E-_

_&oE

"_ ..,,: .-

_0_

to

o3

o"3

o3

03

co

oD

0o

o')o1",.-

04

o'_

o3

oJ

o3

o3

o3

o3

o3

o3

o3

o,,I

o3

o3

o3

.> nO

c- m

°$ff)'r-

"r- "_c-

O9

O3

O4

C_

O3

0"'_

_'V__

if)

If

e_

E8

I'_od

o 0oco 'l-ea e'i

CN

CO

_Y

T--

T--

co

T"

"r-

0

l--Z0oaZ

z

0l.-

Z0:Ev

L_

s--

L._

(D0_OLO

L_

O_

0'3

CO

(/)

o..-_

-_--

.__ o

o o

c_

o

oe_.._

_ _ ub L_

LO(43003LZ')

¢'3

•,'- 0 "_-

CO

0")

CO

CO

0')

CO

CO

CO

O_

CO

CO

LO

CO

"00_0

O0 O0C '_-

0 __ _

c.o

go

co

c_

,t"-

o,I

cooo_

o4

_oo_coo4co

co

04

o0

co

o,I

co

co

",::1"

oo

oo

co

co

co

,,,¢

oo

o4

i.£)

o,1

co

c

"o

-3"o

o.CD..Q

c,-_,"_ "r"

"E m__.C_

0 .C_ "r-

0

0,1

CO

CO

c...,.__IEc_

c

_ o-_

_'__ _ C

"_- O¢'_ v'

"._ ..O

.N .O

W

W

EM69712 December 2001

27 of 27

References

AR 385-16, "System Safety En_zineering and Management," (May 3, 1990).

AR 70-1, "Systems Acquisition_ Policy and Procedures," (1997).Greenauer George, http://safety.army.mil/home.html.Greenfield, Michael A, Dr., "Rb, k As a Resource," Langley Resear_:h Center, (May 14, 1998).

Greenfield, Michael A, Dr., "Ri_k Balancing Profile Tool," Washington, D.C., (NA)Greenfield, Michael A, Dr., "Ri:_k Management Tools," Langley Research Center, (May 2, 2000).

Headquarters Department of the Army, FM 100-14, "Risk Management," Washington, D.C., (April 23, 1998).

Internal Relations and Communications Office, http://www.msfc.nasa.gov/.MIL-STD-882C, "System Safety Program Requirements," (Januar3 19, 1993).

NASA Safety Training Center, "Risk Management & Fault Tree AJlalysis II."

NPD 8700.1, "NASA Policy tbl Safety and Mission Success," (June 12, 1997).NPG 7120.5A, "NASA Program and Project Management Processes and Requirements," (April 3, 1998).

NPG 8705.x (draft), "Risk Management Procedures and Guidelines"Simmons, James E, "Safety anti Risk Management Integration-M_tre Critical than Ever," Army Aviation, (October

31, 2001), pp. 24-26.Team Redstone, http://www.red_tone.armv.mil/.

United States Army Logistics Management Center, "A Course of Irlstruction In Risk Analysis."United States Army Safety Cen or, http:l/rmis.army.mil/.

2

RISK MANAGEMENT SURVEY

SA = strongly agree; A= agree; D = disagree; SD = strongly disagree, NA = notapplicable

DEMOGRAPHIC

1 Organization TypeGove rnment

Support Contractor

2 Position in the organization

managementtechnical employee (I.e. engineer, designer, scientist)production employeesupport staff (I.e. clerical, human resource)

3 Time in that positionless than 1 year1 to 3 years3 to 7 yearsover seven years

4 Number of employees at yoJr specific siteless than 25between 25 and 150between 150 and 500

greater than 500

RISK MANAGEMENT PLANNING SA A D

½

I,SD NA

My organization has a project charter or equivalent that includes the business needs] and project description at a level appropriate to the needs of the project.

2 Risk management has not been used in my organization.

.:3My organization does not have predefined methods for qualitative risk analysis.

4 My organization does not have predefined methods for quantitative risk analysis.My organization has predefined roles, responsibilities, and authority levels for

5 decision-making that influer_ce planning.

6 Tolerances for risk are expressed in policy statements or revealed in actions.A template for my organization's risk management plan exists and is adaptable to

7 each project by the project manager or the risk management team.

8 The risk management template is improved based on experience from each project.

Meetings are conducted that are designed to adapt the risk management plan9 template to the current project.

10

My organization's risk management plan documents how risk identification,assessment, quantification, response planning, monitoring, and control will bestructured and performed during the pro]ect life cycle.

RISK IDENTIFICATION

]Process outputs are reviewed to identify possible risks.Risk categories are well defined and reflect common sources of risk for the industry

2,Ior application area.

SA A D SD NA

4

3 Historical information on pricr projects is available for review by the project team.

My organization performs structured documentation review(s) of one or more of the

following: project plans and assumptions, prior project files, and other applicable

4 information as an initial step by project teams.

My organization utilizes one or more information gathering techniques in risk

5 identification.

My organization's risk identification process provides adequate indications that a risk

6 has occurred or is about to occur.

,A system is in place at my o'ganization to use identified risks as inputs to other

7 )rocesses.

QUALITATIVE AND QUAN'I_ITATIVE RISK ANALYSIS SA A D SD NA

5

5

6

7

8

9

10

11

1 Risk probability and/or risk impact are risk analysis tools used by my organization.

2 Probability / impact risk rating matrix is a risk analysis tool used by my organization.

In my organization's risk analysis process, identified project assumptions are tested

against the stability of the assumption and against the impact on the project if the

3 assumption is false.

My organization examines tlie extent of the understanding of a risk, the data

available about the risk, the quality and integrity of the data, and the reliability of the

data in order to evaluate the degree to which the data about risks are useful for risk

management.

Risk analysis is used to provide an overall risk ranking for the project which is used:

to assign personnel or other resources to projects with different risk rankings, to

make a benefit-cost analysis decision about the project, and/or to support a

recommendation for project cancellation.

Risks classified as high or moderate would be prime candidates for more analysis,

including quantitative risk analysis, and for risk management action.

My organization utilizes appropriate inputs for quantitative risk analysis

As a part of the risk analysis process, my organization utilizes appropriate tools and

techniques.

Risk analysis is used by my organization to provide a prioritized list of quantified

risks.

Risk analysis is used by my organization to provide a probabilistic analysis of the

Iproject.

,Risk analysis is used by my organization to provide the probability of achieving the

)roject cost and time object yes.

RISK RESPONSE PLANNING

To avoid specific known risks, my organization changes the project plan to eliminate

the risk or condition and/or lo protect the project objectives from its impact.

To reduce the probability ar_J/or impact of a risk to below an acceptable threshold,

my organization takes early action to mitigate the risk.

If my organization decides to accept a risk, a contingency plan may be developed in

case the risk occurs, or the project team may deal with the risk as it occurs.

A risk response plan or equivalent exists and is written to the level of detail at which

the actions will be taken.

The risk response plan (or _:quivalent) allows for identification of residual risks and/or

secondary risks.

SA A D SD NA

6

The risk response plan (or equivalent) allows for identificatior, of contractual6 agreements.

RISK MONITORING AND CONTROL

ISA A D SD

INA

7

] Project performance and/or risk reports are used to monitor and control risks.My organization implements risk identification, assessment, quantification andresponse planning for potential risks that surface as a result of measuring project3erformance.

When required, my organization implements new risk analysis and response plans3 (or equivalent) as a result of scope changes.

My organization utilizes appropriate tools and techniques for risk monitoring and4 control.

Plans are updated as appropriate based on risk monitoring and control, workaround5 _lans, corrective action, project change requests, and/or risk response.

My organization implements and maintains a risk database that is used in the risk6 management process.

My organization updates the risk identification checklists (or _quivalent) as7 appropriate based on risk mc)nitoring and control.

OPTIONAL QUESTIONS

consider the following tools and/or techniques to be greatly effective in my

1 organizations risk management process:

I do not consider the followirg tools and/or techniques to be !jreatly effective in my2 organizations risk managem!.,nt process: