Upload
skirossignol
View
221
Download
0
Embed Size (px)
Citation preview
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 1/13
Engineering Management Journal
Engineering Management Journal
EMJ, 2010 – 12(34): 567-890 – Online ISSN: 1234-5678
Risk Quantification for the Design of Boeing 777 Engine Nacelles
Steven Broussarda, Josh Goldschmid
b, William Harkness
b, Damon Slaughter
c
a The Boeing Company ─ St. Louis, MO
P.O.Box 516, St. Louis, MO 63166, [email protected] bThe Boeing Company ─ Everett, WA
9801 27th Ave W, Everett, WA 98208-4316, [email protected], [email protected]
cUnited States Army-- Fort Leonard, MO
1805 Cooley Avenue, Fort Leonard Wood, MO, 65473, [email protected]
Abstract ─ Identifying and mitigating project risks
are crucial steps in managing successful complex
projects. This article proposes the application of a
Risk Mitigation Quantification methodology
quantify, analyze, and mitigate project risks.Popularity and Success Rubric charts were
developed organize, grade, and communicate our
expectation of the product evaluated. The rubrics
essentially help determine how successful the
product will meet the criteria.
The payoff for management and engineerswill be a process that implements historical risk
mitigation steps with known success.The risk mitigation quantization methodology
proposed enables effective planning to mitigate
any consequences from occurring and thus
providing a safe and reliable product that complies
to high-level government regulations as well as
minimizing unnecessary work, reducing overall
program cost, and keeping programs on schedule.
Index Terms ─ Risk Mitigation Quantification,
Project Management, Risk Severity, EngineNacelles, Design, Risk Rubric.
I. INTRODUCTIONIdentifying and mitigating project risks are
crucial steps in managing successful complex
projects. This article proposes the application of a
Risk Mitigation Quantification and Mitigation
methodology quantify, analyze, and mitigateproject risks.
Imagine the risk of developing a new product,
taking a leap of faith into the design world and
hoping the risk pays off in the end. How would
that risk be handled? In the past, one would findthe smartest person in the room and ask them what
they think the risks of the new product would be.
Unfortunately, not one will have a perfect answer
as complex design escapes human abilities.
Collaborative efforts across engineeringdisciplines such as propulsion, structures,materials, project management, aerodynamics, and
safety are necessary actions to identify and
quantify risk and risk mitigation items. There are
strides being made in risk and the end-product will
help remove some of the ambiguity from the risk
world. Grantham Lough, Stone, and Tumer have
developed a process called The Risk in Early
Design (RED)1 which uses historical data to
identify risk as early in the design phase a possible
and assign a likelihood and consequence to the
risk. But, how would one mitigate the risk?Risk Mitigation Quantification is feeding
off a similar idea. At this point a process will not
be developed, however the first step is gathering
the data needed to create a process. Risk
Mitigation techniques are used in every program,so the idea is to gather as many risk mitigation
techniques as possible, document when the
technique was used, how it was used, how often it
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 2/13
Engineering Management Journal
was used. Who assigned the technique? Who is
responsible for the technique? Most important
though, was the technique successful for reducing
risk, and by how much?
Management will benefit from this
method the most. With detailed risk data and
mitigation items will provide a manager thecapacity to make intelligent decisions.
As a secondary outcome of using this
strategy management funding will potentially go
further. Earned Value Management has two key
objectives that need to be addressed.
Encourage contractors to use effective
internal cost and schedule management
control system
Provide the customer timely data
produced by those systems for
determining contract status2.
Earned value is a metric that is used to reportto the customer on progress. Technicalperformance measures (TPM) and the risk are
always key program tracking points that must be
reported to the customer. By using this method,
management can quantify what was done in the
past. Estimations for tracking the risk on the
program and meeting mitigation goals will have
more merit and historical data to back up the
approach.
The first step is to gather the historical
data. The intent of this paper is to take the risk
data and provide a means of showing how mucheach mitigation step has reduced the overall risks,
using the example of the design of a Boeing 777
engine nacelle. The function of the nacelle is to
provide housing for the airplane engine, fuel lines,
and electronic equipment. In simple terms, the
engine nacelle is analogous to the hood of a car.
Cylindrically shaped, the nacelle forms the outer,
aerodynamically smooth covering for the engine
(see figure 1).
The design of the Boeing 777 nacelles is
extremely complex and takes approximately eight
(8) months to complete. The nacelle designcomplexity often surpasses human ability in many
aspects- the calculation, modeling, and simulation
of considerable amount of design and analytical
data provides many opportunities for considerable
consequences. Thus, an understanding of
identifying and quantifying risk and risk
mitigation items is a prerequisite for making
intelligent decisions.
Figure 1: Boeing 777 Engine Nacelle
II. RISK MITIGATION
QUANTIFICATIONRisk quantification is a process of the
independent quantification of probability and
severity associated with a particular product,
service or scenario. The product of the probabilityand severity provides a risk value needed to make
decisions as to what should be done about them.
Risk mitigation quantification, however,
involves the quantification of reduction in the
probability and the reduction in severity. Before
we could quantify risk mitigation, we identified
the risk elements associated with the design of the
Boeing 777 engine nacelles, determined the
likelihood and severity values for each risk, and
determined mitigation strategies.
A. Identification and quantification of riskelements of the 777 nacelle designSeveral tools have been used to identify ten (10)
risky scenarios shown in Figure 2 (larger pictures in
Appendix A):
Failure Modes and Effects Analysis
(FMEA)
Physical inspection of components and
systems
Logic models (event trees and fault trees)
Engineering and statistical tools and analyses havebeen used to estimate the likelihood and
consequences in the event that the risk is
materialized. The tools used for quantifying
probabilities/likelihood were:
Failure analysis
Failure rate data and probability analysis
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 3/13
Engineering Management Journal
Subject Matter Expert (SME) opinion and
subjective probability analysis
Bayesian probability analysis
Human error analysis
For each risk, the relative likelihood and severity of
consequence lead to the calculation of the risk factor product below:
R = P x S (eq. 1)
Where R is the risk score (chance of an identified
consequence occurring), S is severity of harm
(consequence), P is the likelihood of the occurrence
of that harm.
According to Figure 2, our top four biggest
concerns in the design of the nacelles are:
1. Excess weight2. Excess production cost
3. Non-compliance to regulations
4. Safety requirements not met
Consequences are large if these risk items occur.
Fortunately, there are mitigation strategies (Figures
3, 4, and 6) to reduce the probability and severity of
the consequences. The mitigation steps are
allocated to the risk items to support the goal of a
safe nacelle design.
Figure 2: 10 Risk Elements and Risk Score
B. Waterfall chart and risk mitigation of the
777 nacelle design
For each essential safety planning elements,the SMEs were asked to use their experience
and judgement to identify risk mitigation
strategies and to rate what they believe theaverage reduction in probability and severity
of a disaster. Shown in Figure 3 and 4 is a risk
management plan (the full management plancan be seen in the appendix) with job roles
assigned to each mitigation step; the risk
waterfall chart for a specific risk mitigationitem to reduce the likelihood of the
consequences. Activities are introduced to
progressively reduce risk by increasing
knowledge and confidence in the re-design of the 777 nacelles. Assuming each activity is
successful, risk is shown to decrease from
high to low as each activity is completed.Unfortunately, these quantitative estimates of
risk reduction are subjective basing on
SMEs’s estimates; however with the props of the cross-functional team’s engineering andmanufacturing immeasurable experience and
sophisticated skill, the waterfall chart
communicates a well thought out plan with
each risk assigned to a responsible engineer ormanager. If an activity fails, the chart would
be modified to show increasing risk andadditional activities.
Risk Title:777 Nacelle Design Change0
25
50
75
100
H
i
g
h
M
o
d
e
r
a
t
e
L
o
w
Q3Q1 Q2
P e r c e n t L i k e l i h o o d t h a t t h e 7 7 7 N a c e l l e s D o N o t F a i l O v e r t h e L i f e - C y c l e
1
2
3
7
6
5
4
8
9
10
11
12
13
15
16
17
18
19
20
21
Figure 3: Nacelle Risk Management Plan
Figure 4: Risk Identifier and job role assignment for
the waterfall chart (Figure 3)
C. Risk Plot of the 777 nacelle designThis plot below shows the 10 identified risk
values plotted with potential severity of
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 4/13
Engineering Management Journal
consequences on the x-axis and relative likelihood
of consequences on the y-axis.
Figure 5 : Nacelle Risk Plot
D. Assessing the 777 nacelle designBy pulling out the data from the nacelle design
one can begin the process of quantifying the risk
mitigation effort. Figure 6 shows some of the key
points from the risk mitigation effort. The figure
can also be viewed in the appendix.Mitigation Effort.
What does themitigation effortdescribe?
Does it reduce likelihood? If
so, by how much?
Does it reduce consequence? If
so, by how much?
Project proposal for re-design Propose design Yes, 2 No
Integrated, multi-discipline, IntegratedProduct/Process (IPP) team formed
Form team Yes, 2 No
Develop detailed project managementaspects
Develop plan Yes, 2 No
Develop detailed work statement Develop strategy Yes, 4 No
Initial studies and cross-functional researchincluding economics
Conduct research Yes, 4 No
Develop plan for concept design Develop plan Yes, 2 No
Early involvement with leadership andregulatory authority
Assess progress Yes, 10 Yes, 10
Identify applicable regulations andrequirements
Identify limits Yes, 6 Yes, 6
Define plan to meet regulatory and safetyrequirements
Define plan Yes, 6 Yes, 6
Validate certification process Validate process Yes, 4 No
Validate design process Validate process Yes, 6 No
Design the nacelle per the requirements Design product Yes, 2 No
Perform careful materials selection Select material Yes, 4 Yes, 4
Validate build process Validate process Yes, 4 No
Develop prototype for testing Develop prototype Yes, 2 Yes, 2
Supplier management controls in place Implement controls Yes, 6 Yes, 6
Develop conformity and safety test plans Develop Plan Yes, 8 No
Perform conformity and safety tests(flammability, aerodynamics, andappreciable effects of design)
Conduct test Yes, 4 Yes, 4
Identify safety and certification issues-perform failure analyses
Identify concerns Yes, 4 No
Perform technical metrics and audit Conduct checks Yes, 4 No
Conduct production readiness review Conduct review Yes, 4 No
Figure 6: Nacelle Risk Mitigation DataFigure 6 details how each risk mitigation step
affected a particular part of the nacelle design.
The figure also shows the impact to the overall
risk on the redesign. These parts will be important
to determine which mitigation steps are critical to
the success of the risk mitigation effort. For
example one of the mitigation efforts listed is
forming an integrated product process team
(IPPT). Just by forming the team there is a
positive impact on reducing the likelihood a
failure event might occur in the nacelle design,
which is by two (2) percent. The consequence of afailure is not affected by this step. This IPPT is
made up of a core group of engineers and
managers that can make critical and intelligent
decisions on the nacelle design. Early involvement
with leadership and regulatory authorities and
identifying safety and regulatory requirements has
the greatest impact in reducing the likelihood of
the failure consequences as well as reducing the
severity of these consequences. This particular
role goes to regulatory administrators delegated by
the Federal Aviation Administration and the
Boeing Certification Engineers (Figure 4). Other
important risk mitigation strategies include
performing careful materials selection, having
supplier management controls in place, andperforming conformity and safety tests, performed
by Design and Certification Engineers and
Regulatory Administrators.
Figure 7: Risk Mitigation Allocation to ReduceConsequences
Figure 7 shows the allocation of risk mitigation
items from Figure 4 to the risk items to reduce the
severity of the consequences. For instance,
developing a nacelle prototype will ensure design
and safety requirements are met as well as
supporting the understanding of production to
reduce production costs and time.
III. POPULARITY AND SUCCESS
RUBRIC FOR RISK QUANTIFICATION
A. Popularity RubricThe popularity Rubric chart in Figure 8. is
modeled after Dr. Grantham’s work with slightmodifications to meet the generic needs of risk
quantification. See Appendix for clearer picture of
the Rubric Chart. The Rubric organizes, grades,
and communicates our expectation of the product
evaluated.
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 5/13
Engineering Management Journal
Figure 8. Popularity Rubric
With popularity, we used the measurement of
good, moderate, and poor which will be convertednumerically into a 1-3 scale (3 being best, and 1
worst). We can evaluate the success of each
criteria to see whether the risk mitigation met the
requirement outlined by the engineers. It has the
advantage of being a top level review of a productbefore we move into lower level review, which is
covered by the next type of rubric, Success Rubric.
B. Success RubricSuccess Rubric Chart (See Figure 9) is nearly
similar to the Popularity Rubric Chart with the
only exception; it actually evaluates the success of
the product that has gone through the Popularity
Rubric Evaluation initially. Success Rubric helps
determines how successful the product itself was
able to meet the criteria outlined at a much lower
level. It takes into consideration Dr. Grantham’sRED methods as a crucial aspect of the design
phase for risk mitigation. See Appendix for a
clearer picture.
Figure 9. Success Rubric
IV. IMPLICATIONS TO THEENGINEERING MANAGER
When we talk about risk quantification,
Bernstein (1998)3 stated that “without numbers,risk is wholly a matter of gut.” To minimize
arbitrary decisions with absolute subjectivity, we
need to collectively form an integrated team with
sufficient level of skill and experience to identify
and quantify all possible associated risks with a
product design. Risks have a major impact on
business decisions that often reflect on the
business case net present value (NPV) or internalrate of return (IRR).
We make business decisions on the basis of
limited data and likelihood of consequences
occurring and thus, in the face of probabilities,
risk management and quantification plays a crucial
role in decision-making.A corporate objective is to maximize value for
the shareholders and itself. The role of engineering
manager is to provide value to the company,
shareholders, and the customer in the form of a
functional product. The goal of the engineering
manager is to provide a product that is reliable andsafe.
The risk mitigation quantization methodology
proposed enables effective planning to mitigate
any consequences from occurring and thus
providing a safe and reliable product that complies
to high-level government regulations.
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 6/13
Engineering Management Journal
IV. CONCLUSIONThis method of quantifying risk mitigation can
redefine how risk is managed. There are
important strategies that need to be taken away
from this process:
Collect the data from risk mitigation
plan
Use the Success Rubric to ensure all
necessary data is was recorded in the
risk mitigation efforts.
Use the data for future risk mitigation
efforts
Many people will benefit from the risk
mitigation process in this paper. At the
engineering level, historical data will finally be
used instead of subjectively guessing at an answer.
From a managers perspective effective planning
for risk mitigation will save schedule and cost on
the overall program while having the confidence
corners are not being cut when providing a reliable
product to the customer.
When a process is created from these initial
steps it will save time and money for all of
industry who is looking for innovative ways for
risk mitigation.
ACKNOWLEDGMENT
Dr. Grantham’s Rubric Chart.
REFERENCES[1] Gratham Lough, Stone, Tumer (2008) The Risk in
Early Design (RED) Method
[2] NAVSO P-3686 (1998) Top Eleven Ways to
Manage Technical Risk. Office of the Assistant
Secretary of the Navy (RD&A) Acquisition and
Business Management.
[3] Bernstein, P. (1998) Against The Gods – The
Remarkable Storey of Risk. Published by John
Wiley & Sons, Inc. New York, USA
[4] Project Management Institute, A Guide to the
Project Management Body of Knowledge
(PMBOK® Guide), Project Management Institute
(2000).[5] H. Kumamoto, E. Henley (1996) Probabilistic Risk
Assessment and Management for Engineers and
Scientists. Published by IEEE Press.
About the Authors
Josh Goldschmid received his M.S. in
Engineering Management from DrexelUniversity and a B.S. In Biological
Systems Engineering from UC Davis. Heis a very cool guy with 20 years of experience in skiing and welcomeseveryone for free lessons.
Steve Broussard received his B.S. inMechanical Engineering from The
Pennsylvania State University and an A.S.in Engineering Science from SUNYMorrisville. In 8 years I have worked forNAVAIR, Lockheed Martin and currently
work for The Boeing Co. I hate movingand I personally hate to ski and am not acool guy.
William Harkness received his M.S. inEngineering Management from DrexelUniversity and a B.S. In MechanicalEngineering from California State
University, Chico. He is a very laid back guy with 0 years of experience in skiing
and doesn’t understand why anybodywould want to go up, down, up, down,
up, and down a frozen snow covered hill only to break theirleg or collar bone in the process of doing so.
Damon Slaughter received his B.S. in
Engineering Management from Park University. He has never tried snow
skiing but would like to at some pointas long as he does not break his leg orcollar bone in the process.
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 7/13
Engineering Management Journal
APPENDIX A
Figure 2: 10 Risk elements and Risk Score
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 8/13
Engineering Management Journal
Figure 3 : Nacelle Risk Management Plan
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 9/13
Engineering Management Journal
Figure 4: Risk identifier and assigned job roles for the waterfall chart (Figure 3)
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 10/13
Engineering Management Journal
Figure 5 : Nacelle Risk Plot
Figure 6 : Nacelle Risk Mitigation Data
Mitigation Effort.What does themitigation effortdescribe?
Does it reduce likelihood? Ifso, by how much?
Does it reduce consequence? If
so, by how much?
Project proposal for re-design Propose design Yes, 2 No
Integrated, multi-discipline, IntegratedProduct/Process (IPP) team formed
Form team Yes, 2 No
Develop detailed project managementaspects
Develop plan Yes, 2 No
Develop detailed work statement Develop strategy Yes, 4 NoInitial studies and cross-functional researchincluding economics
Conduct research Yes, 4 No
Develop plan for concept design Develop plan Yes, 2 No
Early involvement with leadership andregulatory authority
Assess progress Yes, 10 Yes, 10
Identify applicable regulations andrequirements
Identify limits Yes, 6 Yes, 6
Define plan to meet regulatory and safetyrequirements
Define plan Yes, 6 Yes, 6
Validate certification process Validate process Yes, 4 No
Validate design process Validate process Yes, 6 No
Design the nacelle per the requirements Design product Yes, 2 No
Perform careful materials selection Select material Yes, 4 Yes, 4
Validate build process Validate process Yes, 4 No
Develop prototype for testing Develop prototype Yes, 2 Yes, 2
Supplier management controls in place Implement controls Yes, 6 Yes, 6
Develop conformity and safety test plans Develop Plan Yes, 8 No
Perform conformity and safety tests(flammability, aerodynamics, andappreciable effects of design)
Conduct test Yes, 4 Yes, 4
Identify safety and certification issues-perform failure analyses
Identify concerns Yes, 4 No
Perform technical metrics and audit Conduct checks Yes, 4 No
Conduct production readiness review Conduct review Yes, 4 No
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 11/13
Engineering Management Journal
Figure 7 : Risk Mitigation Allocation to Reduce Consequences
8/3/2019 EMJ SysEng427 Project Final Rev5
http://slidepdf.com/reader/full/emj-syseng427-project-final-rev5 12/13
Engineering Management Journal
Figure 8. Popularity Rubric:
[modeled after Dr. Grantham’s Rubric Charts]