Risk  Management  is  How  Adults  Manage  Projects  March  2008  

 

1   Niwot  Ridge  Consulting,  LLC  

 

Risk  management   is  essential   for   the   success  of   any   significant  project.   1   Information  about   key  project   cost,  performance,  and  schedule  attributes  is  often  unknown  until  the  project  is  underway.  Risks  that  can  be  identified  early   in   the   project   that   impacts   the   project   later   are   often   termed   “known   unknowns.”   These   risks   can   be  mitigated,  reduced,  or  retired  with  a  comprehensive  risk  management  process.  For  risks  that  are  beyond  the  vision  of   the  project   team  a  properly   implemented   risk  management  process  can  be  used   to   rapidly  quantify   the   risks  impact  and  provide  sound  plans  for  mitigating  its  affect.

Risk  management   is  concerned  with  the  outcomes  of  a   future  event,  whose  exact   impacts  are  unknown,  and  with  how  to  deal  with  this  uncertainty.  Outcomes  are  categorized  as  favorable  or  unfavorable.  Risk  management  is  the  art  and  science  of  planning,  assessing,  handling,  and  monitoring  future  events  to  ensure  favorable  outcomes.  A  good   risk   management   process   is   proactive   and   fundamentally   different   than   reactive   issue   management   or  problem  solving.  

This  paper  describes  the  fundamentals  of  Risk  Management  with  5  simple  concepts: 1. Hope  is  not  a  strategy  –  Hoping  that  something  positive  happens  will  not  lead  to  success.  Preparing  for  success  

is  the  basis  of  success.  2. All  single  point  estimates  are  wrong  –  Single  point  estimates  of  cost,  schedule  and  technical  performance  are  

no  better  than  50/50  guesses  in  the  absence  of  knowledge  about  the  variances  of  the  underlying  distribution.    3. Without   integrating   Cost,   Schedule   and   Technical   Performance   you   are   driving   in   the   rearview   mirror.   The  

effort   to   produce   the   product   or   service   and   the   resulting   value   cannot   be   made   without   making   these  connections.  

4. Without   a   model   for   risk   management,   you   are   driving   in   the   dark   with   the   headlights   turned   off   –   Risk  management   is   not   an   ad   hoc   process   that   you   can   make   up   as   you   go.   A   formal   foundation   for   risk  management  is  needed.  Choose  one  that  has  worked  in  high-­‐risk  domains  –  defense,  nuclear  power,  manned  spaceflight.  

5. Risk  Communication  is  everything  –  Identifying  risks  without  communicating  them  is  a  waste  of  time.  

Risk  management  is  an  important  skill  that  can  be  applied  to  a  wide  variety  of  projects.  In  an  era  of  downsizing,  consolidation,   shrinking   budgets,   increasing   technological   sophistication,   and   shorter   development   times,   risk  management  provides  valuable  insights  to  help  key  project  personnel  plan  for  risks.  It  alerts  them  of  potential  risk  issues,  which   can   then  be   analyzed,   and  plans   developed,   implemented,   and  monitored   to   address   risks   before  they  surface  as  issues  and  adversely  affect  project  cost,  performance,  and  schedule.  

Hope  is  Not  a  Strategy  

Hoping  that  the  project  will  proceed  as  planned  is  naïve  at  best  and  poor  management  at  worse.  These  same  naïve   project   managers   constantly   seek   ways   to   eliminate   or   control   risk,   variance   and   uncertainly.   This   is   a  hopeless  pursuit.    

Managing   “in   the   presence”   of   risk,   variance   and   uncertainty   is   the   key   to   success.   Some   projects   have   few  uncertainties  –only  the  complexity  of  tasks  and  relationships  is  important  –  but  most  projects  are  characterized  by  several   types   of   uncertainty.   Although   each   uncertainty   type   is   distinct,   a   single   project   may   encounter   some  combination  of  four  types:  2  

1. Variation  –  comes  from  many  small  influences  and  yields  a  range  of  values  on  a  particular  activity.  Attempting  to  control  these  variances  outside  their  natural  boundaries  is  a  waste  of  time.  

2. Foreseen  Uncertainty  –  are  uncertainties  identifiable  and  understood  influences  that  the  team  cannot  be  sure  will  occur.  There  needs  to  be  a  mitigation  plan  for  these  foreseen  uncertainties.  

3. Unforeseen  Uncertainty  –   is  uncertainty  that  can’t  be   identified  during  project  planning.  When  these  occur,  a  new  plan  is  needed.  

4. Chaos  –  appears  in  the  presence  of  “unknown  unknowns”  

                                                                                                                         1  “Risk  Management  during  Requirements,”  Tom  DeMarco  and  Tim  Lister,  IEEE  Software,  September/October,  2003  2  “Managing  Project  Uncertainty:  From  Variation  to  Chaos,”  Arnoud  De  Meyer,  Christoph  H.  Loch  and  Michael  T.  Pich,  MIT  Sloan  Management  

Review,  Winter  2002  

Risk  Management  is  How  Adults  Manage  Projects  March  2008  

 

2   Niwot  Ridge  Consulting,  LLC  

 

Plans  are  strategies  for  the  successful  completion  of  the  project.  Plans  are  different  than  schedules.  Schedules  show  “how”  the  project  will  be  executed.  Plans  show  “what”  accomplishments  must  be  performed  and  the  success  criteria  for  these  accomplishments  along  the  way  to  completion.  

The  Plan  describes  the  increasing  maturity  of  the  project  through  “maturity  assessment”  points.  The  unit  of  measure  for   this  maturity  must   be  meaningful   to   the   stakeholders.  Something   that   can   be   connected   to   the   investment   they  have  made  in  the  project.  

When  we  speak  the  word  “Hope,”  it  lays  the  foundation  for   failure.   In   the   use   of   Hope  we   really  mean   “success   is  possible   but   not   probable.”   When   we   speak   the   word  “Plan,”  it  does  not  assure  success,  but  success  is  a  probable  outcome.   It   is   the   definition   of   the   probability   of   success  P(s),   that   is   the   foundation   of   the   Plan.   Having   a   Plan–A,   Plan–B   and   possibly   a   Plan–C   exposes   risk,   assigns  mitigations  and  measures  the  probability  of  success.    

The  idea  of  a  Plan  as  a  Strategy  is  critical  to  making  changes  in  the  behavior  of  project  teams  that  can  then  lead  to  “risk  adjusted  project  management.”  Without  a  Plan,  the  schedule  is  just  a  list  of  activities  to  be  performed.  The  reason   for   their   performance  may   be   understood,   but   it   is   unlikely   these   activities   fit   in   any   cohesive   Strategy.  Strategies  have  goals,  critical  success  factors,  and  key  performance  indicators.  No  Single  Point  Estimate  of  Cost,  Schedule  or  Technical  Performance  Can  Correct  

How  long  will  this  take?  How  much  is  it  going  to  cost?  What  is  the  confidence  in  those  two  numbers?  These  are  three  questions  that  must  be  answered  for  the  project   team  to  have  a  credible  discussion  with  the  stakeholders  about  success.  Deciding  what  accuracy  is  needed  to  provide  a  credible  answer  is  a  starting  point.  But  that  does  not  address  the  question  –  “how  can  that  accuracy  be  obtained.”  

There  are  many  check  lists  for  estimating  cost  and  schedule,  with  simple  guidance  on  how  to  build  estimates.  Most  of  this  advice  is  wrong  in  a  fundamental  way.  The  numbers  produced  by  the  estimating  process  do  not  have  their   variance   defined   in   any   statistically   sound   manner.   By   statistically   sound   I   mean   that   the   underlying  probability  distributions  are  known.  If  they  are  unknown,  then  some  form  of  estimating  taking  this  unknown  into  account  must  be  used.    

The  PMI  advice  of  producing  three  estimates  –  optimistic,  most  likely,  pessimistic  is  fraught  with  error.  How  are  these  numbers  arrived  at?  Are   they  based  on  best  engineering   judgment?  Based   in  historical  data?  What   is   the  variance  on  the  variance  of  this  distribution  –  the  2nd  standard  deviation?  

The   use   of   point   estimates   for   duration   and   cost   is   the   first   approach   in   an   organization   low  on   the   project  management   maturity   scale.   Understanding   that   cost   and  durations   are   actually   “random   variables,”   drawn   from   an  underlying  distribution  of  possible  value  is  the  starting  point  for  managing  in  the  presence  of  uncertainty.  

In  probability  theory,  every  random  variable  is  attributed  to  a   probability   distribution.   The   probability   distribution  associated  with  cost  or  duration  describes  the  variance  of  these  random   variables.   A   common   distribution   of   probabilistic  estimates  for  cost  and  schedule  is  the  Triangle  Distribution.    

The   Triangle   Distribution   in   Figure   2   can   be   used   as   a  subjective   description   of   a   population   for  which   there   is   only  limited   sample   data,   and   especially   where   the   relationship  between   variables   is   known   but   data   is   scarce.   It   is   based   on  the   knowledge   of   the   minimum   and   maximum   and   a   “best  guess”  of  the  modal  value  (the  Most  Likely).    

Figure  1  –  The  Plan  for  the  project  must  assure  risk  is  being  reduced  in  proportion  to  the  project’s  tolerance  for  risk    

 

Figure  2  –  triangle  distributions  are  useful  when  there  is  limited  information  about  the  characteristics  of  the  random  variables  are  all  that  is  available.    

Risk  Management  is  How  Adults  Manage  Projects  March  2008  

 

3   Niwot  Ridge  Consulting,  LLC  

 

Using  the  Triangle  Distribution  for  cost  and  duration,  a  Monte  Carlo  simulation  of  the  network  of  activities  and  their  costs  can  be  performed.   In   technical   terms,  Monte  Carlo  methods  numerically   transform  and   integrate  the  posterior  quantitative  risk  assessment   into  a  confidence   interval.  The  result   is  a  “confidence”  model   for  the  cost  and  completion  times  for  the  project  based  on  the  upper  and   lower  bounds  of  each  distribution  assigned  to  the  duration  and  cost.  

Integrating  Cost,  Schedule,  and  Technical  Performance  

In  many  project  management  methods  –  cost,  schedule  and  quality  are   described   as   an   “Iron   Triangle.”   Change   one   and   the   other   two  must   change.   This   is   too   narrow   a   view   of   what's   happening   on   a  project.   It’s   the   Technical   Performance   Measurement   that   replaces  Quality.  Quality  is  one  Technical  Performance  measure.  

Cost   and  Schedule  are  obvious  elements  of   the  project.   Technical  Performance   Measures   (TPM)   describes   the   status   of   technical  achievement  of  the  project  at  any  point  in  time.  The  planned  technical  achievement  is  part  of  the  Performance  Measurement  Baseline  (PMB).  

The  Technical  Performance  Measurement  System  (TPMS)  uses  the  techniques  of  risk  analysis  and  probability  to  provide  project  managers  with  the  early  warnings  needed  to  avoid  unplanned  costs  and  slippage  in   schedules.   Systems   engineering   uses   technical   performance  measurements  to  balance  cost,  schedule,  and  performance  throughout  the  project  life  cycle.    

Connecting   Cost,   Schedule,   and   Technical   Performance   Measures   closes   the   loop   on   how   well   a   project   is  achieving  its  technical  performance  requirements  while  maintaining  its  cost  and  schedule  goals.  IEEE  1220,  EIA  632  and   "A   Guide   to   the   Project   Management   Body   of   Knowledge“all   provide   guidance   for   TPM   planning   and  measurement  and  for  integrating  TPM  with  cost  and  schedule  performance  measures  (Earned  Value).  3  

Technical  performance  measurements  compare  actual  versus  planned  technical  development  and  design.  They  report  the  degree  to  which  system  requirements  are  met  in  terms  of  performance,  cost,  schedule,  and  progress  in  implementing   risk   retirement.   Technical   Performance   Measures   are   traceable   to   user–defined   capabilities.  Integrating  these  three  attributes  produces  a  Performance  Measurement  Baseline  that:  ! Is   a   plan   driven   by   product   quality   requirements   rather   than   a   description   of   the   labor   and   tasks.   The   PMB  focuses  on  technical  maturity  and  quality,  in  addition  to  cost  and  schedule.    

! Focuses  on  progress  toward  meeting  success  criteria  of  technical  reviews.    ! Enables  insightful  variance  analysis.   ! Ensures  a  lean  and  cost–effective  approach  to  project  planning  and  controls. ! Enables  scalable  scope  and  complexity  depending  on  risk.   ! Integrates  risk  management  activities  with  the  performance  measurement  baseline.   ! Integrates  risk  management  outcomes  into  the  Estimate  at  Completion.

The  Cost  and  Schedule  “measures”  are  straightforward  in  most  cases.  The  measures  of  Technical  Performance  involve  measures  Effectiveness  and  Performance.    

Measures  of   Effectiveness   (MOE)   are   the  operational  mission   success   factor   defined  by   the   customer.   These  are:  1. Stated  from  the  customer  point  of  view  2. Focused  on  the  most  critical  mission  performance  needs  3. Independent  of  any  particular  solution  4. Actual  measures  at  the  end  of  development  

                                                                                                                         3  Performance  Based  Earned  Value,  Paul  Solomon  and  Ralph  Young,  John  Wiley  &  Sons,  2006.  

 

Figure   3   –   the   “new”   triangle   must   be   used.  One   where   cost,   schedule,   and   technical  performance  are  interconnected.  

Risk  Management  is  How  Adults  Manage  Projects  March  2008  

 

4   Niwot  Ridge  Consulting,  LLC  

 

Measures  of  Performance  (MOP)  characterize  physical  or  functional  attributes  relating  to  the  system  operation:    5. Supplier’s  point  of  view  6. Measured  under  specified  testing  or  operational  conditions  7. Assesses  delivered  solution  performance  against  critical  system  level  specified  requirements  8. Risk  indicators  that  are  monitored  progressively  

Programmatic  Risk  Must  Follow  a  Well  Defined  Process  

Using  an  ad  hoc   risk  management  process   is   its   self   risky.  The  first  place  to  start  to  look  for  risk  management  processes  is  where  managing   risk   is   mandatory   –   aerospace,   defense,   and   mission  critical   projects   and   projects.   These   also   include   ERP   and  Enterprise  IT  projects.  

Technical  performance  is  a  concept  absent  from  the  traditional  approaches  to  risk  management.  Yet  it  is  the  primary  driver  of  risk  in  many   technology   intensive  projects.  Cost  growth  and  schedule  slippage   often   occur   when   unrealistically   high   levels   of  performance   are   required   and   little   flexibility   is   provided   to  degrade  performance  during   the   course  of   the  project.  Quality   is  often   a   cause   rather   than   an   impact   to   the   project   and   can  generally  be  broken  down  into  Cost,  Performance,  and  Schedule  components.  

The  framework  shown  in  Figure  4  provides  guidance  for:  ! Risk  management  policy  ! Risk  management  structure  ! Risk  Management  Process  Model  ! Organizational  and  behavioral  considerations  for  implementing  risk  management  ! The  performance  dimension  of  consequence  of  occurrence  ! The  performance  dimension  of  Monte  Carlo  simulation  modeling  ! A  structured  approach  for  developing  a  risk  handling  strategy    Risk  Communication  

To  be  effective  the  activities  of  risk  management  must  properly  communicate  risk  to  all  the  participants.  Risk  is  usually  a  term  to  be  avoided  in  normal  business.  Being  in  the  risk  management  business   is  not  desirable  in  most  businesses  –  except  insurance.  It  is  common  to  “avoid”  the  discussion  of  risk.    

Communicating  risk  is  the  first  step  in  managing  risk.  Listing  the  risks  and  making  them  public  is  necessary  but  far  from  sufficient.  Risk  communication  is  the  basis  of  risk  mitigation  and  retirement.  It  serves  no  purpose  to  have  a  risk  management  plan  and  the  defined  mitigations  in  the  absence  of  a  risk  communication.  

The  Risk  Management  Plan  must  address:  ! Executive  summary  –  a  short  summary  of  the  project  and  the  risks  associated  with  the  activities  of  the  project.  Each  risk  needs  an  ordinal  rank,  a  planned  mitigation  if  the  risk  is  active  (a  risk  approved  by  the  Risk  Board),  and  the  mitigations  shown  in  the  schedule  with  associated  costs.  

! Project  description  –  a  detailed  description  of  the  project  and  the  risk  associated  with  each  of  the  deliverables.  This  description  should  be  “operational”  in  nature,  with  the  consequences  description  in  “operational”  terms  as  well.  

! Risk  reduction  activities  by  phase  –  using  some  formal  risk  management  process  that  connects  risk,  mitigation  and  the  IMS.  The  efforts  for  mitigation  need  to  be  in  the  schedule.  

! Risk  management  methodology  –  using   the  DoD  Risk  Management  process   is   a   good   start.   4   This   approach   is  proven  and  approved  by  high  risk,  high  reward  projects.  The  steps  in  the  processes  are  not  optional  and  should  be  executed  for  ALL  risk  processes.  

                                                                                                                         4  Risk  Management  Guide  for  DoD  Acquisition  2003  (Fifth  Edition,  Version  2.0),  www.dau.mil/pubs/gbbks/risk_management.asp    

 

Figure  4  –  this  risk  management  process  is  the  “gold  standard.”  Anything  less  is  inviting  additional  risk.  

Risk  Management  is  How  Adults  Manage  Projects  March  2008  

 

5   Niwot  Ridge  Consulting,  LLC  

 

In   order   to   communicate   risk,   a   clear   and   concise   language   is  needed.   English   is   not   the   best   choice.   Ambiguity   and  interpretation   are   two   issues.   Communicating   in   mathematical  terms   is   also  a  problem,   since   the   symbols   and  units   of  measure  may  be  confusing  and  foreign  to  some  audiences.  

Figure  5   is   from   the  Active  Risk  Manager   5   tool   that   connects  risk  management  with  the  scheduling  system.  ARM  is  a  proprietary  risk  management   system,   but   illustrates   how   risk   is   retired   over  time  in  accordance  with  a  plan.  The  concept  shows  explicitly  when  each   risk   will   be   “bought   down”   or   “retired”   during   the   project  execution.   The  Risk   Registry   and   the   Integrated  Master   Schedule  must  be  connected  in  some  way.  Without  this  connection,  there  is  no  Risk  Management  process  that  can  be  used  to  forecast  impacts  on  cost  or  schedule.  

At   each   project   maturity   point,   current   risks,   the   planned  retirements  of  these  risks,  and  the   impact  of  the  project  must  be  visible  in  the  schedule.  With  these  connections,  project  managers  can  then  answer  the  questions:  ! What  happens  if  this  risk  is  not  mitigated?  ! What  effort  is  needed  to  retire  this  risk  before  a  specific  point  in  time?  ! If   this  risk  becomes  an   issue,  what   is  Plan-­‐B?  How  much  will  Plan-­‐B  cost?  What   is   the   impact  of  Plan-­‐B  on  the  deliverables?  

! What  cost  and  schedule  reserve  is  needed  to  cover  all  the  currently  active  risks?  

Wrap  Up  

Once  cost,   schedule,  and   techncial  performance  are   integrated   into   the  Performance  Measurement  Baseline,  risk  management  can  be  applied  to  all  three  elements.  With  these  connections  in  place,  the  project  management  team  can  say  with  confidence  –  “we  are  doing  risk  management  on  this  project.”  

The  final  reminder  is  to  make  sure  that  all  five  elements  of  risk  management  are  present.  Leaving  one  out  not  only  reduces  the  effectiveness  of  the  risk  management  process,  but   increases  the  risk  to  the  project.  Project  risk  management  is  a  Practice.  The  theory  of  Project  Risk  Management  is  important,  but  the  Practice  is  how  project  risk  gets  managed.  

                                                                                                                         5  www.strategicthought.com    

Figure  5  –  this  risk  retirement  waterfall  shows  where  in  the  plan  risk  will  be  mitigated  or  retired.