DAR in the Real WorldWar Stories and Practical Help
Steven Thomas
March, 2008
All Rights Reserved © Alcatel-Lucent 2008, #####2 | DAR in the Real World | March 2008
Outline
• Overview of DAR Process Area and core principles
• When to use DAR – Can, Should and Should Not
• If DAR is so simple, why is it so Hard?
• War Stories - Resistance to DAR
• Fast Consensus-oriented decision process based on wide-band Delphi
• Summary
All Rights Reserved © Alcatel-Lucent 2008, #####3 | DAR in the Real World | March 2008
DAR Overview
• Purpose of Decision Analysis and Resolution (DAR) (from CMMI
V1.2) is to:Analyze possible decisionsUsing a formal evaluation process thatEvaluates identified alternativesAgainst established criteria.
• Like other support process areas (MA, CM, …) Don’t think of DAR as a separate process. Think of it as a procedure which is used in executing other processes.
• DAR is one of the easiest CMMI Process Areas to understand, document and train people to use
• The challenges are determining when to apply this procedure and how to overcome the natural resistance.
All Rights Reserved © Alcatel-Lucent 2008, #####4 | DAR in the Real World | March 2008
DAR – Planning and Doing
• Planning to use a formal evaluation procedure (DAR)During Project planning, identify and plan which decisions or types of decisions need a formal evaluation process and will use the DAR procedure and which do not need this.
Establish guidelines (organizational and/or project) for when unplanned decisions should also use a formal evaluation process
• Using the DAR procedureIdentify and document the “issue” including identifying owner and stakeholders
Identify the selection criteria and the evaluation method (SP1.2, 1.4)
Identify and document the alternative solutions (SP1.3)
Evaluate alternative solutions against the selection criteria (SP1.5)
Select the desired alternative and document the decision (SP1.6)
Communicate the decision to the relevant stakeholders
All Rights Reserved © Alcatel-Lucent 2008, #####5 | DAR in the Real World | March 2008
DAR – Scalability
• Any DAR procedure should be scalable. The effort and rigor invested should make sense for the type of decision. A formal evaluation of alternatives can be done in a few hours or it can take months… “It Depends”, “You Decide”.
Involve only top criteria or look at all criteria
Evaluation can be quantitative or qualitative
Evaluation can be by subjective opinion, consensus or modeling and analysis
Combining different people’s analysis can be done formally, mathematically or informally
Documentation can be a few slides or a full report
All Rights Reserved © Alcatel-Lucent 2008, #####6 | DAR in the Real World | March 2008
When to use DAR – First know WHY
• Starting point: Know WHY you want to use a formal decision / evaluation procedure. Some reasons include:
So the project plans each decision including the schedule and effort
So there is a clear method for how to decide each issue
So decisions are made based on agreed upon business and technical criteria
So Decisions are made once and made correctly
So all decisions and their rationales are recorded
So decisions are communicated to all relevant stakeholders
So decisions are not undermined
All Rights Reserved © Alcatel-Lucent 2008, #####7 | DAR in the Real World | March 2008
When to use DAR – You SHOULD use DAR:
• An organization or project SHOULD use a DAR procedure for major technical decisions and for some tactical organizational decisions. (These are the places in the CMMI V1.2 model which reference DAR.)
Technical Decisions• Architectural Decisions• Design Decisions• Interface Design Decisions• Source Selection Decisions• Tool Selection Decisions • Make/Source/Buy Decisions• Product Integration Sequence Decisions
Organizational Decisions• Process Improvement Proposals• Selecting approaches for training or developing organizational expertise• Selecting work tools for the standard work environment
All Rights Reserved © Alcatel-Lucent 2008, #####8 | DAR in the Real World | March 2008
When to use DAR – You CAN use DAR:
• In addition to the above, an organization or project CANuse a DAR procedure for any decision that involves multiple alternatives which can be evaluated against multiple criteria.
Product Decisions• Product Feature Selection• Product Release Contents and Schedules• Product Roadmap Decisions
Business Decisions• Product Trial Selection• Proposal Generation• Outsourcing Decisions• Project Selection
Project Decisions• Life cycle model selection • Process Tailoring Decisions• Risk Mgmt Mitigation Decisions• Project Management Corrective Action re-planning Decisions• Testing Strategy Decisions• Deployment Strategy Decisions
Process Decisions• Process Design Decisions• Manufacturing Process Design• Work environment or tool change or replacement Decisions
All Rights Reserved © Alcatel-Lucent 2008, #####9 | DAR in the Real World | March 2008
When to use DAR – You SHOULD NOT use DAR:
• There are some decisions for which the organization COULD use DAR (they involve multiple solutions evaluated by criteria) but generally SHOULD NOT use DAR (at least in the conventional way)
General Decisions• Decisions with only 1 viable alternative
• Go/No go Checkpoint Decisions
• Decisions driven by a formula rather than by criteria (e.g. some process tailoring decisions)
Business Decisions• Personnel Decisions (Hiring / Force Reduction, Promotions, Raises, etc.)• Business Strategy Decisions which are based on belief as much or more than analysis• Organizational structure Decisions
All Rights Reserved © Alcatel-Lucent 2008, #####10 | DAR in the Real World | March 2008
WHEN should Your Organization Implement DAR?
• DAR procedures CAN probably benefit any class of decision, but if they will benefit your organization depends on:
The leadership style of the organization
The past organizational culture for making this type of decision
What value does the organization want from a DAR procedure
• Other Decision methods can be just as effective and more efficient
• Select types of decision where the value of DAR is clear
Leadership / Decision Styles• Strong Leader driven• Intuition / Expertise driven• Network or Consensus driven• Data Driven
All Rights Reserved © Alcatel-Lucent 2008, #####11 | DAR in the Real World | March 2008
If DAR is so SIMPLE, then why is it so HARD?
All Rights Reserved © Alcatel-Lucent 2008, #####12 | DAR in the Real World | March 2008
DAR is Threatening
• OrganizationallyHow an organization makes Decisions is at the soul of how the organization operates.
DAR means moving toward a data-driven, engineering analysis culture
• IndividuallyDecision owners are challenged in how they make their decision
Owners must involve relevant stakeholders and consider all criteria
A “Process” questions their ability to make good decisions
• To Peers (In Network or Consensus cultures)It empowers decision owners
• Threats felt at all levels (engineering, project management, organization management, executives)
All Rights Reserved © Alcatel-Lucent 2008, #####13 | DAR in the Real World | March 2008
Overcoming Resistance – Preparing for Resistance• The manager and peers of the decision owner must be
persuaded first
• Understand what the previous decision making model was
• Pick the DAR benefits (slide 6) that will most help this situation
• It is not necessary to move to a data-driven, engineering analysis orientation for all types of decisions
• The predominant decision culture will remain until/unless the executive team changes.
• The resistance scenario involves the decision owner, the type of decision and the evaluation methodology
• Need to plan the consequences of not following the DAR procedure
All Rights Reserved © Alcatel-Lucent 2008, #####14 | DAR in the Real World | March 2008
Resistance Techniques (1)
• We have no major decisions on this project, so no DAR decisions are planned.
• Suggestion: Re-examine the areas where DAR SHOULD be applied and then reconsider the project plan
• The boss (or somebody else) already made the decision, we just have to document it
• Suggestion: Follow the “decision” upstream to make that type of decision correctly next time
• Warning: Beware of the “Hot Potato” game where everyone believes that someone else is really making the decision
• We can’t decide that without checking with the boss
• Suggestion: Clear communication is needed to set expectations and empowerment
All Rights Reserved © Alcatel-Lucent 2008, #####15 | DAR in the Real World | March 2008
Resistance Techniques (2)• Just make the decision, “easier to get forgiveness …”
• Suggestion: Are the value justifications for using DAR correct and well-understood?
• Suggestion: Check if the non-compliance consequences are sufficient
• “There is really only 1 alternative to consider”• Suggestion: Helps to have an independent person check this (Project
Mgr or PPQA)
• “The answer is obvious”• Suggestion: If the task of implementing the decision is transitioned
to a new team, will they think the decision was obvious?
• We discuss pros and cons and reach a consensus decision• Suggestion: Pros and Cons are usually the decision criteria, capture
them, but conduct a formal evaluation
All Rights Reserved © Alcatel-Lucent 2008, #####16 | DAR in the Real World | March 2008
Resistance Techniques (3)• Process is for the engineering staff, not for management
• Suggestion: Examine if the organization is ready to use DAR for these decision types. If so it will be driven from above.
• “We don’t look at multiple alternative, we just confirm that the selected / identified alternative is acceptable”• Suggestion: Streamline the entire decision process.
• “The loudest, last voice wins”• Suggestion: Either use a strong facilitator or use a method which
eliminates complete sharing of information (see example later)
• We can’t document how we decided this… (security, privacy or legal concerns)• Suggestion: A secure repository for these decision types may help
• Suggestion: Re-examine the expected benefits of documenting the decision
All Rights Reserved © Alcatel-Lucent 2008, #####17 | DAR in the Real World | March 2008
Fast Wide-Band Delphi tool for Consensus - Overview• This decision making methodology brings together a group
of people knowledgeable on the decision issue and reaches a consensus decision with minimal time, effort and discussion.
• Preparation – Identify the following as usual:Decision Description
Alternative Solutions
Decision Criteria
• Decision MethodBased on the Wide-Band Delphi estimating methodology
Uses multiple rounds of voting to establish consensus ratings
Limits discussion to only what is necessary to get consensus
A moderator is recommended to manage the votes.
All Rights Reserved © Alcatel-Lucent 2008, #####18 | DAR in the Real World | March 2008
Fast Wide-Band Delphi tool for Consensus - Principles
• Diversity of Opinion – Team should represent multiple functional areas
• Minimal Sharing – The team does a first round of voting without discussion. Later voting rounds use minimal feedback and sharing
• Independence & Anonymity – Each team member votes independently and should not be influenced by how others vote
• Consensus – For each evaluation score, the team must reach consensus. The tool helps evaluate the level of consensus.
• Feedback – If the team does not have consensus, 2 verbal presentations are made for why the score should be higher and lower (1 of each)
All Rights Reserved © Alcatel-Lucent 2008, #####19 | DAR in the Real World | March 2008
Fast Wide-Band Delphi Decision Tool Snapshot
# Criteria Weight Alternative 1 Alternative 2
Alternative 3 - The shoebox
alternative
Consensus Score
1 Height 3 4.0 8.3 7.3 32 Weight 5 5.3 7.0 6.3 33 Cost 10 9.5 5.3 6.3 04 Life Expectancy 6 3.8 8.0 9.0 0
000000
TOTAL Weighted SCORE 156 160 170
Consensus Techniques
• Standard Deviation of scores
• Difference between High/Low scores
• Comparison of Ranking
All Rights Reserved © Alcatel-Lucent 2008, #####20 | DAR in the Real World | March 2008
Summary - Hoped for Take-aways
• Understand why you face opposition in implementing DAR –It is not you, it is a simple process – but it affects the soul of how the organization operates
• Tips for developing a Strategy for which decisions to include in a formal decision process framework
• Game plan for overcoming the natural resistance to using a formal evaluation technique for making decisions
• Techniques for overcoming people’s resistance to following a formal decision process
All Rights Reserved © Alcatel-Lucent 2008, #####21 | DAR in the Real World | March 2008
Contact Info & Biography
• Steve [email protected]
• Mr. Thomas is an authorized Intro to CMMI Instructor and an authorized SCAMPI B/C Team Leader. He has over 25 years experience in software development, project management and quality management. He has a BS in Computer Science from Carnegie-Mellon University and an MS in Computer Science from The Ohio State University.
• Mr. Thomas is the creator and owner of the “DAR” process for a business division of Alcatel-Lucent with over 3,000 R&D staff distributed around the globe