Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
page 1
03-23-05September 2009
Supporting the CMMI Metrics Framework thru Level 5Supporting the CMMI Metrics Framework thru Level 5
EDSEDS--Electronic Data Systems do Electronic Data Systems do BrasilBrasil LtdaLtda. .
MMáárciorcio SilveiraSilveira
page 2September 2009Supporting the CMMI Metrics Framework thru Level 5
AgendaAgendaAgenda
• Objective
• EDS Overall Process Improvement Strategy
• Measurement Elements of the CMMI Model
• M&A and High-Maturity Practices
• Processes and Procedures for M&A and High-Maturity
• Lessons Learned
• Q&A
page 3September 2009Supporting the CMMI Metrics Framework thru Level 5
ObjectiveObjective
This presentation shows the Software Engineering Framework that EDS utilizes at its Applications Development and Maintenance Centers in Brazil that supports the measurement requirements of the CMMI® model thru level 5. During the presentation you will learn the basics and advanced measurement requirements from the CMMI® model and see how EDS implemented the processes and procedures to support these requirements.
® CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.
page 4September 2009Supporting the CMMI Metrics Framework thru Level 5
EDS Overall Process Improvement StrategyEDS Overall Process Improvement StrategyEDS Overall Process Improvement Strategy
StrategicBusiness
Plan(SBP)
Corporate ObjectivesBusiness Drivers
Technology DriversPI Opportunities
Voice of CustomerVoice of Management
Strategies/InitiativesMeasurement (Monitor & Control)
Inputs/Drivers
OrganizationImprovement
Strategy
Governance Council(Board)
EDGE - Enabling Delivery and Global Excellence
page 5September 2009Supporting the CMMI Metrics Framework thru Level 5
Edge (Work-types, Processes, Activities)EdgeEdge ((WorkWork--typestypes, Processes, , Processes, ActivitiesActivities))
page 6September 2009Supporting the CMMI Metrics Framework thru Level 5
Tailoring the Project ...TailoringTailoring thethe Project ...Project ...
TailoringTool
Requirementsand Approach
Documentation
Project’sDefined Process
File
Starting PointSchedule
OSSP-Org. PlansTailoring Criteria
Project’sDefined Process
File (PDP)
Tailoring Execution
Waivers
Tailoring Decisions
AmendmentHistory
Project WorkbookStructure
page 7September 2009Supporting the CMMI Metrics Framework thru Level 5
M&A e High-Maturity (CMMI-Dev 1.2)M&A e HighM&A e High--Maturity (CMMIMaturity (CMMI--Dev 1.2)Dev 1.2)
PERFORMED (None)Ad Hoc1
• Requirements Development• Technical Solution• Product Integration• Verification• Validation• Integrated Project Management• Risk Mgmt.• Decision Analysis and Resolution• Organizational Process Focus• Organizational Process Definition• Organizational Training
DEFINEDStandard and Consistent
Process3
MANAGED
• Requirements Mgmt.• Project Planning• Project Monitoring and Control• Measurement and Analysis• Process and Product Quality Assurance• Configuration Mgmt.• Supplier Agreement Mgmt.
2 Disciplined Process
LEVEL FOCUS NAME PROCESS AREAS
QUANTIT.MANAGED
• Organizational Process Performance• Quantitative Project ManagementPredictable Process4
• Organizational Innovation & Deployment• Causal Analysis and Resolution
Continuously ImprovingProcess
5 OPTIMIZING
page 8September 2009Supporting the CMMI Metrics Framework thru Level 5
M&A ObjectivesM&A ObjectivesM&A Objectives
• The purpose of Measurement and Analysis is to develop and sustain a measurement capability that is used to support management information needs by:
Specifying the objectives of measurement and analysis such that they are aligned with identified information needs and objectives
Specifying the measures, analysis techniques, and mechanism for data collection, data storage, reporting, and feedback
Implementing the collection, storage, analysis, and reporting ofthe data
Providing objective results that can be used in making informed decisions, and taking appropriate corrective actions
page 9September 2009Supporting the CMMI Metrics Framework thru Level 5
OPP/QPMOPP/QPM
Supporting M&A Goal 1 Supporting M&A Goal 1 Supporting M&A Goal 1
SP 1.2 Specify Measures to Address Measurement Object.SP 1.3 Specify data collection and storage proceduresSP 1.4 Specify analysis procedures
SP 1.2 Specify Measures to Address Measurement Object.SP 1.3 Specify data collection and storage proceduresSP 1.4 Specify analysis procedures
SP 1.1 Establish Measurement ObjectivesSP 1.1 Establish Measurement Objectives
• SPs Goal 1: Align Measurement and Analysis Activities
Performance and Quality
Goals and Objectives
Performance and Quality
Goals and Objectives
OrganizationalMeasurement
Plans
OrganizationalMeasurement
Plans
Strategic
Business Plan
Strategic
Business Plan
Service Level
Agreement
Service Level
Agreement
ProjectMeasurement
Plans
ProjectMeasurement
Plans
Organization
Project
OrganizationalQuantitative Management
Approach
OrganizationalQuantitative Management
Approach ProjectQuantitative
Management Plan
ProjectQuantitative
Management Plan
page 10September 2009Supporting the CMMI Metrics Framework thru Level 5
• SPs Goal 2: Provide Measurement Results
SP 2.1 Collect Measurement DataSP 2.3 Store Data and Results
SP 2.1 Collect Measurement DataSP 2.3 Store Data and Results
Project
Source
of Data
Project
Source
of Data
Project
Measurement
Plans
Project
Measurement
Plans
Presage
(Enterprise
Metrics
Repository)
Presage
(Enterprise
Metrics
Repository)
Project Tracking Reports (Local
Repository)
Project Tracking Reports (Local
Repository)
SP 2.2 Analyze Measurement data SP 2.3 Store Data and Results SP 2.4 Communicate Results
SP 2.2 Analyze Measurement data SP 2.3 Store Data and Results SP 2.4 Communicate Results
Performance and Quality
Goals and Objectives
Performance and Quality
Goals and Objectives
Project Communication
Plan
Project Communication
Plan
PTR Consolidation
Process
OrganizationalMetrics
Analysis Report
OrganizationalMetrics
Analysis Report
Project MetricsAnalysis Report
Project MetricsAnalysis Report
Project
Organizational Metrics
Repository
Organizational Metrics
Repository
Organization
Supporting M&A Goal 2 Supporting M&A Goal 2 Supporting M&A Goal 2
page 11September 2009Supporting the CMMI Metrics Framework thru Level 5
OPP & QPM ObjectivesOPP & QPM ObjectivesOPP & QPM Objectives• The purpose of Organizational
Process Performance (OPP) is to establish and maintain a quantitative understanding of the performance of the organization’s set of standard processes in support of quality and process-performance objectives by:
Selecting processes to be managed quantitatively
Establishing process performance measurements and quality and process performance objectives
Creating and maintaining Process performance baselines and models
• The purpose of Quantitative Project Management (QPM) is to quantitatively manage the project’s defined process to achieve the project’s established quality and process-performance objectives by:
Establishing and maintaining the project’s quality and process-performance objectives
Identifying suitable sub-processes that compose the project’s defined process based on historical stability and capability data found in process-performance baselines or models
Selecting the sub-processes of the project’s defined process to be statistically managed
Monitoring the project to determine whether the project’s objectives for quality and process performance are being satisfied, and identifying appropriate corrective action
Selecting the measures and analytic techniques to be used in statistically managing the selected sub-processes
Establishing and maintaining an understanding of the variation of the selected sub-processes using the selected measures and analytic techniques
Monitoring the performance of the selected sub-processes to determine whether they are capable of satisfying their quality and process-performance objectives, and identifying corrective action
Recording statistical and quality management data in the organization’s measurement repository
page 12September 2009Supporting the CMMI Metrics Framework thru Level 5
• SPs Goal 1: Establish Performance Baselines and Models
Supporting OPP Goal 1 Supporting OPP Goal 1 Supporting OPP Goal 1
SP 1.4 Establish Process-PerformanceBaselines (PPBs)
SP 1.4 Establish Process-PerformanceBaselines (PPBs)
SP 1.5 Establish Process-Performance Models (PPMs)
SP 1.5 Establish Process-Performance Models (PPMs)
SP 1.1 Select ProcessesSP 1.2 Establish Process-Performance MeasuresSP 1.3 Establish Quality and Process-Performance Object.
SP 1.1 Select ProcessesSP 1.2 Establish Process-Performance MeasuresSP 1.3 Establish Quality and Process-Performance Object.
M&A - SP 1.1 Establish Measurement ObjectivesM&A - SP 1.1 Establish Measurement Objectives
Performance and Quality
Goals and Objectives
Performance and Quality
Goals and ObjectivesStrategic
Business Plan
Strategic
Business Plan
Service Level
Agreement
Service Level
Agreement
PPMsPPMsStatistical & Simulation Tools
Statistical & Simulation Tools
Organization
Project
OrganizationalQuantitative Management
Approach
OrganizationalQuantitative Management
Approach
ProjectQuantitative
Management Plan
ProjectQuantitative
Management Plan
Organizational Metrics
Repository
Organizational Metrics
RepositoryPPBsPPBs
page 13September 2009Supporting the CMMI Metrics Framework thru Level 5
• SPs Goal 1: Quantitatively Manage the Project
Supporting QPM Goal 1 Supporting QPM Goal 1 Supporting QPM Goal 1
SP 1.4 Manage Project Performance (more in Goal 2)SP 1.4 Manage Project Performance (more in Goal 2)SP 1.1 Establish the Project’s ObjectivesSP 1.2 Compose the Defined ProcessSP 1.3 Select the Sub-processes that w/be Statis. Managed
SP 1.1 Establish the Project’s ObjectivesSP 1.2 Compose the Defined ProcessSP 1.3 Select the Sub-processes that w/be Statis. Managed
Organization
Project
ProjectQuantitative
Management Plan
ProjectQuantitative
Management Plan
Project Defined Process(PDP)
Project Defined Process(PDP)
Project Tracking Reports (Local
Repository)
Project Tracking Reports (Local
Repository)
Project MetricsAnalysis Report
Project MetricsAnalysis Report
ProjectQuantitative
Management Plan
ProjectQuantitative
Management PlanPerformance and
QualityGoals and Objectives
Performance and Quality
Goals and Objectives
OrganizationalQuantitative Management
Approach
OrganizationalQuantitative Management
Approach
page 14September 2009Supporting the CMMI Metrics Framework thru Level 5
• SPs Goal 2: Statistically Manage Sub-process Performance
Supporting QPM Goal 2 Supporting QPM Goal 2 Supporting QPM Goal 2
SP 2.2 Apply Statistical Methods to Understand VariationSP 2.3 Monitor Performance of the Selected Sub-processesSP 2.4 Record Statistical Management Data
SP 2.2 Apply Statistical Methods to Understand VariationSP 2.3 Monitor Performance of the Selected Sub-processesSP 2.4 Record Statistical Management Data
SP 2.1 Select Measures and Analytic TechniquesSP 2.1 Select Measures and Analytic Techniques
Organization
Project
Project Tracking Reports (Local
Repository)
Project Tracking Reports (Local
Repository)Project MetricsAnalysis Report
Project MetricsAnalysis Report
Project Storyboard
Project Storyboard
ProjectQuantitative
Management Plan
ProjectQuantitative
Management Plan Statistical & Simulation Tools
Statistical & Simulation Tools PPBs & PPMsPPBs & PPMs
page 15September 2009Supporting the CMMI Metrics Framework thru Level 5
Strategic Business Planning – Key elementStrategic Business Planning Strategic Business Planning –– Key elementKey element
Performance and Quality - Goals
and Objectives
Business Objectives
Market Drivers ,
Improvement Opportunities
Customer’s expectations
+
+
=
Strategic Planning
page 16September 2009Supporting the CMMI Metrics Framework thru Level 5
Performance and Quality Goals and ObjectivesPerformance and Quality Goals and ObjectivesPerformance and Quality Goals and Objectives
• Defines the process-performance and quality goals and objectives from the organization. It is a key output of the SBP activities. It defines:
√ Suggested measurements and objectives at organizational level and project level.
√ High-level measurement plan
√ Reporting mechanism
page 17September 2009Supporting the CMMI Metrics Framework thru Level 5
Organizational Measurement Plans (Pattern)Organizational Measurement Plans (Pattern)
• Defines the process to collect, store, analyze and report the measurements collected by the organization and projects. Basically contains:
√ Linkage between the measurements and the performance and quality goals objectives
√ Detailed information about the metrics that will be collected, stored and analyzed.
√ Analysis and reporting procedures
√ Triggers
√ Roles & responsibilities
√ Organizational Metrics Repository information
page 18September 2009Supporting the CMMI Metrics Framework thru Level 5
Project Measurement PlansProject Measurement Plans
• It is an instantiation of the “Organizational Measurement Plans (Pattern)”.
• Projects reuse the pattern and :
√ Justify what doesn´t apply to the project.
√ Add project specific needs
page 19September 2009Supporting the CMMI Metrics Framework thru Level 5
Organizational Quantitative Management ApproachOrganizational Quantitative Management Approach
• It addresses establishment of process-performance and quality objectives for the organization. These objectives will be used by projects to establish project-level process-performance and quality objectives, and determine the measurements that will be used to support these objectives. Basically contains :
√ Quality goals and process-performance objectives
Common Organizational Objectives
Common Project Objectives (Project Type)
√ Organizational Baselines locations (PPBs)
√ Organizational Performance Models (PPMs) Performance
√ Quantitative Management General Procedures
Organizational Procedures (PPBs, PPMs, Tools, etc.)
Project Procedures
page 20September 2009Supporting the CMMI Metrics Framework thru Level 5
Project Quantitative Management PlanProject Quantitative Management Plan
• It is an instantiation of the “Organizational Quantitative Management Approach”.
• Projects follow the organizational approach by :
√ Selecting the common objectives that are aplicable to the project.
√ Identifying project specifics objectives
√ Identifying processes and sub-processes affected by the objectives.
√ Creating a plan to collect, store, analyze and report on quantitative management data. (DMAIC)
√ Identifying Baselines and Models that will be created and/or reused.
√ Documenting all specific procedures that must be followed.
page 21September 2009Supporting the CMMI Metrics Framework thru Level 5
SAP/MSPS/PresageSAP/MSPS/SAP/MSPS/PresagePresage
SAP
MSPS
Tasks Assignments
Effort
Consolidation&
Upload
Presage
page 22September 2009Supporting the CMMI Metrics Framework thru Level 5
Project Tracking Reports (PTR) – Local RepositoryProject Tracking Reports (PTR) Project Tracking Reports (PTR) –– Local RepositoryLocal Repository
page 23September 2009Supporting the CMMI Metrics Framework thru Level 5
Project Tracking Reports (PTR) – Analysis ProceduresProject Tracking Reports (PTR) Project Tracking Reports (PTR) –– Analysis ProceduresAnalysis Procedures
page 24September 2009Supporting the CMMI Metrics Framework thru Level 5
Effort DataChange
Request data Size data …
Consolidation Process
Project Tracking Reports - Project 1
Project Tracking Reports – Project 2
.
Project Tracking Reports – Project N
Project Tracking Reports (PTR) – ConsolidationProject Tracking Reports (PTR) Project Tracking Reports (PTR) –– ConsolidationConsolidation
Organizational Metrics
Repository
Organizational Metrics
Repository
page 25September 2009Supporting the CMMI Metrics Framework thru Level 5
Organizational Metrics Analysis Report Organizational Metrics Analysis Report Organizational Metrics Analysis Report
Avg Review Effort by WPR Completed (hrs)
2.51
4.17
3.182.87
1.250.84
2.47
0.000.501.001.502.002.503.003.504.004.50
Jan
Feb
Mar
Apr
May
Jun
Avg Review Effort per Work Product Review CompletedOrg Avg Review Effort per Work Product Review CompletedLinear (Avg Review Effort per Work Product Review Completed)
Client Satisfaction Survey 2006 Summary
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Within schedule Meetrequirements
Quality Prompt, clearand effective
communications
Opencommunication
Englishproficiency
Changes effectivelymanaged.
Issues/Riskseffectivelymanaged.
Overall projectmanagement.
Appropriate skills Adequate level ofknowledge
Target
n=34
• Some examples :
Productivity:
Hours per FP per platform
KLOC per resource per month per platform
Quality Assurance:
Average number of non-conformances per audit
Average days for non-conformances closing
Defects:
Defect density (#defects / size * 100)
Defect containment (#defeitos detected in peer reviews/adosem revisões/#total defects) by deliverable/phase.
Estimates:
Effort variation by project/Service Request by platform/language
Duration variation by project/Service Request by platform/language
OPP Baselines and Models
Customer Satisfaction Index
page 26September 2009Supporting the CMMI Metrics Framework thru Level 5
Statistical & Simulation Tools – Minitab/Crystal BallStatistical & Simulation Tools Statistical & Simulation Tools –– MinitabMinitab//CrystalCrystal BallBall
7 1 16 4 056 949 842 73 562 8 52 1 41 4 37 21
6 0
4 0
2 0
0
O b s e rv a tio n
Indi
vidu
al V
alue
_X = 2 .1 4
U C L = 7 .6 7
L B = 0
7 1 16 4 056 949 842 73 562 8 52 1 41 4 37 21
6 0
4 0
2 0
0
O b s e rv a tio n
Mov
ing
Ran
ge
_ _M R = 2 .0 8U C L = 6 .7 9
L C L = 0
111
111111111
11
1
1
11111
111
1
11
11
1
1
1
1
1
11
11
11111
1
11111111111111
11
11
111111
1111
11
11
1111
11
11
1
11
11
1
11
1111
T ic k e ts f r o m 2 0 0 7I -M R C h a r t o f E f f o r t
page 27September 2009Supporting the CMMI Metrics Framework thru Level 5
Process Performance Baselines (PPBs) Process Performance Baselines (Process Performance Baselines (PPBsPPBs) )
Name Description Main Characteristics
Design
Defect Density
It shows the expected range of Defect Density (number of Defects by Pages of Documentation * 100) for the External Reviews.
- Design Phase- Health Care- Design Work Products- Minor Enhancement
Testing Productivity
It shows the range of expected Appl.CRproductivity when a Test Case is created.
- Produce Phase- Health Care, Transportation- Test Case- Minor Enhancement
Effort Variation
It shows the range of expected effort variation that would normally be achieved by a Application CR following the minor modification process with similar characteristics.
- Main Phases (Design, Produce and Testing).
- Health Care, Transportation
- Primary Language (Cobol, Cool:Gen, Java, Shell Script)
- Minor Enhancement
Ticket Effort Resolution
It shows the behaviourof the ticket resolutionprocess.
- On-going support
- Ticket category
- Effort ticket resolution
page 28September 2009Supporting the CMMI Metrics Framework thru Level 5
Process Performance Models (PPMs) Process Performance Models (Process Performance Models (PPMsPPMs) )
Planning(Assignment)
Design
Produce
Testing
Non Dev
Ticket
Dev
Ticket
Predictability
80706050403020100
40
30
20
10
0
-10
Code line Qty _avg
Codi
ng
S 6.09834R-Sq 30.6%R-Sq(adj) 24.6%
Regression95% C I95% PI
Fitted Line Plot - Metavance Defect Team - only SEs, all PCVsCoding = 10.86 - 0.4746 Code line Qty _avg
+ 0.007352 Code line Qty _avg**2
Apr/2007 to Apr/2008
Productivity QualityGAD QMSLife Cycle
0.90.80.70.60.50.40.30.20.10.0
4
3
2
1
0
% Reused Test Cases
Test
Cas
e Cr
eati
on P
rodu
ctiv
ity
S 0.514988R-Sq 66.5%R-Sq(adj) 65.4%
Regression95% CI95% PI
Fitted Line PlotTest Case Creation Productivity = 0.4834 + 2.622 % Reused Test Cases
Pilot and After Pilot
3.02.52.01.51.0
150
100
50
0
-50
Work Product Creator Business Skill
Exte
rnal
Def
ect
Den
sity
S 20.7319R-Sq 70.6%R-Sq(adj) 66.2%
Regression95% PI
Fitted Line PlotDefect Density = 848.4 - 1236 Business Skill
+ 589.2 Business Skill**2 - 91.02 Business Skill**3
3.02.52.01.51.0
150
100
50
0
-50
Work Product Creator Business Skill
Exte
rna
l Def
ect
Den
sity
S 20.7319R-Sq 70.6%R-Sq(adj) 66.2%
Regression95% PI
Fitted Line PlotDefect Density = 848.4 - 1236 Business Skill
+ 589.2 Business Skill**2 - 91.02 Business Skill**3
Standard Process
Standard Process
Standard Process
Test Case Creation Reuse
Library Reuse
Knowledge Repository
Training Guideline
Knowledge Base
Business Skill
Business Skill Business Skill
Business Skill
Business Skill
Business Skill
Cheklist Mentoring
Standard Process
Standard Process
Standard Process
Standard Process
page 29September 2009Supporting the CMMI Metrics Framework thru Level 5
Project Storyboard Project Storyboard Project Storyboard
26 / 10 Sep 2008 / EDS INTERNAL APPS Rio – HM Approach Overview
DMAIC Initiatives – Quality
Metavance is the main client of ASFOA Rio. It represents 4 managed projects with 150 resources (about 30% of the Centre).
It was very important to consolidate the perception of this client from ASFOA Rio as a high quality centre.
The Business NeedThe Business Need
•• Increase the quality of the final product;
• Reduce the bottleneck on the application US SME.
Client:
2321191715131197531
100
50
0
-50
Work Product Externally Reviewed
Exte
rnal
Def
ect D
ensi
ty
_X=26.1
UCL=133.9
LCL=-81.8
2321191715131197531
150
100
50
0
Work Product Externally Reviewed
Exte
rnal
Def
ect D
ens
ity
__MR=40.6
UCL=132.5
LCL=0
I-MR Chart of External Defect DensityHealth Care industry, Design phase, Developer role(Period Considered: from April to September 2007)
27 / 10 Sep 2008 / EDS INTERNAL APPS Rio – HM Approach Overview
DMAIC Initiatives – Quality
The analysis of defects raised at ASFOA Rio External Reviews showed that:
• Health Care industry detects 98.5% of defects;
• Design phase is the origin of 68.9% of defects;
• Developer role injects 88.0% of defects.
•>
Root Cause Root Cause AnalysisAnalysis
There was a correlation between External Defect Density and Work Product creator Business Skill.
The Correlation Coefficient (R) is 0.81 and it explains 66.2% of population, what was a good correlation.
The Defect Density increases when the Business Skill is lower than 2.0.
The Main Contributor
3.02.52.01.51.0
150
100
50
0
-50
Work Product Creator Business Skill
Exte
rnal
Def
ect
Den
sity
S 20.7319R-Sq 70.6%R-Sq(adj) 66.2%
Regression95% PI
Fitted Line PlotDefect Density = 848.4 - 1236 Business Skill
+ 589.2 Business Skill**2 - 91.02 Business Skill**3
28 / 10 Sep 2008 / EDS INTERNAL APPS Rio – HM Approach Overview
DMAIC Initiatives – Quality
The SimulationThe SimulationIn order to evaluate the alternatives a simulation was performed, evaluating the impact on External Defect Density and Cost. Cristal Ball, a Monte Carlo simulator, was the tool used.
Option 1 Mean Cost: US$ 2.312 ( +7.8%)Option 1 Mean Defect Density: 35.2 (-36%)
Current Mean Cost: US$ 2143Current Mean Defect Density: 55.6
Option 2 Mean Cost: US$ 2.174 ( +1.4%)Option 2 Mean Defect Density: 54.4 (-2,2%)
Option 3 Mean Cost: US$ 2.184 ( +1.9%)Option 3 Mean Defect Density: 55.6 (0%)
29 / 10 Sep 2008 / EDS INTERNAL APPS Rio – HM Approach Overview
DMAIC Initiatives – Quality
The ImprovementThe ImprovementA model was developed and deployed on Health Care projects. The Project Manager will use it to analyze the benefits of perform an internal review to retain defects.
ProjectAppl. CR ID
PhaseSystem Area
Work Product Design documentPOD 50
WP Creator Business Skill 1.30Internal Reviewer Business Skill 3.70
Average
al Defects without Internal Review
Max (UCL)Average
Min (LCL)
The Project Manager decides to perform an internal review considering the possibility of retain 74% of the forecasted defects.
Analysis Result
Regression EquationDefect Density = 848.4 - 1236 (WP Creator Business Skill) + 589.2 (WP Creator Business Skill)**2 - 91.02 (WP Creator Business Skill)**3
50
Forecasted Total of External Defects without Internal Review
19
Forecasted Total of External Defects with Internal Review
19
Organizational Process Performance ModelDefect Density by Business Skill
ABC1234
Design Application Change Billing
Project Data
Confidence Intervals for Forecasted Total of External Defects
0
2
4
6
8
10
12
14
16
18
20
Forecasted Total of External Defectswithout Internal Review
Forecasted Total of External Defects withInternal Review
Fore
caste
d T
ota
l of
Exte
rnal
Defe
cts
30 / 10 Sep 2008 / EDS INTERNAL APPS Rio – HM Approach Overview
DMAIC Initiatives – Quality
The ResultsThe Results
645750433629221581
100
50
0
-50
-100
Work Product Reviewed
Exte
rnal D
efe
ct D
en
sity
_X=25.9
_X=18.6 _
X=3.2
UC L=129.3
UC L=91.2
UC L=17.4
LC L=-77.4LC L=-54.0
LC L=-11.0
Before Improv ement (Apr to Sep, 2007) P ilot (Sep to Nov , 2007) A fter Improvement (F eb to A pr, 2008)
645750433629221581
120
90
60
30
0
Work Product Reviewed
Exte
rnal
De
fect
Densi
ty
__MR=38.9 __
MR=27.3__MR=5.3
UC L=127.0
UC L=89.2
UC L=17.4
LC L=0 LCL=0 LC L=0
Before Improv ement (A pr to Sep, 2007) P ilot (Sep to Nov , 2007) A fter Improv ement (F eb to A pr, 2008)
11
111
I-MR Chart of External Defect Density
Mean from 18.6 to 3,2 defects by 100PODs
88%
page 30September 2009Supporting the CMMI Metrics Framework thru Level 5
Lessons LearnedLessons LearnedLessons Learned1. M&A/OPP/QPM will force you to create a structured process for Strategic
Business Planning. If you don´t have it will be very difficult to implement M&A/OPP/QPM practices
2. Have a long-term strategy to store measurement data, not only M&A but also SPC data.
3. Have an integrated set of tools supporting your Software Engineering projects, otherwise metrics collection will be a nightmare.
4. Reduce the effort, as much as you can, for metrics collection process, if possible, collect at the same time that the task is being performed.
5. Do specialize people on SPC, Six Sigma otherwise levels 4 and 5 will be a very hard journey. Consider to have statisticians around.
6. Excel will not be sufficient in many cases for QPM/OPP.
7. Do start with few performance indicators and then sophisticate and add additional ones, please remember, they must be aligned to companies´sgoals/objectives. 100 performance indicators will not allow you to take any conclusion.
8. Keep these indicators stable, as much as possible, so that you can analyze them periodically.
9. Do not individualize data at personal level, particularly on QPM/OPP analysis where sub-process analysis may lead you to see detailed data on people.
page 31September 2009Supporting the CMMI Metrics Framework thru Level 5
Q&A ? Q&A ? Q&A ?
page 32
03-23-05September 2009
eds.comeds.com