Embed Size (px)
DESCRIPTION
There is a tendency to avoid using measurements in the organization as it is assumed they are not accurate. It is important to understand the difference between counting and measurement. By counting we determine the exact number or value of an indicator (hours worked, billable hours, Facebook fans, defects), while measurement can be defined as a quantitatively expressed reduction of uncertainty, based on one or more observations. So, no matter how fuzzy or inaccurate a measurement is, it is still valuable if it tells us more than what we knew before.
Citation preview
Muntean RamonaMeasurements & Best Practices @ISDC
Implementing a successful metrics system in an IT company
WE CAN MANAGE WHAT WE CAN EASILY MEASURE
IT’S IMPORTANT TO YOU
Better informed decisionsTracking the progress of the objectives Managing the achievements of goals
IT’S REDUCING UNCERTAINTY
TELLS YOU MORE THAN YOU KNEW BEFORE
Key Performance Indicators
Goal
Success Factors
GOALSBusiness growth rateProfit margin Bids success rate Personnel turnover
SUCCES FACTORSProductivityPredictabilityQualityCompetitivenessClient SatisfactionEmployee Satisfaction
TOP DOWN APPROACH – COMPANY GOALS
Metrics / Data
Projects, Processes, People
QUALITY
Cost of Quality (CoQ)
Reviews (1st time) Code Requirements Design Test plans, test scriptsTesting (1st time)Audits
Cost of Appraisal (CoA)
Training Procedures, methods Tools, systems Planning Quality improvement Measurements
Cost of Prevention (CoP)
Fixing defects Re-work code review Re-work doc review Re-reviews Re-testing
Cost of Non-Conformance (CNC)
QUALITY TRENDS
Im-proveKPI’s
ProductivityPredictabilityQualityClient SatisfactionTeam Satisfaction
BOTTOM UP APPROACH – PROJECT’S KPI
Projects, Processes, People
Tooling, Data
Key Performance Indicators
Metrics, Trends
METRICS ANALIZED Velocity: Story Points/sprintEstimation deviation%Re-work, DDR, CoQ trendsDemo / Release success score
TRENDS
S4 S5 S6 S8 S9 S10 S11 S12 S131015202530354045505560
41
22
43 43
2923
55
32
22
Story Points done / sprint
TRENDS
We can learn how review is influencing the product quality in correlation with defects found by testers.
S4 S5 S6 S7 S8 S9 S10 S11 S12 S130.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
Defect Detection Rate (DDR)
DDR_Review DDR_Testing DDR_Customer
DEVELOPING MEASUREMENT PROCESS
Identify scope:- Goals, objectives - “Problems” that need answer- Measurements needs
Collect metrics:- Tailor procedures (if the case)- Record and store data- Use proper tools & templates - Check data accuracy
Analyze metrics:- Aggregate, correlate data- Prepare reports- Discuss trends, identify thresholds- Revise procedures
Evolve process:- Evaluate progress- Evaluate process- Recommend improvements
Define procedures:- Metrics to be collected- Collection practices & tooling- Analysis & reporting- Feedback integration
PLAN DO
CHECKACT
MEASUREMENT PARADOX?
Test drive 1 Test drive 2 Test drive 3 Average
Car A 1 km/l 1 km/l 4 km/l 2 km/l
Car B 1.5 km/l 1.5 km/l 1.5 km/l 1.5 km/l
I want a car that uses as little petrol as possible. I have chosen 2 cars and will do 3 test drives with both of them.
The car that uses less petrol is car A which does more kilometres per litre.
But, if I had measured litres per kilometre, car B uses less litres per kilometre and is the better car.
Test drive 1 Test drive 2 Test drive 3 Average
Car A 1 l/km 1 l/km 0.25 l/km 0.75 l/km
Car B 0.667 l/km 0.667 l/km 0.667 l/km 0.667 l/km
