Software Defect Prediction & Release Readiness Assessment
Rakesh Rana
Rakesh Rana
PhD in Software Engineering, 2015
Double Masters from Handelshögskolan i Göteborg
Tekn. Lic in 2013
Industrial collaborations with Volvo Car Group, Ericsson, Saab
> 15 publications
About Myself
Number of defects / trouble reports (TRs)
Number of defects / trouble reports (TRs)
Have you ever wondered?
• What is the trend of defect inflow, what does it indicate?
• Have we found most defects already or will we find them late?
• How many defects are we likely to find in this project/release?
• Are we ready to release? or do we need to test more?
Data used
Early or late defects?
ConcaveS-shape Convex
Early or late defects?
Projects/
Releases
Defect inflow intensity trend until half-way through the project Predicted
shape of
defect inflow
profile
Overall
trend
Trend after
reaching
maximum
Defect inflow intensity trend characteristics
A1, A3, A4
& C1Increasing Decreasing
Defect inflow intensity first increases, maximizes near
to half-way and then decreasesS-shape
B1, B3 &
B4Decreasing Decreasing
Early defects, defect inflow intensity maximum early
then decreases smoothlyConcave
A2, B2, B5
& C2Increasing Increasing
Late defects, defect inflow intensity trend is positive
throughout half-way of project timelineConvex
Using predicted shape to choose a better model
Predicted
shape of
defect inflow
profile
Recommended SRGMs
For testing resource(s)
allocation
For release readiness
assessment using
current project data
S-shape Logistic Logistic
Concave Gompertz Gompertz
Convex Delayed-S Logistic
Predicting Defect Inflow (Shape & Expected No)
is useful for:
– Software engineers to make early intervention in design and
implementation to take corrective actions early,
– Quality assurance managers to plan and allocate human and
test resources optimally, and
– Project managers to manage release cycle decisions and
monitor progress.
Defect Predictions: Other useful techniques
• Predicting which files/modules are most likely to have a defect:
– Regression Models
– Machine Learning Models
File/module likely tocontain defect/not
No ofChanges
Size
Complexity
Defect Prediction: Assessing Release Readiness
Proj/Release-1LOC: 100KDefects found: 524Defect density: 5.24/KLOC
Proj/Release-4LOC: 250KDefects found: 1063Defect density:
Proj/Release-2LOC: 162KDefects found: 992Defect density: 6.12/KLOC
Proj/Release-3LOC: 550KDefects found: 3196Defect density: 5.81/KLOC
Historical Data Current Project
4.5/KLOC
Comparing defect density
Defect Prediction: Assessing Release Readiness
Defect seeding
Automated
Test
Manual
Test
Remaining defectdensity is proportional to not found seededdefects densityDevelopment
Defect Prediction: Assessing Release Readiness
Defect pooling
Test Suite-A
Team-A
Test Suite-B
Team-B
or
Uniquedefects
(A)
Uniquedefects
(B)
Common defects (A,B)
More common defects -> less defects remaining in the system
• For more information on:
– Analysing defect inflow data,
– Release readiness assessment, or
– Using metrics for process improvement.
• Contact:
– Research theme: Metrics
• Miroslaw Staron, [email protected]
• Rakesh Rana, [email protected]