Slide 1

New Directions for Software Metrics

Norman Fenton

Agena Ltd and Queen Mary University of London

PROMISE

20 May 2007

Slide 2

Overview

• History of software metrics

• Good and bad newsHard project constraintsProject trade-offsDecision-making and intervention

• The true objective of software metrics?

• Why we need a causal approach

• Models in action

• Results

Slide 3

Metrics History: Typical Approach

What I really want to measure (Y)

What I can measure (X)

Y = f (X)

Slide 4

Metrics History: the drivers

• ‘productivity’=size/effort

• ‘effort’=a*sizeb

• ‘quality’=defects/size

Slide 5

Metrics history: size matters!

• LOC

• Improved size metrics

• Improved complexity metrics

Slide 6

Some Decent News About Metrics

• Empirical results/banchmarks

• Significant industrial activity

• Academic/research output

• Metrics in programmer toolkits

Slide 7

….But Now the Bad News

• Lack of commercial relevance

• Programmes doomed by data

• Activity poorly motivated

Failed to meet true objective of quantitative risk assessment

Slide 8

Slide 9

Regression models….?

Slide 10

Using metrics and fault data to predict quality

0

0

Post-release faults

10

20

30

40 80 120 160

Pre-release faults

?

Slide 11

Pre-release vs post-release faults: actual

Post-release faults

0

10

20

30

0 40 80 120 160

Pre-release faults

Slide 12

What we need

What I think is... ?

Slide 13

The Good News

• It is possible to use metrics to meet the real objective

• Don’t need a heavyweight ‘metrics programme’

• A lot of the hard stuff has been done

Slide 14

A Causal Model (Bayesian net)

Residual DefectsTesting Effort

Problemcomplexity

Defects found and fixed

Defects IntroducedDesign processquality

Operational defectsOperational usage

Slide 15

A Model in action

Slide 16

Slide 17

https://intranet.dcs.qmul.ac.uk/courses/coursenotes/DCS235/

Slide 18

Slide 19

Slide 20

A Model in action

Slide 21

Slide 22

Slide 23

A Model in action

Slide 24

Slide 25

Slide 26

Slide 27

Slide 28

Slide 29

Actual versus predicted defects

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 500 1000 1500 2000 2500

Actual defects

pre

dic

ted

def

ects

Slide 31

Conclusions

• Heavyweight data and classical statistics NOT the answer

• Empirical studies laid groundwork• Causal models for quantitative risk

Slide 32

…And

You can use the technology NOW

www.agenarisk.com