Empirical Validation of OO Metrics in Two Different

Iterative Software Processes

Mohammad Alshayeb

Information and Computer Science Department

King Fahd University of Petroleum and Mineralshttp://www.alshayeb.com

Alshayeb, M. and Wei Li, “An Empirical Validation of Object-Oriented Metrics in Two Iterative Processes,” IEEE Transactions

on Software Engineering, Vol. 29, No. 11, November 2003.

4 January 2004 2

Agenda

Research Objective. Introduction.Research Data.Research Hypotheses.Validation Studies.Conclusion.Future Work.

4 January 2004 3

Objective

Validate the Predictive Capability of Object-Oriented (OO) Metrics in Two Iterative Software Processes.

4 January 2004 4

Measurement Acceptance

No Wide Acceptance of Software Measurements Because: Management personnel rarely know how

to analyze, interpret or use the software metrics.

Various ways of analyzing software metrics.

Lack of measurement validation.

4 January 2004 5

Measurement Validation

Validation Assures That Measurements Assess What They Are Supposed to Measure.

Types of Validation: Theoretical: the measure is a proper numerical

characterization of a claimed attribute. Empirical: establishes the accuracy of a prediction

system by comparing the model performance with known data in a given environment.

4 January 2004 6

Software Processes

The Waterfall ModelPrototype ModelRapid Application Development ModelThe Incremental ModelThe Spiral ModelComponent Assembly ModelThe Iterative Development Process ModelExtreme Programming

4 January 2004 7

The Iterative Development Process

Domain Analysis

Release Iteration

Prototype Highest Risk

Risk Assessment

Software Architecture

Requirements definition

Test Suite & Environment Development

Integrate with Previous Iteration

Customer Requirements

Iteration

Luckey et al. , “Iterative Development Process with Proposed Applications,” 1992

4 January 2004 8

Agile Process: Extreme Programming (XP) “A lightweight, efficient, low-risk,

flexible, predictable, scientific, and fun way to develop software”

-Beck,

XP Applicability: Projects with vague or changing requirements. Small groups of programmers. The need for customer input.

4 January 2004 9

XP Is Different

Continuity of Feedback Throughout Its Short Cycles.

Incremental Planning.Flexibility in Scheduling on the Basis of

Business Needs.Dependency on Unit Tests Written by

Both Developers and Customers.Pair Programming.

4 January 2004 10

XP Breakdown

Customer System

Priority Organization of user stories

Story 1 Story 2 Story 3

Division of system into user stories

Iteration 1 Iteration 2 Iteration 3

Iterations assigned for the development of each story

Story 2 Story 3 Story 1

Release 2 Release 3Task 1 Task 3

Test Case 2Test Case 1 Test Case 3

Release 1Working, tested system with story 2

integrated

Passed, integrate with the system

Task 2

4 January 2004 11

XP and Other Software Processes

Analysis

Design

Implementation

Testing

Waterfall Long Cycled Iterative Process Short Cycled Iterative Process XP

Beck, “Embracing changes with Extreme programming,” 1999

4 January 2004 12

Frameworks

“A Framework is an OO class hierarchy plus a built-in model which defines how the objects derived from the hierarchy interact with one another”

–Lewis et al,

4 January 2004 13

OO Metrics Suites

C&K suite Weighted Methods per

Class (WMC) Depth of Inheritance Tree

(DIT) Number Of Children

(NOC) Coupling Between Objects

(COB) Response For a Class

(RFC) Lack of Cohesion of

Methods (LCOM)

Li Suite Number of Ancestor Classes

(NAC) Number of Local Methods

(NLM) Class Method Complexity

(CMC) Number of Descendent

Classes (NDC) Coupling Through Abstract

data types (CTA) Coupling Through Message

passing (CTM)

4 January 2004 14

Research Data

Iterative Software Process

Process: Extreme Programming (XP) Process: the Framework Iterative

Data: Two Java Applications Data: JDK Releases

4 January 2004 15

Research Data

System 1

Story 1

Story 2

Story 3

Story 4

Story 5

Story 6

JDK

JDK 1.0

JDK 1.1

JDK 1.2

JDK 1.3

JDK 1.4

System 2

Story 1

Story 2

Story 3

Story 4

Story 5

Story 6

Story 7

4 January 2004 16

Data Collection

The tasks planned for the current iteration cycle. The tasks come from the user stories, which are supplied by the customer.

A description of the progress or failure of progress towards completing the tasks.

The time spent on each task measured by hours. A description of the problems encountered during

implementing a task and the attempted solutions. A description of the changes made on the

system, the reasons for the changes, and the affected classes.

4 January 2004 17

Data Analysis

Tools support Webgain’s Quality AnalyzerMetamata

Metrics Object-Oriented metrics collection. Collects 12 metrics.

Krakatau Project Manager for Java Collecting development/maintenance effort

measured by lines of code.

4 January 2004 18

Metrics Validation

Criteria Li and Henry

Basili et al. Briand et al.

EL Emam et al.

Briand and Wüst

Suite of metrics

CK metrics CK metrics

Cohesion and

Coupling

C&K, Lorenz

and Kidd

Size, Cohesion

and Coupling

CK& and Li metrics

CK& and Li metrics

Type of products

OO dialect of Ada

C++ Ada Java C++ Java Java

Dependent variable

Number of changes in

components

Fault occurrence

in C++ classes

Fault occurrence

in Ada packages

Fault-proneness of a class

effort Design Changes (Lines

added, changed, and deleted), in a

class.

Maintenance effort, refactoring

effort

Statistical technique

Least-square

regression

Logistic regression

Logistic regression

Logistic regression

Poisson regression

and regression

trees

Least-square regression

Least-square regression

Long-Cycled and Short-Cycled

Iterative Processes

Short-Cycled Iterative Processes

Alshayeb and Li

Process?

4 January 2004 19

Multiple Regression

Investigate the Relationship Between a Dependent Variable and Multiple Independent Variables

y = x +xx The x’s are the independent (predictor or regressor)

variables. y is the dependent (response) variable. —the statistical error—is the difference between the

observed value y and the line (x +xx )

regression coefficient. R2: The variance percent in the dependent variable

based on the knowledge of the independents.

4 January 2004 20

Empirical Validation of Object-Oriented MetricsEmpirically Validate the Predictive

Capability of OO Metrics in the Long-Cycled (Framework) and Short-Cycled (Agile) Iterative Software Processes Investigate the relationship between metrics and

development/maintenance efforts in two forms of iterative processes.

Types of Changes: Lines added (LA), Lines changed (LC), and Lines

deleted (LD). Refactoring and Error-Fix efforts.

4 January 2004 21

Example of Lines Added, Deleted and Changed// Example.java version 1

public class Example{

public static void main(String args[]){

System.out.println("Welcome to java");

System.out.println("Hello");

System.out.println("World");

}

}

// Example.java version 2

public class Example{

public static void main(String args[]){

System.out.println("Hello");

System.out.println("Java");

System.out.println("new World");

}

}

Deleted

Added

Changed

4 January 2004 22

Research Hypotheses

Hypothesis 1: Using OO metrics, we can predict design changes indicated by added, changed, and deleted source lines of code in classes from one iteration (release) to the next in the long-cycled framework iterative process.

Hypothesis 2: Using OO metrics, we can predict added, changed, and deleted source lines of code in classes from one iteration to the next in the short-cycled XP iterative process.

4 January 2004 23

Research Hypotheses

Hypothesis 3: Using OO metrics, we can predict error-fix maintenance effort (measured in man-hours) in classes from one iteration to the next in the short-cycled XP iterative process.

Hypothesis 4: Using OO metrics, we can predict refactoring effort (measured in man-hours) in classes from one iteration to the next in the short-cycled XP iterative process.

4 January 2004 24

Dependant and Independent Variables -Hypotheses 1 and 2Dependant Variables Are:

LA, LC, and LD. Independent Variables

WMC, DIT, LCOM, NLM, CTA, and CTM metrics.

4 January 2004 25

Research Results -Hypothesis 1

Hypothesis 1

Dependent

Variables JDK1.0-1.1 JDK1.1-1.2 JDK1.2-1.3 JDK1.3-1.4

R2 % R2 % R2 % R2 %

LA 42.6 10.6 27.8 32.7

LC 56.0 18.6 24.9 13.6

LD 58.0 11.0 12.9 7.30

4 January 2004 26

Research Results -Hypothesis 1

Object-Oriented Metrics Cannot Predict Design Changes Indicated by Added, Changed, and Deleted, Source Lines of Code in Classes From One Iteration (Release) to the Next in the Long-cycled Framework Iterative Process.

4 January 2004 27

Research Results -Hypothesis 2

Hypothesis 2

Dependent

Variables Story 5-6

R2 %

LA 95.6 LC 97.9 LD 88.5

4 January 2004 28

Research Results -Hypothesis 2

Hypothesis 2

Dependent

Variables Story 4-5 Story 6-7

R2 % R2 %

LA 97.7 54.2 LC 98.4 72.9 LD 71 51.5

4 January 2004 29

Research Results -Hypothesis 2

Object-Oriented Metrics May Predict Design Changes Indicated by Added, Changed, and Deleted Source Lines of Code in Classes From One Iteration to the Next in the Short-cycled XP Iterative Process Only When System Design Accumulates to Certain Mass.

4 January 2004 30

Dependant and Independent Variables -Hypotheses 3 and 4Dependant Variables Are:

Refactoring and Error-Fix efforts. Independent Variables

WMC, DIT, LCOM, NLM, CTA, and CTM metrics.

4 January 2004 31

Research Results –Hypotheses 3,4

Hypotheses 3 and 4

System 1 Refactoring Error Fix

R2 % R2 %

Story 5 76.5 88.3

Story 6 72.9 69.4

4 January 2004 32

Research Results –Hypotheses 3,4

Hypotheses 3 and 4

System 2 Refactoring Error Fix

R2 % R2 %

Story 4 61 44.5

Story 5 66.0 74.4

Story 6 14.3 10.0

Story 7 91.2 90

4 January 2004 33

Research Results –Hypotheses 3,4

Object-oriented Metrics Can Predict the Refactoring Effort and Error-fix Maintenance Effort in the XP-like Process When the System Design Accumulates to Certain Mass.

4 January 2004 34

Conclusion

The Predictive Capability of OO Metrics: OO metrics are reasonably effective in predicting

design changes in the short-cycled XP-like process, but they are largely ineffective in the long-cycled framework evolution process.

OO metrics' predictive capability is limited to the design and implementation changes during the development iterations, not the long-term evolution of an established system in different releases.

OO metrics are effective in predicting error-fix and refactoring efforts in the short-cycled agile process.

4 January 2004 35

Conclusion

The Development Software Process is Related Directly to the Predictive and Project-Progress Indicative Capabilities of OO Metrics: It is thus essential to clearly identify the

software process when validating OO metrics.

4 January 2004 36

Future Work

Studies Can Target OO Metrics Capabilities in Different Software Processes.

Validation Studies More Diverse Data Sets Are Needed to Test the Prediction Accuracy.

Automate the Steps Followed in This Study and Build a Tool to Gather, to Analyze, and to Use Metrics and the Statistical Models in a CASE Tool.

4 January 2004 37

Questions