Presented by Vu Nguyen on behalf of Vu Nguyen, Barry Boehm, Phongphan Danphitsanuphan

University of Southern California

Center for Systems and Software Engineering

© 2009, USC-CSSE 1

Assessing and Estimating Corrective, Enhancive, and Reductive Maintenance

Tasks: A Controlled Experiment *

Presented by Vu Nguyen

on behalf of

Vu Nguyen, Barry Boehm, Phongphan Danphitsanuphan

(*) paper accepted for Asia-Pacific Software Engineering Conference 2009 (APSEC 2009)



© 2009, USC-CSSE 2

Outline

Motivation and Background

Experiment Design

Results and Explanatory Models

Conclusions



© 2009, USC-CSSE 3

Motivation and Background



© 2009, USC-CSSE 4

Maintenance is crucial in software engineering

• Systems are tightly coupled with their environment

– Environment changes require changing its software systems

• Technologies and requirements are continuously changing

– Software systems are outdated quickly

– Software systems must be updated and upgraded to maintain their values

• Maintenance is important in market competition

– New software has more advantages than the existing one

– Software system must be upgraded to keep its market share



© 2009, USC-CSSE 5

Majority of software costs incur after the first operational release (Boehm ’81)

• Maintenance cost is usually 2x to 100x as much as new development cost (Sommerville, 2006)

K$

Time

100 200 300

System 1

System 2

Release 1 Release 2 Release 3

Release 1 Release 2

Adapted from (Summerville, 2006)New Development

Maintenance

Release N



© 2009, USC-CSSE 6

Software estimation community has paid little attention to software maintenance

• Most estimation models regard maintenance estimation as secondary

– COCOMO, SEER-SEM, SLIM, PRICE-S models were built using mainly data of new development projects

• Use of SLOC metrics for models is inconsistent

– Some models use SLOC added, modified, deleted

– Others use only SLOC added and modified

• Impact of different SLOC metrics on productivity has not been investigated

Projects tend to use experience or expert judgment methods to estimate software effort and cost instead



© 2009, USC-CSSE 7

Research Questions and Hypotheses

• RQ1: Are there any differences in the productivity of enhancive, corrective, and deductive maintenance?

– Hypothesis 1 (H1): no difference

• RQ2: Are there any differences in the effort distribution among the maintenance types?

– Hypothesis 2 (H2): no difference

Explanatory models to estimate participant’s maintenance effort



© 2009, USC-CSSE 8

Software Maintenance

• Software Maintenance

– “Modification of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment” [IEEE ‘98]

• Types of Maintenance

– Swanson ’76: Adaptive, Corrective, Perfective

– IEEE ’98: all Swanson’s plus Preventive

– Chapin et al, 2001: 12 types, including three business rules types:• Enhancive• Corrective• Reductive



© 2009, USC-CSSE 9

Experimental Design



© 2009, USC-CSSE 10

Experiment DescriptionTask Group # Tasks # Participants

Enhancive 5 7

Corrective 6 9

Reductive 6 8

• 23 masters’ students and 1 senior, computer science major

• Participants worked on tasks individually in the lab

– Enhancive: add new capabilities

– Corrective: fix the existing capabilities

– Reductive: remove the existing capabilities

• UCC as a target program

– 5K+ source statements (logical SLOC) in 20 C++ classes

• MS Visual Studio 2005 was used for maintenance



© 2009, USC-CSSE 11

Calculating Maintenance SLOC

Equivalent SLOC = TRCF x AAM

1,100

1,])1(1[1( 2

AAFforUNFMxSU

AAF

AAFforUNFMxSUxAAFAAFAAM

TRCF

SAAF

TRCF = the total SLOC of task-relevant code fragments

S = the size in SLOC (added, modified, or deleted)

SU = the software understandability

UNFM = the level of programmer unfamiliarity with the program

Task-relevant code fragment

(TRCF)



© 2009, USC-CSSE 12

Results and Models



© 2009, USC-CSSE 13

Resulted Data

• 24 students participated

• Each task requires four activities

– Task comprehension

– Code isolation

– Editing code

– Unit test

• Timesheet has 490 activity records, totaling 77.02 hours

• Total of 909 SLOC added, modified, and deleted

402 added

216 modified

291 deleted



© 2009, USC-CSSE 14

Effort distribution is different among the groups

• Corrective group spent much time for code isolation

– twice as much as that of the enhancive group

• Enhancive group spent majority of time for editing code

• Effort distribution is statistically different among three groups (p-value = 0.0013)

– H2 is rejectedKruskal-Wallis rank-sum test



© 2009, USC-CSSE 15

Productivity is significantly different among the groups

• Corrective group has lowest productivity

• Reductive group has highest productivity

• Productivity between groups are statistically different (p-value = 0.0004)

– H1 is rejectedEnhancive Reductive Corrective

Pro

du

ctiv

ity

(SL

OC

/Hou

r)

40 30 20 10



© 2009, USC-CSSE 16

Participant Time Explanatory Models

• Two models for participant effort

M1 Effort = 78.1 + 2.2 * S * EAF

M2 Effort = 43.9 + (2.8*Add + 5.3*Mod + 1.3*Del) * EAF

Effort = Time spent by the participant on all maintenance tasks

EAF = Effort adjustment factor, a product of Programmer capability (PCAP), Language experience (LTEX), and Platform Experience (PLEX)

Add, Mod, Del = Equivalent SLOC added, modified, and deleted by the participant, respectively

S = Add + Mod + Del

All the estimates of coefficients are significant (p-value < 0.05)

R2 = 0.5 R2 = 0.75



© 2009, USC-CSSE 17

Model M2 outperforms M1

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Participant #

MR

E

M1 M2

M1 M2

MMRE 33% 20%

PRED(.25) 46% 71%

PRED(.3) 58% 79%

i

iii Actual

EstimateActualMRE



© 2009, USC-CSSE 18

Threats to Validity

• Internal design

– Groups have imbalanced skills and capability

– Imbalanced complexity of tasks on three groups

– Incorrect time recorded

• Generalizability

– Professional programmers are more experienced

– Professional programmers are more familiar with the software maintained

– Maintenance process is different in industry



© 2009, USC-CSSE 19

Conclusions

• Productivity and effort distribution are significantly different among the maintenance types

• SLOC metrics are relevant factors for estimating effort

• Three SLOC metrics (added, modified, and deleted) have different impact on effort

– SLOC deleted is an important factor for estimating effort

– It is more expensive to modify than to add or delete a statement

• Assigning experienced programmers to fixing defect can save up to 40% of effort



© 2009, USC-CSSE 20

References

• Barry W. Boehm, “Software Engineering Economics”, Prentice Hall, 1991

• Ian Sommerville, “Software Engineering,” 8th Ed., Addison-Wesley, 2006

• Ned Chapin, et al., “Types of software evolution and software maintenance,” Journal of Software Maintenance: Research and Practice, v.13 n.1, p.3-30, Jan. 2001

• IEEE, IEEE Standard Glossary of Software Engineering Terminology. Institute of Electrical and Electronics Engineers: New York NY, 1990: 83 pp

• IEEE Std 1219-1998, IEEE Standard for Software Maintenance, IEEE Computer Society, 1998

Documents

Presented by Vu Nguyen on behalf of Vu Nguyen, Barry Boehm, Phongphan Danphitsanuphan