A Neuro-Fuzzy Model with SEER-SEM for Software

Effort Estimation

Wei Lin Du, Danny Ho*, Luiz F. Capretz

Software Engineering, University of Western Ontario, London, Ontario, Canada

* NFA Estimation Inc., Richmond Hill, Ontario, Canada

November 2010

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

Purpose Integrate neuro-fuzzy (NF)

technique with SEER-SEM Evaluate estimation performance

of NF SEER-SEM versus SEER-SEM

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

SEER-SEM SEER-SEM was trademarked by

Galorath Associates, Inc. (GAI) in 1990

Effort estimation is one of the SEER-SEM algorithmic models

SEER-SEMEstimationProcessing

Size

Personnel

Environment

Complexity

Constraints

Effort

Cost

Schedule

Risk

Maintenance

SEER-SEM Effort Estimation

Software Size Lines, function points, objects, use cases

Technology and Environment Parameters Personal capabilities and experience (7) Development support environment (9) Product development requirements (5) Product reusability requirements (2) Development environment complexity (4) Target environment (7)

SEER-SEM Equations,)(

2.1

4.0

C

SDK

te

e

where: E Development effort

K Total lifecycle effort including development and maintenance

Se Effective size

D Staffing complexity

Cte Effective technology

Ctb Basic technology

,393489.0 KE

TURN

ctbx

C tb 511.4

ln70945.3exp2000mentParmAdjust

CC tbte

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

NFA

FM2

…

NFB1

NFBN

Algorithmic Model NFB2

Output MetricMo

FM1

FMN

RF2

RF1

RFN

ARF1

ARFN

Preprocessing

Neuro-Fuzzy

Inference

System

(PNFIS)

ARF2

…

MVVV ,,, 21

where N is the number of contributing factors, M is the number of other variables in the Algorithmic Model, RF is Factor Rating, ARF is Adjusted Factor Rating, NFB is the Neuro-Fuzzy Bank, FM is Numerical Factor/Multiplier for input to the Algorithmic

Model, V is input to the Algorithmic Model,and Mo is Output Metric.

USA Patent No. US-7328202-B2

N

N

NAiN

Ai2

Ai1

…… …

ARFi FMi

FMPi1

FMPiN

FMPi2

w1

wN

Layer1 Layer3 Layer4 Layer5Layer2

NFB

where ARFi is Adjusted Factor Rating for contributing factor i,

is fuzzy set for the k-th rating level of contributing factor i,

is firing strength of fuzzy rule k,

is normalized firing strength of fuzzy rule k,

is parameter value for the k-th rating level of contributing factor i,

and is numerical value for contributing factor i.

1w

Nw

11 iFMPw

iNN FMPw

ikAkw

ikFMPkw

iFM

NF SEER-SEM

ACAP NF1

NF2

NFm

…

Software Estimation

Algorithmic Model

Effort EstimationSEER-SEM

Effort Estimation)(

2.1

4.0

C

SDK

te

e,393489.0 KE

Size, SIBR

P1

P2

P34

AEXP

Complexity (Staffing)

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

Performance Metrics Relative Error (RE)= (Est. Effort – Act. Effort) / Act. Effort Magnitude of Relative Error (MRE)= |Est. Effort – Act. Effort | / Act. Effort Mean Magnitude of Relative Error (MMRE)= (∑MRE) / n Prediction Level (PRED) PRED(L) = k / n

Design of Evaluation

Case ID Description

C1 No outliers

C2 Including all outliers

C3 Excluding part of outliers

C4-175% for Learning, 25% for

testing

C4-250% for Learning, 50% for

testing

MMRE Results

Case IDMMRE (%)

SEER-SEM Validation Change

C1 84.39 61.05 -23.35

C2 84.39 59.11 -25.28

C3 84.39 59.07 -25.32

C4-1 50.49 39.51 -10.98

C4-2 42.05 29.01 -13.04Negative value of MMRE change means improvement

MMRE Results

Summary of MMRE Validation

-23.35% -25.32%-10.98% -13.04%

-19.59%-25.28%-30.00%

-10.00%

10.00%

30.00%

50.00%

70.00%

90.00%

110.00%

C1 C2 C3 C4-1 C4-2 Average

MM

RE

an

d C

han

ge

SEER-SEM

Validation

Change

PRED Results

SEER-SEM

Average of

ValidationChange

PRED(20%)

39.76% 27.48%-

12.28%

PRED(30%)

49.27% 36.46%-

12.81%

PRED(50%)

62.02% 55.35% -6.67%

PRED(100%)

85.55% 97.69% 12.14%

Positive value of PRED change means improvement

Summary of Evaluation Results

MMRE is improved in all cases, with the greatest improvement over 25%

Average PRED(100%) is increased by 12%

NF SEER-SEM improves MMRE by reducing large MREs

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

Conclusion

NF with SEER-SEM improves estimation accuracy

General soft computing framework works with various effort estimation algorithmic models

Future Directions Evaluate with original SEER-SEM

dataset Evaluate general soft computing

framework with: more complex algorithmic models other domains of estimation

THANKS !

Any Questions?