Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
S P I R A L Software Process Improvement and Reliability Assurance Leb.
The Use of Cuckoo Search in
Estimating the Parameters of
Software Reliability Growth Models
2013. 8. 7
Taehyoun Kim
Dr. Najla Akram AL-Saati and Marwa Abd-ALKareem Software Engineering Dept.
College of Computer Sciences & Mathematics, Mosul, Iraq
International Journal of Computer Science and Information Security
(IJCSIS), 2013
Software Process Improvement and Reliability Assurance Lab.
Outlines
Introduction
Related Work
Cuckoo Search (CS)
Experiments
Conclusion & Future Work
Discussion
Software Process Improvement and Reliability Assurance Lab.
Introduction
Software Reliability
The probability of failure-free software operation for a specified
period of time in a specified environment (ANSI/IEEE, “Standard Glossary of Software Engineering Terminology”, STD-729-1991, 1991)
Software Reliability Growth Models (SRGMs)
The theoretical models for estimating software reliability growth
based on failure detection phenomenon during testing phase (S. Yamada and S. Osaki, “Software Reliability Growth Modeling: Models and Applications”, TSE, 1991)
Figure 1. Basic concept of SRGMs 1/21
Actual Failure data
Software Process Improvement and Reliability Assurance Lab.
Parameter Estimation of SRGM
Traditional SRGMs have some parameters which
have physical interpretations. Ex) Goel-Okumoto Model Cumulative failure function
𝑚 𝑡 = 𝑎 1 − 𝑒−𝑟𝑡
• a : the number of total failures
• r : failure detection rate
Time (Weeks)
The
num
ber
of
fail
ure
Figure 2. cumulative failure function
a = 120, b= 0.04
a = 120, b= 0.02
a = 80, b= 0.04
a = 80, b= 0.02
Failure data
Optimal
2/21
Software Process Improvement and Reliability Assurance Lab.
Classical Estimation Techniques
There are two of the most popular estimation techniques.
Maximum Likelihood Estimation (MLE)
Least Square Estimation (LSE)
However, these techniques are not effective to estimate
parameters of complicated function like non-linear function.
SRGMs are non-linear functions. (L. Shanmugam and L. Florence, “An Improved ACO Technique for Parameter Estimation”, European Journal of
Scientific Research, 2012)
The interest in heuristic algorithm began to have effect in
solving this problem.
There are many nature inspired algorithms.
3/21
Software Process Improvement and Reliability Assurance Lab.
Related Work
Genetic Algorithm (GA) Approach : Survival of the genetically fittest (T. Minohara et al., “Parameter estimation of Hyper-
Geometric distribution Software Reliability Growth
model by Genetic Algorithm”, ISSRE, 1995)
Particle Swarm Optimization
(PSO) approach : Flock migration (Zhang Ke- han et al., “Estimating parameters of
software reliability models using PSO” Computer
Engineering and Applications, 2008)
Ant Colony Optimization (ACO)
approach : Shortest path to food source (Changyou Zheng et al., A parameter Estimation Method
for Software Reliability Models, Procedia Engineering,
2011)
4/21
Software Process Improvement and Reliability Assurance Lab.
Cuckoo’s Inspiring Behavior (1/2)
Behavior 1. They have a strange habit of laying their eggs in
public nests.
Behavior 2. They remove other’s eggs to boost the hatching
probability of their own eggs.
Behavior 3.When the first cuckoo baby bird is hatched, he
instinctively throw out host eggs to increase his share of food.
5/21
Software Process Improvement and Reliability Assurance Lab.
Cuckoo’s Inspiring Behavior (2/2)
6/21
Software Process Improvement and Reliability Assurance Lab.
Cuckoo Search
①
②
③
④
⑤
(X.S. Yang and S. Deb, “Cuckoo Search via Levy Flights”, Nature & Biologically Inspired Computing (NaBIC) 2009) 7/21
Software Process Improvement and Reliability Assurance Lab.
Basic Steps of Cuckoo Search (1/5)
Step1. Generate initial population of n host nests. Ex) Goel-Okumoto model
𝑚 𝑡 = 𝑎 1 − 𝑒−𝑟𝑡
𝑎1 𝑟1
𝑛𝑒𝑠𝑡1
𝑎2 𝑟2
𝑛𝑒𝑠𝑡2
𝑎𝑛 𝑟𝑛
𝑛𝑒𝑠𝑡𝑛
• (𝑎𝑖 , 𝑟𝑖) : a candidate for optimal parameters
8/21
Software Process Improvement and Reliability Assurance Lab.
Basic Steps of Cuckoo Search (2/5)
Step2. Lay the egg (𝑎𝑘′, 𝑏𝑘′) in the k nest. K nest is randomly selected.
Cuckoo’s egg is very similar to host egg.
𝑎𝑘′ = 𝑎𝑘 + 𝑅𝑎𝑛𝑑𝑜𝑚𝑤𝑎𝑙𝑘(L𝑒 vy flight)𝑎𝑘
𝑟𝑘′ = 𝑟𝑘 + 𝑅𝑎𝑛𝑑𝑜𝑚𝑤𝑎𝑙𝑘(L𝑒 vy flight)𝑟𝑘
𝑎𝑘 𝑟𝑘
𝑛𝑒𝑠𝑡𝑘
𝑎𝑘′ 𝑟𝑘′
Cuckoo’s egg
9/21
Software Process Improvement and Reliability Assurance Lab.
Basic Steps of Cuckoo Search (3/5)
Step3. Compare the fitness of cuckoo’s egg with the
fitness of the host egg. Root Mean Square Error (RMSE)
𝑛𝑒𝑠𝑡𝑘
𝑎𝑘 𝑟𝑘
𝑎𝑘′ 𝑟𝑘′
Which egg is better? 𝑅𝑀𝑆𝐸 = 1
𝑛 𝑚 𝑡 − 𝑚𝑡
2
𝑛
𝑡=1
• 𝑚 𝑡 : the number of predicted failures by time t • 𝑚𝑡 : the number of actual failures by time t
10/21
Software Process Improvement and Reliability Assurance Lab.
Basic Steps of Cuckoo Search (4/5)
Step4. If the fitness of cuckoo’s egg is better than
host egg, replace the egg in nest k by cuckoo’s egg.
𝑛𝑒𝑠𝑡𝑘
𝑎𝑘 𝑟𝑘
𝑎𝑘′ 𝑟𝑘′
ith generation
𝑛𝑒𝑠𝑡𝑘
𝑛𝑒𝑠𝑡𝑘
𝑏𝑘
i+1th generation
𝑓𝑖𝑡𝑛𝑒𝑠𝑠 𝑐𝑢𝑐𝑘𝑜𝑜′𝑠 𝑒𝑔𝑔≥ 𝑓𝑖𝑡𝑛𝑒𝑠𝑠(ℎ𝑜𝑠𝑡 𝑒𝑔𝑔)
𝑓𝑖𝑡𝑛𝑒𝑠𝑠 𝑐𝑢𝑐𝑘𝑜𝑜′𝑠 𝑒𝑔𝑔< 𝑓𝑖𝑡𝑛𝑒𝑠𝑠(ℎ𝑜𝑠𝑡 𝑒𝑔𝑔)
𝑎𝑘 𝑟𝑘
𝑎𝑘′ 𝑟𝑘′
11/21
Software Process Improvement and Reliability Assurance Lab.
Basic Steps of Cuckoo Search (5/5)
Step5. If host bird notice it, the nest is abandoned
and new one is built. (p <0.25)
(to avoid local optimization)
Iterate steps 2 to 5 until termination criterion is met.
𝑛𝑒𝑤 𝑛𝑒𝑠𝑡𝑘
𝑎𝑘′′ 𝑏𝑘′′
𝑛𝑒𝑠𝑡𝑘
𝑏𝑘 𝑎𝑘′ 𝑏𝑘′
12/21
New egg
Cuckoo’s egg
Software Process Improvement and Reliability Assurance Lab.
Experimental Set-up (1/2)
Comparison with other techniques
reference Parameter estimation techniques Data Set
A. Sheta’s [1] Particle Swarm Optimization (PSO) Data Set 1
Data Set 2
Data Set 3
L. Shanmugam et al.’s [2] Ant Colony Optimization (ACO) Musa Data 1*
Musa Data 2*
Musa Data 3* L. Shanmugam et al.’s [3] Extended Ant Colony Optimization
(Ex-ACO)
[1] A. Sheta, “Reliability Growth Modeling for Software Fault Detection Using Particle Swarm Optimization”, IEEE Congress on
Evolutionary Computation, 2006
[2] L. Shanmugam and L.A. Florence, “Comparison of Parameter Best Estimation Method for Software Reliablity Models”,
International Journal of Software Engineering & Applications (IJSEA), 2012
[3] L. Shanmugam and L. Florence, “An Improved ACO Technique for Parameter Estimation”, European Journal of Scientific
Research, 2012
* John D. Musa, Software Reliability Data, Cyber Security and Information Systems Information Analysis Center, January 1980.
Table1. Estimation techniques and data sets
13/21
Software Process Improvement and Reliability Assurance Lab.
Experimental Set-up (2/2) SRGMs for the experiments
Parameter settings for the cuckoo search
SRGMs Mean value function
(cumulative failure function)
Goel-Okumoto (GO) model 𝑚 𝑡 = 𝑎 1 − 𝑒−𝑟𝑡
Power (POW) model 𝑚 𝑡 = 𝑎𝑡𝑟
Yamada Delayed S-Shaped (DSS) model 𝑚 𝑡 = 𝑎 1 − (1 + 𝑟𝑡)𝑒−𝑟𝑡
Musa-Okumoto (M-O) model 𝑚 𝑡 = 𝑎𝑙𝑛(1 + 𝑟𝑡)
Parameters Value
Lower and Upper bound for parameter a [0.00001 ~ 2000]
Lower and Upper bound for parameter r [0.00001 ~ 1]
Number of Nests 10
Number of iterations(Generations) 100
Table2. SRGMs for the experiments
Table3. Parameter settings for cuckoo search 14/21
Software Process Improvement and Reliability Assurance Lab.
The impact of training and testing data
A balance point is training 70% and testing 30%.
Table4. The impact of training and testing
percentage
(a) data set 1, (b) data set 2, (c) data set 3
(a)
(c)
(b)
15/21
Software Process Improvement and Reliability Assurance Lab.
Comparison with other techniques (1/4)
PSO vs. Cuckoo Search (Data set 1)
Training 70% Testing 30%
PSO CS PSO CS
G-O 20.2565 34.0933 119.4374 16.8945
POW 22.2166 44.8663 152.9372 33.6623
DSS 15.9237 32.6376 26.3015 10.9945
Table5. Comparison with PSO using Data set 1
Figure3. Comparison with PSO using Data set 1
16/21
(A. Sheta, “Reliability Growth Modeling for Software Fault Detection Using Particle Swarm Optimization”, IEEE Congress on
Evolutionary Computation, 2006 )
Software Process Improvement and Reliability Assurance Lab.
Comparison with other techniques (2/4)
PSO vs. Cuckoo Search (Data set 2)
Training 70% Testing 30%
PSO CS PSO CS
G-O 24.9899 33.2311 80.8963 14.2998
POW 32.355 47.0571 149.9684 56.6807
DSS 20.8325 27.9159 17.0638 11.8833
Table6. Comparison with PSO using Data set 2
Figure4. Comparison with PSO using Data set 2
17/21
Software Process Improvement and Reliability Assurance Lab.
Comparison with other techniques (3/4)
PSO vs. Cuckoo Search (Data set 3)
Training 70% Testing 30%
PSO CS PSO CS
G-O 12.8925 13.5404 13.6094 8.9523
POW 11.9446 13.0886 14.0524 13.4669
DSS 18.5807 13.6634 47.4036 15.1916
Table7. Comparison with PSO using Data set 3
Figure5. Comparison with PSO using Data set 3
18/21
Software Process Improvement and Reliability Assurance Lab.
Comparison with other techniques (4/4)
ACO vs. Ex-ACO vs. Cuckoo Search
Musa Data Set 1 - Training 100%
ACO EX-ACO CS
G-O 60.0371 28.5891 41.7971
POW 52.8854 34.0521 45.9783
DSS 52.8854 33.0461 42.2256
M-O 26.0385 17.359 41.7732
Musa Data Set 2 - Training 100%
ACO EX-ACO CS
G-O 71.5489 34.0709 21.7256
POW 57.5801 47.5814 15.5885
DSS 57.5801 48.4914 22.4944
M-O 36.1891 24.126 19.5448
Musa Data Set 3 - Training 100%
ACO EX-ACO CS
G-O 71.4015 35.0007 25.7682
POW 53.2234 34.2645 28.1951
DSS 53.2234 35.2635 25.7294
M-O 33.1728 22.1152 26.4575
Table8. Comparison with ACO and
Extended ACO
(a) musa data set 1,
(b) musa data set 2,
(c) musa data set 3
(a)
(c)
(b)
19/21
(L. Shanmugam and L.A. Florence, “Comparison of Parameter Best Estimation Method for Software Reliablity Models”, International Journal of Software
Engineering & Applications (IJSEA), 2012)
(L. Shanmugam and L. Florence, “An Improved ACO Technique for Parameter Estimation”, European Journal of Scientific Research, 2012)
Software Process Improvement and Reliability Assurance Lab.
Conclusion & Discussion
Conclusion
Cuckoo Search can be successfully applied to find
acceptable solutions to problem of parameter estimation
of SRGM.
Future work
Other swarm intelligent methods can be applied and
compared. Ex) Harmony Search, Firefly algorithm, Accelerated Particle Swarm
Optimization(APSO), etc..
20/21
Software Process Improvement and Reliability Assurance Lab.
Discussion
Cons.
New heuristic algorithm was applied to estimate the
parameters of SRGM.
The results are obtained very quickly and accurately.
Pros.
Experiments were not proper to compare cuckoo search
with other heuristics.
21/21
Software Process Improvement and Reliability Assurance Lab.
Q&A
Thank you
Software Process Improvement and Reliability Assurance Lab.
Least Square Estimation (LSE) (1/2)
Minimize the sum of square of the deviations
between actual data and expected data
Ex) parameter estimation of function 𝑓 𝑥 = 𝑎𝑥 + 𝑏
Figure 2. Basic concept of LSE
𝐸 𝑎, 𝑏 = 𝑦𝑖 − 𝑎𝑥𝑖 + 𝑏2
𝑛
𝑖=1
When 𝑑 𝐸 𝑎,𝑏
𝑑𝑎 = 0,
𝑑 𝐸 𝑎,𝑏
𝑑𝑏 = 0,
The values of a and b are the best estimates of a and
b.
a
b
E(a,b)
Optimal Point
Software Process Improvement and Reliability Assurance Lab.
Least Square Estimation (LSE) (2/2)
Ex) Goel-Okumoto Model
𝑚 𝑡 = 𝑎 1 − 𝑒−𝑟𝑡
𝐸 𝑎, 𝑏 = (𝑦𝑖 − 𝑎(1 − 𝑒−𝑟𝑡𝑖))2
𝑛
𝑖=1
When 𝑑 𝐸 𝑎,𝑏
𝑑𝑎 = 0,
𝑑 𝐸 𝑎,𝑏
𝑑𝑏 = 0,
The values of a and b are the best estimates of
a and b.
a b
E(a,b)
Software Process Improvement and Reliability Assurance Lab.
L𝑒 vy flight (1/2)
A random walk in which the step-lengths have a probability
distribution that is heavy-tailed (to avoid local optimization)
Current point
Step Size
Step Size * random()
Figure6. the basic concept of random walk (L𝑒 vy flight)
Step Size * random()
- Normal Distribution - Heavy tail distribution
Software Process Improvement and Reliability Assurance Lab.
L𝑒 vy flight (2/2)
Normal distribution VS. Heavy tail distribution
Figure7. an example of Brownian motion
(Normal distribution)
Figure8. an example of L𝑒 vy flight
(Heavy tail distribution)