8/10/2019 sa_paper

1/18

1

Association Rule Mining through Adaptive Parameter Control in Genetic Algorithm and

Particle Swarm Optimization

Abstract: Association rule mining is a data mining task for which a great deal of academic research has

been done and many algorithms are proposed. Evolutionary algorithms Among EAs Genetic Algorithm(GA) and Particle Swarm Optimization (PSO) is more suited for mining Association rules. The

bottleneck inboth GA and PSO is setting the precise values for their control parameters for the right

problem. Both ga and pso parameters are to be tuned. This paper proposes adaptive methodology forparameter control of both GA and PSO. In Adaptive Genetic Algorithm (AGA) the mutation rate is

varied and in Adaptive Particle Swarm Optimization the acceleration coefficients are adjusted throughEstimation of Evolution State (EES) and inertia weight adaptation is based on fitness values. Bothmethods tested on five datasets from UCI repository proved to generate association rules with better

accuracy and rule measures when compared to simple GA and PSO.

Keywords: Association Rule Mining, Genetic Algorithm, Particle Swarm Optimization, Adaptive GA,Adaptive PSO, Estimation of Evolution State.

1.

Introduction

Association rule mining aims in extracting interesting correlations or patterns among sets of items in the

transaction databases or other data repositories. It is one of the most important and well researchedtechniques of data mining. The application area of data mining varies from market analysis to businessintelligence, has been now extended to medical domain, temporal/spatial data analysis and web

mining.AR mining Atsk Hence the accuracy of the association rules mined and the relationship betweenattributes has become an important issue. The standard association rule mining methods such as Apriori

[5,7], FP growth tree [6,7] scans the whole dataset for each attribute match, increasing the input/outputoverhead of the system. The rules generated are with single objective aiming at accuracy alone and the

number of rules generated is vast. Pruning and summarization are needed to filter the significant rules.

The efficiency of association rule mining can be enhanced by

Reducing the number of passes over the database

Making the process as multiobjective

Sustaining the search space efficiently

Evolutionary Algorithms (EA) such as Genetic Algorithm (GA) and Particle Swarm Optimization (PSO),provides solution for meeting the above three requirements for mining association rules[1,2,3]. Bothmethods have proven to generate association rules with better predictive accuracy and reduction inexecution time. However one laborious aspect of all EAs, including PSO and GA, is performing the

appropriate parameter adjustments [11].

In solving problems with GA and PSO, their properties affect their performance. The properties of bothGA and PSO depend on the parameter setting and hence user needs to tune the parameters to optimize the

performance. The interaction between the parameters has complex process and a single parameter willhave a different effect depending on the value of the others. Without prior knowledge of the problem,

parameter tuning is difficult and time consuming. Different combinations of parameter values are to betried to find the best setting. The two major ways of parameter setting are through parameter tuning and

parameter control. Parameter tuning is the commonly practiced approach that amounts to find the

appropriate values for the parameters before running the algorithm. Parameter control steadily modifies

8/10/2019 sa_paper

2/18

2

the control parameter values during the run. This could be achieved either through deterministic oradaptive or self-adaptive techniques[citation needed].

Deterministic parameter control takes place using a deterministic rule that modifies the strategy parameterwithout any feedback. This method becomes unreliable for most problems due to As the parameteradjustments must rely on the status of the problem at current time and this method becomes unreliable for

most problems. Some connection words as contrary needed. In self-adaptive approach the parameters tobe controlled are encoded into the candidate solution which may result to deadlock change the statementsof deadlock obtaining the good solution depends on finding the good setting of parameters, butmeanwhile, obtaining the good setting of parameters depends on finding the good solution, which is akind of the chicken or the egg causality dilemma. Moreover, extra bits are required to store these strategy

parameters, so the dimensionality of the search space is increased. Thus the corresponding search space

becomes larger and hence the complexity of the problem is increased.

Therefore the adaptive method is the solution. By using adaptive operatorin genetic algorithm proposedby Srinvivas [12] is an improvement of the basic genetic algorithm. By using adaptive operator, high

convergence speed and high convergence precision has been obtained . An adaptive parameter control isapplied for inertia control in Eberhart and Kennedy [13], for linearly decreasing inertia over the

generations in Arumugam and Rao [14], and for a fuzzy logical controller in Luo and Yi [15]. It was alsoused for the acceleration coefficients in Ratnaweera et al. [16] and Arumugam and Rao [14] by balancingthe cognitive and the social components. The analysis of change in the parameters over evolution and

adaptation accordingly can enhance the performance of the EA. In this paper the parameter controlmechanism (EES) is proposed for adopting the acceleration coefficients in PSO and the inertia weight

parameters are adopted during the evolutionary process based on the fitness value Revise.

The rest of the paper is organized as follows. Section 2 introduces the preliminaries of association rules,

GA and PSO. Section 3 reviews the literature related to proposed methodology, section 4 reports theexperiment settings, presents experimental results and discussions. Finally Section 5 draws a conclusion.

2.

Preliminaries

This section briefly discusses about association rules and their related factors. The basic features of

multiobjective optimization, Genetic Algorithm and Particle Swarm Optimization are also discussed.

2.1 Association Rule

Association rules are a class of important regularities in data. Association rule mining is commonly stated

as [4]: Let I = {i1,i2, , in} be a set of n binary attributes called items. Let D = {t 1, t2, , tm} be a set oftransactions called the database. Each transaction in D has a unique transaction ID and contains a subset

of the items in I. A rule is defined as an implication of the form X Y where X, Y I and X Y = .The sets of itemsets X and Y are called antecedent (left-hand-side or LHS) and consequent (right-hand-side or RHS) of the rule. Often rules are restricted to only a single item in the consequent.

There are two basic measures for association rules, support and confidence. Support of an association rule

is defined as the percentage/fraction of records that contain X Y to the total number of records in thedatabase. The count for each item is increased by one every time the item is encountered in differenttransaction T in database D during the scanning process. Support is calculated using the following

equation

8/10/2019 sa_paper

3/18

3

(1)Confidence of an association rule is defined as the percentage/fraction of the number of transactions that

contain XY to the total number of records that contain X, where if the percentage exceeds the thresholdof confidence an interesting association rule XY can be generated.

(2)Confidence is a measure of strength of the association rule

2.2Multiobjective Optimization

A general minimization problem of M objectives can be mathematically stated as: Given = [x1,x2, . . . , xd], where d is the dimension of the decision variable space,

Minimize :

[

], subject to :

gj() 0, j= 1, 2, . . . , J, andhk() = 0, k = 1, 2, . . .,K, where is the ithobjective function, gj() is the jth inequalityconstraint, and hk() is the kthequality constraint.

A solution is said to dominate another solution if it is not worse than that solution in all the objectives andis strictly better than that in at least one objective. The solutions over the entire solution space that are notdominated by any other solution are called Pareto-optimal solutions.

Association rule mining using GA and PSO is treated as a multiobjective problem where the objectivesare

Predictive Accuracy

Laplace Conviction

Leverage

Predictive Accuracy measures the effectiveness of the rules mined. The mined rules must have highpredictive accuracy.

(3)where |X&Y| is the number of records that satisfy both the antecedent X and consequent Y, |X| is thenumber of rules satisfying the antecedent X.

Laplace is a confidence estimator that takes support into account, becoming more pessimistic as the

support of X decreases[34,35].It ranges within [0, 1] and is defined as

(4)Convictionis sensitive to rule direction and attempts to measure the degree of implication of a rule[33].It

ranges within[0.5,]. Values far from 1 indicate interesting rules.

8/10/2019 sa_paper

4/18

4

(5)Leverage also known as Piatetski-Shapiro measure [36], measures how much more counting is obtainedfrom the co-occurrence of the antecedent and consequent from the expected, i.e., from independence. Itranges within [0.25, 0.25] and is defined as

(6)2.3

Genetic Algorithm

Genetic algorithm (GA) is a search method for solving optimization problems and modeling evolutionary

systems. A typical GA works with a population of individuals, each being a solution of the optimizationproblem with an associated fitness value. Successive generations of the population are produced based on

the principle of survival of the fittest. The evolution process is achieved through genetic operations, suchas crossover and mutation, on the population of individuals. The steps in Genetic Algorithm are as given

below.

Genetic algorithm ( ){Initialize population randomly;Evaluate fitness of each individual in the population;While stopping condition not achieved

{Perform selection;

Perform crossover and mutation;Evaluate fitness of each individual in the population;

}

}

In traditional GA, the three basic operators namely, selection, crossover and mutation operators are fixedapriori. The speed and success of the GA depends greatly on the correct mix of GA operators and the

probabilities of crossover and mutation operations. The optimum parameters for these operators depend

on problem on which the GA is applied and also on the fitness of the current population.

2.4

Particle Swarm Optimization

Swarm Intelligence (SI) [9] is an innovative distributed intelligent paradigm for solving optimization

problems that originally took its inspiration from the biological examples by swarming, flocking andherding phenomena in vertebrates. Particle Swarm Optimization (PSO)[8] incorporates swarming

behaviors observed in flocks of birds, schools of fish, or swarms of bees, and even human social behavior.

PSO is initialized with a group of random particles (solutions) and then searches for optima by updating

generations. During all iterations, each particle is updated by following the two best values. The firstone is the best solution (fitness) it has achieved so far. The fitness value is also stored. This value is calledpBest The other best value that is tracked by the particle swarm optimizer is the best value obtainedso far by any particle in the population. This best value is a global best and is called gBest After findingthe two best values, each particle updates its corresponding velocity and position.

8/10/2019 sa_paper

5/18

5

Each particle p, at some iteration t, has a position x, and a displacement velocity v. The particles best(pBest) and global best (gBest) position are stored in the associated memory. The velocity and position

are updated using equations 7 and 8 respectively. (7)

(8)

Where is the inertia weightvi is the particle velocity of the i

thparticle

xi is the ith, or current, particle

i is the particles number

d is the dimension of searching spacerand ( ) is a random number in (0, 1)

c1 is the individual factorc2 is the societal factor

pBest is the particle bestgBest is the global best

Inertia weight controls the impact of the velocity history into the new velocity and balances global andlocal searches. Suitable fine-tuning of cognitive and social parameters c1 and c2 result in fasterconvergence of the algorithm and alleviate the risk of settling in one of the local minima .

The Pseudo code for PSO is given below.

For each particle

{Initialize particle

}

END

Do{

For each particle{

Calculate fitness value

If the fitness value is better than the best fitness value (pBest) in historyset current value as the new pBest

}End

Choose the particle with the best fitness value of all the particles as the gBestFor each particle

{Calculate particle velocity according to equation (7)

Update particle position according equation (8)}

End}While maximum iterations or minimum error criteria is not attained

8/10/2019 sa_paper

6/18

6

3.

Literature review

There are many methods to control the parameter setting during an EA run. Researchers have proposeddifferent self-adaptive approaches in EA to make the parameters evolve by themselves . Back [23] embeds

the mutation rate into chromosomes in GA to observe the performance of the self-adaptive approach in

different functions. Spears [24] adds an extra bit in the schematic of chromosome to investigate therelative probability of operating two-point crossover and uniform crossover in the GA. Hop andTabucanon [25] presented a new and original approach to solve the lot size problem using an adaptivegenetic algorithm with an automatic self-adjustment of three operator rates namely, the rate of crossover

operation, the rate of mutation operation and the rate of reproduction operation . Adaptive directedmutation operator in real coded genetic algorithm was applied to solve complex function optimization

problems [31]. It enhances the abilities of GAs in searching global optima as well as in speedingconvergence by integrating the local directional search strategy and the adaptive random search strategies.

A hybrid and adaptive fitness function, in which both filter and wrapper approaches was applied forfeature selection via genetic algorithm to generate different subsets for the individual classifierswasproposed in [32]. revise

cf of particle swarm optimization the evolution direction of each particle is redirected dynamically byadjusting the two sensitive parameters i.e. acceleration coefficients of PSO in the evolution process [30].Several strategies [26,27,28] have been proposed to tune the contribution of various sub-algorithmsadaptively according to their previous performance, and have shown pretty good effect.

4.

Methodology

Any heuristic search can be characterized by two concepts, exploitation and exploration. These conceptsare often mutually conflicting in the sense that if exploitation of a search is increased, then exploration

decreases and vice versa. The manipulation of control parameters in GA and PSO balances the aboveproblem. This paper proposes adaptive parameter control for both Genetic Algorithm and Particle SwarmOptimization.

4.1 Adaptive Genetic Algorithm (AGA)

Genetic algorithms when applied for mining association rules perform global search and copes better withattribute interaction. With roulette wheel selection the parents for crossover and mutations are selected

based on their fitness values, i.e. if a candidate is having high fitness value then the chances of selection ishigh. The crossover and mutation operations are manifested by the crossover and mutation rate usually

defined by the user. The efficiency of the rules mined by genetic algorithm mainly depends on themutation rate, while the crossover rate affects the convergence rate [ add citation]. Therefore tuning thevalue for the mutation rate becomes an important criterion for mining association rules with GA. Higher

mutation rate results in generation of chromosomes much deviated from original values thereby resultingin higher exploration time. Lower mutation rate results in cluster of reproduced chromosomes crowded

towards the global optima region thereby limiting the search space. Hence setting the mutation rateadaptively results to a better solution where the change from generation to generation based on the

feedback and the fitness value is the solution.

The algorithm for adaptive genetic algorithm for mining association rules is as given below{

Initialize population randomly;Evaluate fitness of each individual in the population;

While the stopping condition is not achieved

8/10/2019 sa_paper

7/18

7

{Perform selection;

Perform crossover and mutation;Evaluate fitness of each individual;Change mutation operator.

}

}

The mutation operator is made adaptive as given in equation below.

(8)is the (n+1)thgeneration mutation rate. The first generation mutation rate is , fi(m) is the fitnessof the n

thindividual itemset i. fmax

(n+1)is the highest fitness of the (n+1)

thindividual stocks. fi

nis the fitness

of the nthindividual i. m is the number of item sets. is the adjustment factor, which is set within range[0,1].

4.2 Adaptive Particle Swarm Optimization (APSO)

A swarm[17] consists of a set of an integer number, M, of particles, xi, moving within the search space,

SRd, each representing a potential solution of the problem asFind; subject to appropriate constraints;where F is the fitness function associated with the problem, which we consider to be a minimization

problem without loss of generality.

PSO is mainly conducted by three key parameters important for the speed, convergence and efficiency ofthe algorithm [10]: the inertia weight and two positive acceleration coefficients(c1 and c2). Inertiaweight controls the impact of the velocity history into the new velocity. Acceleration parameters aretypically two positive constants, called the cognitive and social parameters, respectively.

The role of the inertia weight is considered critical for the PSO algorithms convergence behavior. As itbalances global and local searches, it has been suggested to have it decrease linearly with time, usually in

away to first emphasize global search. Suitable fine-tuning of cognitive and social parameters c1and c2may result in faster convergence of the algorithm and alleviate the risk of settling in one of the localminima. The pseudo code for the Adaptive PSO is given below

/* Ns: size of the swarm, C: maximum number of iterations, Of: the final output */

1) t = 0, randomly initialize S0,

Initialize xi, i, i {1, . . .,Ns} /* xi : the ithparticle */ Initialize vi, i, i {1, . . .,Ns} /* vi: the velocity of the ithparticle*/ Pbi xi,i, i {1, . . .,Ns} /* Pbi: the personal best of the ithparticle */ Gb xi /* Gb : the global best particle */

2) for t = 1 to t = C, for i = 1 to i = Ns

8/10/2019 sa_paper

8/18

8

f(xi) = conf(xi ) log (sup (xi) (length(x) + 1) /* f(xi) : Fitness of xi */If ( f(xi) < f(Pbi)

Pbi xi /* Update particle best */Gb min(Pb1, Pb2, , PbN) /* Update global best */

adjust parameters(i, c1i, c2i) /* Adaptive Adjustment */vi(t) =

ivi(t-1)+ c1

ir1(Pbi xi) + c2ir2(Gb xi ) /* Velocity Updation */

xi(t)= xi(t-1) + vi /* Position Updation */ At non dominated(StAt) /* Updating the Archive*/

3) Of Atand stop /* Of: Output*/

adjust parameters(i, c1i, c2i) in the above pseudo code is achieved through adaptive mechanism isproposed here. The proposed approach distributes the evolution into four states based on evolutionarystate estimation: Convergence, Exploration, Exploitation and Jumping out.

Estimation of Evolutionary State (EES)

Based on the search behaviors and the population distribution characteristics of the PSO the EES is doneas follows

1. The distance between particles is calculated using the Euclidean distance measure for eachparticle i using the equation

(9)where N is the population size, xi and xjare the i

than jthparticle in the population respectively.

2. Calculate the evolutionary state estimator e, defined as

(10)Where is the distance measure of the gBest particle, , are the maximum andminimum distance measures respectively from step1.

3. Record the evolutionary e factor for 100 generations for each dataset individually

4. Classify the estimator e into the states: Exploration, Exploitation, Convergence, Jumping out for

the datasets based on the evolutionary states through fuzzy classification techniques.

5. The intervals arrived through fuzzification is as shown in table below

States/Datasets Lenses

Car

Evaluation

Habermans

Survival

Postoperative

Patient Zoo

Convergence 0.0, .03 0.0, 0.15 0.0, 0.4 0, 0.5 0.0, 0.15

Exploitation 0.1, 0.4 0.15, 0.25 0.3, 0.7 0.2, 0.6 0.1, 0.3

Exploration 0.2. 0.7 0.1, 0.3 0.6, 0.9 0.4, 0.8 0.2, 0.4

Jumping out 0.6, 1 0.3, 1 0.8, 1 0.7, 1 0.3, 1

8/10/2019 sa_paper

9/18

9

The change of state reflected as per the PSO sequence is Convergence Exploitation Jumping Out ConvergenceAdaptive Control of Acceleration Coefficients

The acceleration coefficients are made adaptive through the estimation of evolutionary states. Parameter

c1 represents the self-cognition that pulls the particle to its own historical best position, helping inexploration of local niches and maintaining the diversity of the swarm. Parameter c2represents the socialinfluence that pushes the swarm to converge to the current globally best region, helping with fast

convergence. Both the coefficients are initialized to 2 at the start of evolution.

The strategy to adopt for the four states is as given in table.

Control Strategies of c1and c2

State/Acceleration

Coefficient

c1 c2

Exploration Increase Decrease

Exploitation Increase Slightly Decrease SlightlyConvergence Increase Slightly Increase Slightly

Jumping out Decrease Increase

Exploration: During exploration particles should be allowed to explore as many optimal regions aspossible. This avoids crowding over single optima, probably the local optima and explores the targetthoroughly. Increase in the value of c1and decrease in c2values facilitate this process.

Exploitation: In this state based on the historical best positions of each particle they group towards thosepoints. The local information of the particles aids this process. A slight increase in c1 advances the searcharound particle best (pBest) positions. At the same time the slight decrease in c2avoids the deception of

local optima as the final global position has yet to be explored.

Convergence: In this state the swarm identifies the global optima. All the other particles in the swarmshould be lead towards the global optima region. The slight increase in the value of c2helps this process.To fasten up the process of convergence a slight increase in the value of c1 is adopted.

Jumping Out:The global best (gBest) particle moves away from the local optima towards global optima,

taking it away from the crowding cluster. Once any particle in the swarm reaches this region then allparticles are to follow the same pattern rapidly. A large c2along with a relatively small c1value helps toobtain this goal.

The adjustments on the acceleration coefficients should be minimal. Hence, the maximum increment or

decrement between two generations is bounded by the range of [0.05,0.1]. The sum of the acceleration

coefficients is limited to 4.0, when the sum exceeds this limit then both c1 and c2are normalized based onequation.

(11)Inertia Weight Adaptation

8/10/2019 sa_paper

10/18

10

The inertia weight controls the impact of previous flying experience, which is utilized to keep the balancebetween exploration and exploitation. The particle adjusts its trajectory according to its best experience

and enjoys the information of its neighbors. In addition, the inertia weight is also an importantconvergence factor; the smaller the inertia weight, the faster the convergence of PSO. A linear decrease ininertia weight gradually may swerve the particle from their global optima. Hence a nonlinear adaptationof inertia weight as proposed in the given equation is the solution. The global best particle is derived

based on the fitness value of the particles in the swarm. The proposed methodology for adopting theinertia weight is based on the fitness values exhibited by the particles.

{

( ) (12)

5.

Experimental Results and Discussions

Five datasets from University of California Irvine (UCI) repository [39]: Lenses, Habermans Survival,

Car Evaluation, Postoperative Patient and zoo have been used for the experiment. The results arecompared with the general traditional PSO and GA performance [38]. The experiments were developedusing Java and run in windows environment. The best of the five executions were recorded. The numberof iterations was fixed to hundred. The datasets considered for the experiments is listed in Table 1.

Table 1. Datasets Description

Dataset Attributes Instances Attribute characteristics

Lenses 4 24 Categorical

Car Evaluation 6 1728 Categorical, Integer

Habermans Survival 3 310 Integer

Post-operative Patient Care 8 87 Categorical, IntegerZoo 16 101 Categorical, Binary,

Integer

In adaptive GA methodology mutation rate was made self adaptive, based on the analysis of controlparameters [37], where the mutation operation was found to influence the accuracy of the system whilethe crossover operation affects the convergence rate alone. Hence the crossover rate is kept at fixed value.For the mutation rate in addition to the feedback from earlier mutation rate, the fitness value is also

considered while adaption. This enhances the accuracy of the association rules mined. The predictiveaccuracy achieved using adaptive methods for both GA and traditional GA is given table 2.

Table 2. Accuracy Comparison between GA and Adaptive GA

Dataset Traditional GA Adaptive GA

Accuracy No. of Generations Accuracy No. of Generations

Lenses 75 38 87.5 35

Habermans Survival 52 36 68 28

Car Evaluation 85 29 96 21

8/10/2019 sa_paper

11/18

11

Postoperative Patient 84 63 92 52

Zoo 82 58 91 47

The predictive accuracy of the rules mined by the AGA is improved in comparison with the traditionalGA methodology. The convergence rate has also been reduced due to effectiveness in fixing up of the

search space by adaptation methodology. The mutation rate of the adaptive GA at the final iteration isnoted. This mutation rate is replaced for static mutation rate in traditional GA. The predictive accuracy ofthe association rules mined by this method is compared with the results of original GA and Adaptive GAas plotted in figure 1.

Figure 1. Comparison of AGA with GA and GA with AGA mutation Rate

The adaptive GA methodology gives better performance than GA. The performance of GA with themutation rate obtained at final iteration of AGA is reduced than the traditional GAs performance. The

Adaptive methodology thus has been established to have impact on the superior performance of GA.Accuracy alone is not the objective of mining association rules. The interestingness of the association

rules mine is also measured through Laplace, Conviction and Leverage measures. Table 3 presents theresults of Laplace, Conviction and Leverage measures of the rules mined from for the datasets taken upfor analysis.

Table 3. Multiobjective Measures of AGA

Lenses

Haberman's

Survival

Car

Evaluation

Postoperative

Patient ZooLaplace 0.65201 0.64782 0.652488 0.6528 0.6524

Conviction infinity Infinity Infinity infinity Infinity

Leverage 0.0324 0.0569 0.04356 0.0674 0.0765

The Laplace and Leverage values away from 1 for all the datasets indicates that the rules generated are ofinterest. The conviction value, infinity again signifies the importance of the rules generated.

0

20

40

60

80

100

120

Lenses Habermans

Survival

Car Evaluation Postoperative

Patient

Zoo

PredictiveAccuracy(%)

GA

AGA

GA with AGA

mutation

8/10/2019 sa_paper

12/18

12

The APSO methodology for mining association rules adapts the acceleration coefficients based on theEstimation of evolutionary state. The estimate factor(e) determines one among the four states:

Exploration, Convergence, Exploitation, Jumping out , in which the particle is and adjusts the accelerationcoefficients accordingly. Based on the fitness values the inertia weight is adapted. Then the velocitymodification is based on the state in which the particle lies. This balances between exploration andexploitation thus escaping from premature convergence. The predictive accuracy of the association rules

mined is improved through the evolution process.The adaptation of the acceleration coefficients c1 and c2change of states in estimation of evolution statefor Zoo dataset shown in figure 2.

Figure 2. Adaptation of Acceleration Coefficients through EES

The acceleration coefficients converges at the stage where the predictive accuracy is maximum

(generation 60 from figure 8).

The predictive accuracy for the adaptive PSO methodology over 100 generations is shown in figure 3.

1.5

1.61.7

1.8

1.9

2

2.1

2.2

2.3

2.4

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61

Generation Number

C1

C2

84

86

88

90

92

94

96

98

100

Car Haberman Lens Postop Zoo

Pre

dictiv

eAccuracy

(%)

10

20

30

40

50

60

70

80

90

100

8/10/2019 sa_paper

13/18

13

Figure3. Predictive Accuracy of Adaptive PSO over hundred Generations

The adaptive PSOs performance on Car Evaluation dataset and Zoo data set is consistent as seen fromthe figure above. The performance on Habermans survival dataset and Post operative patient dataset ismaximum at the end of evolution. This avoids premature convergence at initial stages. The performanceof Lenses dataset where the dataset size is small is standard.

The Laplace, Conviction and Leverage measures for the five datasets for the adaptive PSO is recorded asin table 3.

Table 4. Multiobjective Measures of APSO

LensesHaberman's

SurvivalCar

EvaluationPostoperative

PatientZoo

Laplace 0.52941 0.501608 0.502488 0.5028 0.5024

Conviction infinity Infinity Infinity infinity Infinity

Leverage 0.026 0.003039 0.002394 0.0301 0.0249

The Laplace measure when away from 1 indicates that the antecedent values are dependent on theconsequent values and hence the rules generated are of significance. The conviction measure when

infinity for all datasets shows that the rules generated are interesting. The Leverage measure beingfaraway from 1 again insists on the interestingness of the rules generated. The predictive accuracy

achieved form the adaptive PSO methodology is compared with the traditional PSO as in figure 4. TheAdaptive PSO performs better than the traditional PSO for all the five datasets when applied forassociation rule mining.

Figure 4. Predictive Accuracy comparison of Adaptive PSO with PSO

The number of rules generated by APSO with predictive accuracy less than 0.05 percentage of the

maximum predictive accuracy arrived is plotted for mining association rules against the traditional PSOmethodology for the datasets in figure 5.

75

80

85

90

95

100

105

Lenses Car

Evaluation

Habermans

Survival

Po-opert

Care

Zoo

Pre

dictiveAccura

cy

(%)

PSO

APSO

8/10/2019 sa_paper

14/18

14

Figure 5 Comparison of No. of Rules Generated by APSO and PSO

In terms of number of rules generated adaptive PSO performs better for Habermans Survival, Zoo andcar evaluation datasets. The number of rules generated for lens and postoperative patient dataset is

marginally better than PSO. This is the due to the reduced number of the records in the two datasets.

The execution time of APSO for mining association rules from the datasets in comparison with PSO

methodology is given in figure 6. The number of iteration at which the predictive accuracy is at maximumis taken as point at which the execution time is recorded. The execution time of the APSO for mining

association rules is more when compared with PSO. The adaption mechanism of acceleration coefficientsbased on evolution factor, and inertia weight adaptation increases the complexity of the algorithmresulting in higher execution time when compared to PSO.

Figure 6 Comparison of Execution Time of APSO and PSO

When number of instances in the dataset is less then increase in execution time is marginal. Thus the

execution time difference for mining rules from the Lenses, Postoperative Patient and Zoo datasets areminimum for APSO than PSO. For the Habermans survival dataset with moderate number of instancesthere is noticeable increase in execution time, whereas for the car Evaluation dataset the difference in

execution time is large. The consistent performance of the APSO in term of predictive accuracythroughout evolution and the predictive accuracy achieved from APSO for mining association rules

balances for the execution time difference.

0

5

10

15

20

25

30

35

40

Lens Postop Haberman Zoo Car

No

.o

fRu

lesGenerate

d

PSO

APSO

0

2000

4000

6000

8000

10000

12000

14000

Car Haberman Lens Postop Zoo

ExecutionTime(ms

)

PSO

APSO

8/10/2019 sa_paper

15/18

15

The major drawback of the traditional PSO is its premature convergence where the particle fixes some

local optima as the target (global search area), where all particles converge locally. One of the objectivesof Adaptive PSO is to avoid the premature convergence, thereby balancing between exploration andexploitation. The iteration number at which predictive accuracy is high is plotted for both the APSO andPSO methodologies for mining association rules for the datasets used in figure 7.

Figure 7. Comparison of Convergence of APSO and PSO

The adaptive particle Swarm Optimization methodology when applied for mining association rules

generates rules with increase in predictive accuracy, increase in the number of rules generated along withbetter rule set measures. The increase in the execution time of APSO when compared to PSOmethodology in also minimum taking into account the complexity of the methodology and the increase in

accuracy achieved over PSO.

The balance in setting the global search space between exploration and exploitation is also attained whilemining association rules with APSO for all the datasets used. This is noted from the shift in iteration

number of APSO when compared to PSO where highest predictive accuracy is noted. Adopting theacceleration coefficients c1(cognitive factor) and c2(social) through EES methodology makes theadjustments in fixing up the global search space effectively with the help of local optimas. The fitnessvalue of the particles is used in adapting inertia weight. This helps in better balance between explorationand exploitation when particles fly through the search space.

6.

Conclusion

Genetic algorithm and particle Swarm Optimization are fast and efficient search methods. Both GA andPSO are problem dependent and setting of the control parameters is a trivial task. This paper proposes

adaptive methods of parameter control for association rule mining using GA and PSO. The fitness valueplays a vital role in passing data over generations and adaptation mainly depends on the fitness values ofthe chromosomes or particles. In Adaptive GA the mutation operator was made self adaptive based on

feedback of previous generations and fitness function. The crossover rate was kept constant. In AdaptivePSO the acceleration coefficients were adapted trough estimation of evolutionary state. In each generation

particles were set into one of the four evolutionary states and based on the states the accelerationcoefficients were altered. The inertia weight parameter adaptation relied on the fitness values andmaximum and minimum values set for inertia weight.

0

20

40

60

80

100

120

Lenses Zoo Car

Evaluation

Habermans

Survival

Postop

IterationNumber

PSO

APSO

8/10/2019 sa_paper

16/18

16

When tested on five datasets from UCI repository both the AGA an APSO generated association rules

with better predictive accuracy and good rule set measures. The execution time of both the methods formining association rules is found to be optimum taking into account the accuracy achieved.

References

1. P. Deepa Shenoy, K.G. Srinivasa, K.R. Venugopal, Lalit M. Patnaik, Evolutionary approach formining association rules on dynamic databases, in: Proc. of PAKDD, LNAI, 2637, Springer-Verlag, 2003, pp. 325336.

2. Jacinto Mata Vzquez, Jos Luis lvarez Macas, Jos Cristbal Riquelme Santos, Discovering Numericassociation rules via evolutionaryalgorithm, PAKDD, 2002, pp. 4051.

3. Kuo. R. J, Chao C.M, Chiu Y. T (2011) , Application of Particle Swarm Optimization toAssociation rule Mining , Journal of Applied soft computing, vol 11, Issue 1, Jan 2011

4. Agrawal R, Imielinski T, Swami A. Mining association rules between sets of items in largedatabases. In: Buneman P, Jajodia S, eds. SIGMOD 93: Proceedings of the 1993 ACM SIGMOD

International Conference on Management of Data. New York: ACM Press; 1993, 207216.5. R.Agrawal and R.Srikant, Fast Algorithms for Mining Association Rules in Large Databases,

proceedings of the 20th International Conference on Very Large Data Bases(VLDB94), MorganKaufmann, 1994, pp 478-499.

6. J. Han, J. Pei, and Y. Yin. Mining frequent patterns without candidate generation, In Proc.

SIGMOD00, Dallas, TX, May 2000, pp 1-12.7. J.Han and M.Kamber, Data mining: Concepts and Techniques, Morgan Kaufmann, 2001.8. J. Kennedy, R.C. Eberhart, Particle swarm optimization, Proceeding of the IEEE International

Conference on Neural Networks 4 (1995) 19421948.9. J. Kennedy, R.C. Eberhart, Swarm Intelligence, Morgan Kaufman, 2001.

10. Yao, X., Liu, Y., Lin, G., 1999. Evolutionary programming made faster. IEEE Transactions on

evolutionary Computation 3 (2), 82102.

11. Ueno, G., Yasuda, K., Iwasaki, N., 2005. Robust adaptive particle swarm optimization. IEEE

International Conference on Systems, Man and Cybernetics 4, 39153920.12. S. Mand P. L. M, Adaptive probabilities of crossover and mutation in genetic algorithms,

System, Man and Cybernetics, Vol. 24, No. 4, 1994,pp:656-667.13. Eberhart, R., Kennedy, J., 1995. A new optimizer using particle swarm theory. In: Proceedings of

the Sixth International Symposium on Micro Machine and Human Science, Piscataway, NJ, IEEE

Service Center, pp. 3943

14. Arumugam, M.S., Rao, M.V.C., 2008. On the improved performances of the particle swarm

optimization algorithms with adaptive parameters, cross-over opera- tors and root mean square

(RMS) variants for computing optimal control of a class of hybrid systems. Applied Soft

Computing 8 (1), 324336.

15. Luo, Q., Yi, D., 2007. A co-evolving framework for robust particle swarm optimization. Applied

Mathematics and Computation, Applied Mathematics and Computation 199 (2), 611622.16. Ratnaweera, A., Halgamuge, S.K., Watson, H.C., 2004. Self-organizing hierarchical particle

swarm optimizer with time-varying acceleration coefficients. IEEE Transactions on Evolutionary

Computation 8 (3), 240255.

17. A.P. Engelbrecht, Fundamentals of Computational Swarm Intelligence, John Wiley & Sons,

2006.

18. Y. Shi, R.C. Eberhart, Parameter selection in particle swarm optimization, in: Proceeding of the7th Conference Evoluationary Programming, New York,1998, pp. 591600.

8/10/2019 sa_paper

17/18

17

19. K.E. Parsopoulos, M.N. Vrahatis, UPSOA unified particle swarm optimization scheme,Proceeding in Lecture Series on Computational Sciences (2004)868873.

20. K.E. Parsopoulos, M.N. Vrahatis, Parameter selection and adaptation in unified particle swarmoptimization, Mathematical and Computer Modelling 46 (2007) 198213

21. Yau-Tarng Juang, Shen-Lung Tung, Hung-Chih Chiu, Adaptive fuzzy particle swarmoptimization for global optimization of multimodal functions, Information Sciences 181 (2011)

4539454922. Shing Wa Leung, Shiu Yin Yuen, Chi Kin Chow, Parameter control system of evolutionary

algorithm that is aided by the entire search history, Applied Soft Computing 12 (2012) 30633078

23. T. Back, Self-adaptation in genetic algorithms, in: Proc. of the 1st European Conf. on ArtificialLife, 1992, pp. 263271.

24. W.M. Spears, Adapting crossover in evolutionary algorithms, in: J. McDonnel, R. Reynolds, D.Fogel (Eds.), Evolutionary Programming IV: Proc. of the Fourth Annual Conf. on Evolutionary

Programming, MIT Press, Cambridge, MA, 1995, pp. 367384.25. Hop, N.V., Tabucanon, M.T., Adaptive genetic algorithm for lot- sizing problem with self-

adjustment operation rate. International Journal of Production Economics 98 (2)2005, 129135.

26. E.K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Ozcan, R. Qu, A survey of hyper-heuristics,Tech. Rep. NOTTCS-TR-SUB-0906241418-2747, School of Computer Science and InformationTechnology, University of Nottingham, 2009.

27. A.K. Qin, V.L. Huang, P.N. Suganthan, Differential evolution algorithm with strategy adaptationfor global numerical optimization, IEEE Transactions on Evolutionary Computation 13 (2) (2009)398417.

28. Jasper A. Vrugt, Bruce A. Robinson, James M. Hyman, Self-adaptive multimethod search forglobal optimization in real-parameter spaces, IEEE Transactions on Evolutionary Computation 13

(2) (2009) 243259.29. Amita Mahor, Saroj Rangnekar, Short term generation scheduling of cascaded hydro electric

system using novel self adaptive inertia weight PSO, International Journal of Electrical Power &Energy Systems, Volume 34, Issue 1, 2012, Pages 1-9

30. Ying Wang, Jianzhong Zhou, Chao Zhou, Yongqiang Wang, Hui Qin, Youlin Lu, An improved

self-adaptive PSO technique for short-term hydrothermal scheduling, Expert Systems withApplications, Volume 39, Issue 3,2012, Pages 2288-2295

31. Ping-Hung Tang, Ming-Hseng Tseng, Adaptive directed mutation for real-coded geneticalgorithms, Applied Soft Computing, In Press, Uncorrected Proof, Available online 23 August2012

32. Canuto, A.M.P ,A genetic-based approach to features selection for ensembles using a hybrid andadaptive fitness function , The 2012 International Joint Conference on Neural Networks (IJCNN),

2012, pp.1-833. S. Brin, R. Motwani, J. D. Ullman, and S. Tsur. Dynamic itemset counting and implication rules

for market basket data. In J. Peckham, editor, Proceedings of the 1997 ACM SIGMODInternational Conference on Management of Data, pages255264, Tucson, Arizona, 1315 June1997.

34. I.J. Good, The Estimation of Probabilities: An Essay on Modern Bayesian Methods, MIT Press,1965.

35. [16] P. Clark and R. Boswell, Rule Induction with cn2: Some recent improvements,In Proc. ofthe European Working Session on Learning,pp. 151-163, 1991.

36. G. Piatetsky-Shapiro. Discovery, analysis, and presentation of strong rules. In Knowledge

Discovery in Databases, pages 229248. AAAI/MIT Press, 1991.

8/10/2019 sa_paper

18/18

18

37. K. Indira ,S. Kanmani, Association Rule Mining Using Genetic Algorithm: The role ofEstimation Parameters, International conference on advances in computing and communications,

Communications in Computer and Information Science, 2011, Volume 190, Part 8, 639-64838. K.Indira1 et al. Population Based Search Methods in Mining Association Rules,

Third International Conference on Advances in Communication, Network, and Computing CNC 2012, LNICST pp. 255261, 2012.

39. C.J. Merz, P. Murphy, UCI Repository of Machine Learning Databases, 1996(http://www.cs.uci.edu/~mlearn/ MLRepository.html).

http://www.springerlink.com/content/?Author=K.+Indirahttp://www.springerlink.com/content/?Author=S.+Kanmanihttp://www.springerlink.com/content/?Author=S.+Kanmanihttp://www.springerlink.com/content/?Author=K.+Indira