Modeling selection pressure in XCS for proportionate and tournament selection

Modeling Selection Pressure in XCS for Proportionate and

Tournament SelectionAlbert Orriols-Puig1,2 Kumara Sastry2

Pier Luca Lanzi1,3 David E. Goldberg2

Ester Bernadó-Mansilla1

1Research Group in Intelligent SystemsEnginyeria i Arquitectura La Salle, Ramon Llull University

2Illinois Genetic Algorithms LaboratoryDepartment of Industrial and Enterprise Systems Engineering

University of Illinois at Urbana Champaign

3Dipartamento di Elettronica e InformazionePolitecnico di Milano

Illinois Genetic Algorithms Laboratory and Group of Research in Intelligent Systems Slide 2GECCO’07

Motivation

Facetwise modeling to permit a successful understanding of complex systems (Goldberg, 2002)

Analysis of selection schemes in XCS: Proportionate vs. Tournament

– Tournament selection is more robust to parameter settings and noise than proportionate selection (Butz, Sastry & Goldberg, 03, 05)

– Proportionate selection is, at least, as robust as tournament selection if the appropriate fitness separation is used (Karbat, Bull & Odeh, 05)

In GA, these schemes were studied through the analysis of takeover time(Goldberg & Deb, 90; Goldberg, 02)

Aim: Model selection pressure in XCS through the analysis of the takeover time


Outline

1. Description of XCS

2. Modeling takeover time

3. Comparing the two models

4. Experimental validation

5. Modeling generality

6. Conclusions


Description of XCS

Representation:– fixed-size rule-based representation

– Rule Parameters: Pk, εk, Fk, nk

Learning interaction:– At each learning iteration Sample a new example

– Create the match set [M]

– Each classifier in [M] votes in the prediction array

– Select randomly an action and create the action set [A]

1. Description of XCS2. Modeling Takeover Time3. Comparing the two Models4. Experimental Validation5. Modeling Generality6. Conclusions


Description of XCS

Rule evaluation: reinforcement learning techniques.– Prediction:

– Prediction error:

– Accuracy of the prediction:



Description of XCS

Rule evaluation: reinforcement learning techniques.– Relative accuracy:

– Fitness computed as a windowed average of the accuracy:



Description of XCS

Rule discovering:– Steady-state, niched GA

– Population-wide deletion

Proportionate selection• Probability proportionate to rule’s fitness.

Tournament selection• Selects τ percent of classifiers from [A]

• Selects the classifier with higher microclassifier fitness



Outline






6. Conclusions


Modeling Takeover Time

In GA:– Usually, the fitness of an individual is constant

– Selection and replacement are performed over the whole population

In XCS:– Fitness depends on the other rule’s fitness in the same niche

– Selection is niched-based, whilst deletion is population-wide



Modeling Takeover Time

Assumptions in our model– XCS has evolved a set of non-overlapping niches

– Simplified scenario: niche with two classifiers cl1 and cl2.

– cl1 is the best rule in the niche: k1 > k2

– Classifier clk has: • prediction error εk

• fitness Fk

• numerosity nk

• microclassifier fitness fk

– cl1 and cl2 are equally general Same reproduction opportunities

– We assume niched deletion



Proportionate Selection

Fitness is an average of cl1 and cl2 respective accuracies

Then, the probability of selecting the best classifier cl1 is:

Probability of deletion: Pdel(clj) = nj/n where n=n1+n2

Num. ratio: nr = n2/n1Accuracy ratio: ρ=k2/k1




Evolution of cl1 numerosity

1. The numerosity of cl1 increases if cl1 is selected by the GA and another classifier is selected to be deleted

2. The numerosity of cl1 decreases if cl1 is not selected by the GA but it is selected by the deletion operator

3. The numerosity of c1 remain de same otherwise.




Grouping the above equations we obtain

Rewritten in terms of proportion of classifiers cl1 in the niche:

Considering Pt+1 – Pt ≈ dp/dt

Pt = n1/n


Integrate:Initial proportion: P0Final proportion: P



This gives us that the takeover time for proportionateselection is guided by the following expression:

P0: initial proportion of cl1 in the niche

P: final proportion of cl1 in the niche

ρ: accuracy ratio between cl2 and cl1n: niche size

If ρ 1: trws ≈ ∞

If ρ 0:

A higher separation between fitness enables a higher ability in identifying accuraterules, as announced by Karbat, Bull & Odeh, 2005.

n



Tournament Selection

Assumptions– Fixed tournament size s

– cl1 is the best classifier in the niche: f1 > f2 , that is, F1/n1 > F2 /n2




This gives us that the takeover time for tournamentselection is guided by the following expression:



s: tournament size

n: niche size

Tournament selection does not depend on the accuracy ratio between the best classifier and the others in the same [A], as pointed by Butz, Sastry & Goldberg, 2005

As s increases, this expression decreases

It does not depend on the individual fitness

For large s:



Outline






6. Conclusions


Proportionate vs. Tournament

Values of s and ρ for which both schemes result in the same takeover time. Require: t*RWS = t*TS

– We obtain

For P0 = 0.01and P = 0.99



Outline






6. Conclusions


Design of Test Problems

Single-niche problem– One niche with 2 classifiers:

• Highly accurate classifier cl1:

• Less accurate classifier cl2:

• Varying ρ we are changing the fitness separation between cl1 and cl2

– Population initialized with• N · P0 copies of cl1

• N · (1 – P0) copies of cl2



Results on the Single-Niche Problem


Accuracy ratio: ρ= 0.01

RWS

Tournament s=9

Tournament s=3

Tournament s=2





RWS

Tournament s=9

Tournament s=3

Tournament s=2





RWS

Tournament s=9

Tournament s=3

Tournament s=2


Design of Test Problems

Multiple-niche problem– Several niches with 1 maximally accurate classifier each niche.

– One over-general classifier that participates in all niches

– The population contains:• N · P0 copies of maximally accurate classifiers

• N · (1 – P0) copies of the overgeneral classifier

– The problem violates two assumptions of the model• Overlapping niches

• The size of the different niches differ from the population size

– Deletion can select any classifier in [P]



Results on the Multiple-Niche Problem

For small ρ the theory slightly underestimates the empirical takeover time

The model of proportionate selection is accurate in general scenarios if the ratio of accuracies is small

In situations where there is a small proportion of the best classifier in one niche competing with other slightly inaccurate and overgeneral, the overgeneral may take over the population.

Further experiments, show that for ρ >= 0.5, the best classifiers is removed from the population


RWS ρ = 0.01RWS ρ = 0.20

RWS ρ = 0.30RWS ρ = 0.40

RWS ρ = 0.50


Results on the Multiple-Niche Problem

For high s the theory slightly underestimates the empirical takeover time

The model of tournament selection is accurate in general scenarios if the tournament size is high enough

Only in the extreme case (s=2), the experiments strongly disagree with the theory


TS s = 2TS s = 3

TS s = 9


Outline






6. Conclusions


Modeling Generality

Scenario– The best classifier cl1 appears in the niche with probability 1

– cl2 appears in the niche with probability ρm




The takeover time for proportionate selection is guided by the following expression:



ρ: accuracy ratio between cl2 and cl1ρm: occurrence probability of cl2N: niche size

If cl1 is either more accurate or more general than cl2, cl1 will take over the population.




The takeover time for tournament selection is guided by the following expression:



s: tournament size

ρm: occurrence probability of cl2

n: niche size

For low ρm or high s the right-hand logarithm goes to zero, so that the takeover timemainly depends on P0 and P



Results of the Extended Model onthe one-niched Problem


RW

S

Tour

nam

ent


Outline






6. Conclusions


Conclusions

We derived theoretical models for proportionate and tournament under some assumptions– Models are exact in very simple scenarios

– Models can qualitatively explain both selection schemes in more complicated scenarios

Models support that tournament is more robust (Butz, Sastry & Goldberg, 2005)

Fitness separation is essential to guarantee that the best classifier will take over the population in proportionate selection (Karbhat, Bull & Oates, 2005)– It may fail in domains where there are slightly inaccurate classifiers

(real-world domains).

– Eg: Real-world problems where many slightly inaccurate and more general classifiers may take over accurate classifiers.


Modeling Selection Pressure in XCS for Proportionate and


Albert Orriols-Puig1,2 Kumara Sastry2

Pier Luca Lanzi2 David E. Goldberg2

Ester Bernadó-Mansilla1

1Research Group in Intelligent SystemsEnginyeria i Arquitectura La Salle, Ramon Llull University

2Illinois Genetic Algorithms LaboratoryDepartment of Industrial and Enterprise Systems Engineering

University of Illinois at Urbana Champaign




1. The numerosity of cl1 increases if cl1 participates in the tournament and another classifier is selected to be deleted

2. The numerosity of cl1 decreases if cl1 does not participate in the tournament but it is selected by the deletion operator



Probability that the cl1 participates in the tournament

Probability that the cl1 participates in the tournament



Empirical results perfectly match theory

For ρ=0.01 proportionate selection produces a faster increase

Roulette wheel selection is negatively influenced by the increase of ρ

Coherently to what was empirically shown in (Butz& Sastry, 2003), tournament selection is more robust to variations on the fitness scaling.



Description of XCS

Representation:– fixed-size rule-based representation

– Michigan-style LCS: solution represented by the entire rule-set

– Map of condition-action-predictions.

Rule evaluation: reinforcement learning techniques.– Prediction:

– Prediction error:

– Accuracy of the prediction:

)pβ(Rpp kkk −+=

)- |pRβ(| kkkk εεε −+=



Compare the values of values of s and ρ

for which both schemes have the same initial increase of the proportion of the best classifier

for which both schemes have the same takeover time




Grouping the above equations we obtain

Rewritten in terms of proportion of classifiers cl1 in the niche:

Considering Pt+1 – Pt ≈ dp/dt




Values of s and ρ for which both schemes have the same initial increase of the proportion of the best classifier

– Requiring that

– We obtain

The tournament size s has to increase as ρdecreases

Tournament selection produces a stronger pressure toward the best classifier in scenarios in which slightly inaccurate but initially hihgly numerous classifiers are competing against highly accurate classifiers

For P0 = 0.01and P = 0.99



Modeling Generality

Scenario– The best classifier cl1 appears in the niche with probability 1

– cl2 appears in the niche with probability ρm

Model for proportionate selection

1. The numerosity of cl1 increases if cl1 and cl2 appear in the niche, cl1 is selected by the GA, and cl2 is chosen by deletion, or when cl1 is the only classifier in the niche and another classifier is deleted.

2. The numerosity of cl1 decreases if cl1 and cl2 are in the niche, cl1 is not selected by the GA but is selected by deletion





Which gives us that the takeover time for proportionateselection is guided by the following expression:



ρ: accuracy ratio between cl2 and cl1ρm: occurrence probability of cl2N: niche size

If cl1 is either more accurate or more general than cl2, cl1 will take over the population.





1. The numerosity of cl1 increases if cl1 and cl2 appear in the niche, cl1 is selected by the GA, and cl2 is chosen by deletion, or when cl1 is the only classifier in the niche and another classifier is deleted.

2. The numerosity of cl1 decreases if cl1 and cl2 are in the niche, cl1 is not selected by the GA but is selected by deletion

3. The numerosity of cl1 remain de same otherwise.




Which gives us that the takeover time for tournamentselection is guided by the following expression:



s: tournament size

ρm: occurrence probability of cl2

n: niche size

For low ρm or high s the right-hand logarithm goes to zero, so that the takeover timemainly depends on P0 and P



Aim

Model selection pressure in XCS through the analysis of the takeover time– Consider that XCS has converged to an optimal solution

– Write differential equations that describe the change in proportion of the best individual

– Solve the equations and derive a closed form solution

– Validate the model empirically

Technology

Modeling selection pressure in XCS for proportionate and tournament selection