Department of Informatique and Recherche Opérationnelle ...ferland/Seminars/Local_search/1. LSM.pdf · Department of Informatique and Recherche Opérationnelle Université de Montréal

METAHEURISTIC

Jacques A. FerlandDepartment of Informatique and Recherche Opérationnelle

Université de Montréal

[email protected]

www.iro.umontreal.ca/~ferland/

March 2015

Introduction

• Introduction to some basic methods

• My vision relying on my experience

• Not an exaustive survey

• Most of the time ad hoc adaptations of basic methods are used

to deal with specific applications

(greedy) to determine a

Heuristic

procedure good

solu

for an optimisation protion blem

guided identifying a

sequence of feasible solutions for an

optimisation problem by

Metaheuristic

iterative procedure

combining

different searching con

in order

cepts

to near-optimalfind efficie solutionsntly

Advantages of using metaheuristic

• Intuitive and easy to understand

• With regards to enduser of a real world application:

- Easy to explain

- Connection with the manual approach of enduser

- Enduser sees easily the added features to improve the results

- Allow to analyze more deeply and more scenarios

• Allow dealing with larger size problems having higher degreeof complexity

• Generate rapidly very good solutions

Disadvantages of using metaheuristic

• Quick and dirty methods

• Optimality not guaranted in general

• Few convergence results for special cases

Overview

• Heuristic Constructive Techniques:

Generate a good solution

• Local (Neighborhood) Search Methods (LSM):

Iterative procedure improving a solution

• Population-based Methods:

Population of solutions evolving to mimic a natural process

Preliminary: Problems used to illustrate

• General problem

• Assignment type problem: Assignment of resources j to activities i

( )min x X

f x∈

( )

1

min

s.t. 1 1, ,

0 or 1 1, , ; 1, ,

m

ij

j

ij

f x

x i n

x i n j m

=

= =

= = =

∑ …

… …

Problem Formulation

• Assignment type problem

• Graph coloring problem : Graph G = (V,E).

V = { i : 1≤ i ≤ n} ; E = {(i, l) : (i, l) edge of G}.

Set of colors {j : 1≤ j ≤ m}

( )

1

min

s.t. 1 1, ,

0 or 1 1, , ; 1, ,

m

ij

j

ij

f x

x i n

x i n j m

=

= =

= = =

∑ …

… …

( ), 1

1

min

s.t. 1 1, ,

0 or 1 1, , ; 1, ,

m

ij lj

i l E j

m

ij

j

ij

x x

x i n

x i n j m

∈ =

=

= =

= = =

∑ ∑

∑ …

… …

Graph coloring example

• Graph coloring problem : Graph G = (V,E).

V = { i : 1≤ i ≤ n} ; E = {(i, l) : (i, l) edge of G}.

Set of colors {j : 1≤ j ≤ m}

The following encoding of a solution x is interesting for solving the graph coloring problem:

x =>

where for each vertex i,

( ), 1

1

min

s.t. 1 1, ,

0 or 1 1, , ; 1, ,

m

ij lj

i l E j

m

ij

j

ij

x x

x i n

x i n j m

∈ =

=

= =

= = =

∑ ∑

∑ …

… …

( ) ( ) ( )1 , 2 , ,j j j n …

( )

( )

1

0 for 1, , ;

ij i

ij

x

x j m j j i

=

= = ≠…

Heuristic Constructive Techniques

• Greedy

• GRASP (Greedy Randomized Adaptive Search Procedure)

Heuristic Constructive Techniques

• Values of the variables are determined sequentially:

at each iteration, a variable is selected,

and its value is determined

• The value of each variable is never modifided once it is

determined

• Techniques often used to generate initial solutions for iterative

procedures

Greedy method

Next variable to be fixed and its

value are selected to optimize the

objective function given the

values of the variables already

fixed

Graph coloring problem:

Vertices are ordered in

decreasing order of their degree

Vertices selected in that order

For each vertex, select a color in

order to reduce the number of

pairs of adjacent vertices already

colored with the same color


• Graph with 5 vertices

• 2 colors available: red

blue


• Vertices in decreasing order of degree

• Vertex degree color

2 3

1 2

3 2

4 2

5 1




2 3

1 2

3 2

4 2

5 1

• Vertex 2

color number of adj. vert. same color

red 0

blue 0 <=




2 3 blue

1 2

3 2

4 2

5 1

• Vertex 2


red 0

blue 0 <=




2 3 blue

1 2

3 2

4 2

5 1




2 3 blue

1 2

3 2

4 2

5 1

• Vertex 1


red 0 <=

blue 1




2 3 blue

1 2 red

3 2

4 2

5 1

• Vertex 1


red 0 <=

blue 1




2 3 blue

1 2 red

3 2

4 2

5 1




2 3 blue

1 2 red

3 2

4 2

5 1

• Vertex 3


red 1 <=

blue 1




2 3 blue

1 2 red

3 2 red

4 2

5 1

• Vertex 3


red 1 <=

blue 1




2 3 blue

1 2 red

3 2 red

4 2

5 1




2 3 blue

1 2 red

3 2 red

4 2

5 1

• Vertex 4


red 0 <=

blue 1




2 3 blue

1 2 red

3 2 red

4 2 red

5 1

• Vertex 4


red 0 <=

blue 1




2 3 blue

1 2 red

3 2 red

4 2 red

5 1




2 3 blue

1 2 red

3 2 red

4 2 red

5 1

• Vertex 5


red 1

blue 0 <=




2 3 blue

1 2 red

3 2 red

4 2 red

5 1 blue

• Vertex 5


red 1

blue 0 <=

GRASPGreedy Randomized Adaptive Search Procedure

• Next variable to be fixed is selected randomly among those inducing the

smallest increase.

• Referring to the general problem,

i) let J’ = { j : xj is not fixed yet}

and δj be the increase induces by the best value that xj can take ( j є J’ )

ii) Denote

and α є [0, 1] .

iii) Select randomly j’ є {j є J’ : δj ≤ ( 1 / α ) δ* }

and fix the value of xj’

{ }* minj

j Jδ δ

′∈=


• Graph with 5 vertices

• 2 colors available: red

blue

• α = 0.5


Vertex j best color δj color

1 any 0

2 any 0

3 any 0

4 any 0

5 any 0



1 any 0

2 any 0

3 any 0

4 any 0

5 any 0

J’ = {1, 2, 3, 4, 5}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1, 2, 3, 4, 5}

� Select randomly vertex 3

color blue



1 any 0

2 any 0

3 any 0 blue

4 any 0

5 any 0

J’ = {1, 2, 3, 4, 5}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1, 2, 3, 4, 5}


color blue



1 red 0

2 red 0

3 blue

4 any 0

5 any 0



1 red 0

2 red 0

3 blue

4 any 0

5 any 0

J’ = {1, 2, 4, 5}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1, 2, 4, 5}


color red



1 red 0

2 red 0

3 blue

4 any 0 red

5 any 0

J’ = {1, 2, 4, 5}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1, 2, 4, 5}


color red



1 red 0

2 any 1

3 blue

4 red

5 blue 0



1 red 0

2 any 1

3 blue

4 red

5 blue 0

J’ = {1, 2, 5}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1, 5}


color blue



1 red 0

2 any 1

3 blue

4 red

5 blue 0 blue

J’ = {1, 2, 5}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1, 5}


color blue



1 red 0

2 any 1

3 blue

4 red

5 blue



1 red 0

2 any 1

3 blue

4 red

5 blue

J’ = {1, 2}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1}

� Select vertex 1

color red



1 red 0 red

2 any 1

3 blue

4 red

5 blue

J’ = {1, 2}

δ* = 0

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {1}

� Select vertex 1

color red



1 red

2 blue 1

3 blue

4 red

5 blue



1 red

2 blue 1

3 blue

4 red

5 blue

J’ = {2}

δ* = 1

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {2}

� Select vertex 2

color blue



1 red

2 blue 1 blue

3 blue

4 red

5 blue

J’ = {2}

δ* = 1

α = 0.5

{j є J’ : δj ≤ ( 1 / α ) δ* }= {2}

� Select vertex 2

color blue



1 red

2 blue

3 blue

4 red

5 blue

J’ = Φ

Local(Neighborhood) Search Methods

(LSM)

• A Local (Neighborhood) Search Method (LSM) is an iterative procedurestarting with an initial feasible solution x0 .

• At each iteration:

- we move from the current solution to a new one in its

neighborhood N(x)

- becomes the current solution for the next iteration

- we update the best solution x* found so far.

• The procedure continues until some stopping criterion is satisfied

x X∈ x X′∈

x′

Local(Neighborhood) Search Methods

(LSM)

• Descent (D)

• Tabu Search (TS)

• Simulated Annealing (SA)

• Improving strategies

- Intensification

- Diversification

• Variable Neighborhood Search (VNS)

• Exchange Procedure (EP)

• Adaptive Large Neighborhood Search (ALNS)

Neighborhood

Neighborhood N(x) :

The neighborhood N(x) varies with the problem, but its elements are always generated by slightly modifying x.

If we denote M the set of modifications (or moves) to generate neighboring solutions, then

N(x) = {x' : x' = x ⊕ m , m ∈ M }

Neighborhood for assignment type problem

The elements of the neighborhood

N(x) are generated by slightly

modifying x:

N(x) = {x' : x' = x ⊕ m , m ∈ M }1, ,i n= …

( ) ( ) ( ) 1 , , , ,j j i j nx ⇒ … …

( )

( )

( ) ( ) ( )

( )( )

1 ,

Let

,

b

, ,

Each solution is obtained by selecting an activity

and modifying

e as follows: for each 1, , :

1

its resource

0 for 1, , ;

or

ij i

ij

j j i

x i n

x

x j m j j i

x j n

x N x i

j i

⇒

′

=

= = ≠

∈

=

… …

…

…

[ ]( )

( )

( ) ( )

i.e., the modification can be denoted , :

0

1

to some o

for all other ,

or

t

1 , , ,

her

,

ij i

ip

ij ij

m i p

x

x

x x i j

x j p j

p

n

=

′ =

′ =

′ =

′ ⇒ … …

Descent Method (D)

• At each iteration, a best solution is selected as the

current solution for the next iteration.

• Stopping criterion:

f(x') ≥ f(x)

i.e., the current solution cannot be improved or

a first local minimum is reached.

( )x N x′∈

Descent Method (D)

( )

0

0

Initialize

Repeat until stopping criterion is sat

Determine ' ( ) such that

Select an

isfie

' arg min{ ( )}

If ( ')

initial solutio

( )

n

d

Let

z N x

x N x

x f z

f x

x

x

x X

x

f

∈

∈

=

=

∈

=

then stopping criterion is satisfied

else '

is the solution generated

x x

x

=

denote the

current solution

x

Descent Method (D)

( )

0

0

Initialize


Determine ' (

Select an initial solution

Let

If ( '

) such that

' arg min

) ( )

{

isfie

(

)}

d

z N x

x N x

x X

x x

f x

z

x

x f

f

∈

∈

=

=

∈

=


else '

is the solution generated

x x

x

=

( )

At each iteration a best solution

is selectedx N x′∈

Descent Method (D)

( )

0

0

Initialize



Let


If ( ') ( )

' arg min{

isfie

( )}

d

z N x

x X

x x

x N x

x f

x x

z

f f

∈

∈

=

∈

≥

=

then s

else

topping criterion is satisfied

is the solution generat

'

ed

x

x

x=

stopping criterion

a first local minimum

is reachedx

Descent Method (D)

( )

0

0

Initialize



Let


' arg min{ ( )}

If ( ') ( )

isfied

z N x

x X

x x

x N x

x f z

f x f x

∈

∈

=

∈

=

≥


is the solution

else

generated

'

x

x x= ( ) becomes the current

solution for the next iteration

x N x′∈

Selecting x'

Selecting x'

0x

Selecting x'

0x

1x

Selecting x'

0x

1x

2x

Knapsack Problem

• Problem formulation:

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to 2x1 + 2x2 + x3 + x4 ≤ 3 (*)

x1 , x2 , x3 , x4 = 0 or 1.

• Neighborhood N(x) specified by the following modification or move:

The value of one and only one variable is modified (from 0 to 1 or

from 1 to 0) to generate a new solution satisfying constraint

2x1 + 2x2 + x3 + x4 ≤ 3 (*)

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

1 2 3 42 2 3x x x x+ + + ≤

Knapsack Problem

• Initial sol. x0 = x = [1, 0, 0, 0] ;

f(x) = 18

• N(x) f

[0, 0, 0, 0] 0

[1, 0, 1, 0] 29

[1, 0, 0, 1] 32

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Initial sol. x0 = x = [1, 0, 0, 0] ;

f(x) = 18

• N(x) f

[0, 0, 0, 0] 0

[1, 0, 1, 0] 29

[1, 0, 0, 1] 32

• x' = [1, 0, 0, 1] ; f(x') = 32

• Since f(x') >f(x), then

x' replaces the current sol. x

and we start a new iteration

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• current sol. x = [1, 0, 0, 1] ;

f(x) = 32

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Current sol. x = [1, 0, 0, 1] ;

f(x) = 32

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

• x' = [1, 0, 0, 0] ; f(x') = 18

• Since f(x') < f(x), then

the procedure stops with

x* = x

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Tabu Search (TS)

• Tabu Search is an iterative neighborhood or local search

technique

• At each iteration we move from a current solution x to a new

solution x' in a neigborhood N(x) of x,

• until we reach some solution x* acceptable according to some

criterion

0

0

Let

0

Initialize

Repeat until stopping criterio

Select an

n satisfie

initial solutio

n

1 ; 1

D

d

etermine a

Let *

su

TL

iter niter

iter iter nite

x X

x

r nit r

x x

e

φ=

= =

= + = +

∈

= =

bset ( ) ( ) of solutions such that

such that the move

or

( ) ( *)


' := argmin{ (z NC

NC x N x

z x m m TL

f z f x

x NC x

x f∈

∈

= ⊕ ∉

<

∈

)}

'

If ( ) ( *) then * , and = 0

If max or max


else update Tabu List

* is

z

x x

f x f x x x niter

iter iter niter niter

TL

x

=

< =

= =

the best solution generated

Tabu Search (TS)

denotes the current solution

* denotes the best solution found so far

x

x

0

0


Let *

Let

0

Initialize

Repeat until stopping criterion satisfied

1 ; 1

Determine a su

x X

x x x

TL

iter niter


φ

∈

= =

=

= =

= + = +

such that

such that the move

or

bset ( ) ( ) of solutions

Determine ' ( ) such th

at

( ) ( *)

' := argmin{ (

z NC

z x m m TL

f z f x

NC x N x

x NC x

x f∈

= ⊕ ∉

<

∈

∈

'

If ( ) ( *) then * , and = 0

If max or max



*

)}

is

x x

f x f x x x niter

iter

z

iter niter niter

TL

x

=

< =

= =


Tabu Search (TS)

( ) ( )

( )

At each iteration determine

a specific subset

a best solution in

NC x N x

x NC x

∈

′

i

i

0

0


Let *

Let

0

Initialize


1 ; 1

Determine a su

x X

x x x

TL

iter niter


φ

∈

= =

=

= =

= + = +


such that the move

or

( ) ( *)


' := argmin{ (z NC

NC x N x

z x m m TL

f z f x

x NC x

x f∈

∈

= ⊕ ∉

<

∈

)}

If ( ) ( *) then * , and = 0

If max or max



* is

'

z

f x f x x x niter


TL

x x

x

< =

= =

=


Tabu Search (TS)

( ) ( )

( )

At each iteration determine

a specific subset

a best solution in

NC x N x

x NC x

∈

′

i

i

( ) becomes the current

solution for the next iteration

x NC x′∈

As long as is better than , the behavior of the

procedure is similar to the descent method

x x′

Otherwise, moving to as the next current

solution induces no improvement or a

deterioration of the objectice function, but it

allows moving out of a local minimum

x′

0

0


Let *

0

Initialize

Repeat until stopping criter

ion satisfie

Le

d

1 ; 1

Determine a su

t

x X

x x x

iter niter

iter iter niter nite

T

r

L φ

∈

= =

= =

= + =

=

+

bset ( ) ( ) of solutions

or

( ) ( *)

Determine

such that

such that the mov

' ( ) such that

' := argm (

n

e

i {z NC

NC x N x

f z f x

x N

z x m m L

x

f

T

C

x∈

= ⊕

∈

<

∉

∈

)}

'

If ( ) ( *) then * , and = 0

If max or max


update Tabu else

*

L

st

i

i

s

z

x x

f x f x x x niter


x

TL

=

< =

= =


Tabu Search (TS)

As long as is better than , the behavior of the

procedure is similar to the descent method

x x′

Otherwise, moving to as the next current

solution induces no improvement or a

deterioration of the objectice function, but it

allows moving out of a local minimum

x′

( )

To prevent cycling, recently visited solutions

are eliminated from using a Tabu listN x

Tabu Lists (TL)

• Short term Tabu list TL is used to remember attributes or characteristics of the modification used to generate the new current solution

• A Tabu List often used for the assignment type problem is the following:

If the new current solution x' is generated from x by modifying the resource of i from j(i) to p, then the pair (i, j(i)) is introduced in the Tabulist TL

If the pair (i, j) is in TL, then any solution where resource j is to be

assigned to i is declared Tabu

• The Tabu list is cyclic in order for an attribute to remain Tabu for a fixed number k of iterations

0

0


Let *

Let

0

Initialize


1 ; 1

Determine a su

x X

x x x

TL

iter niter


φ

∈

= =

=

= =

= + = +

or


such that the move


( ) ( *)

' := argmin{ (z NC

NC x N x

z x m m TL

f z f x

x NC x

x f∈

∈

= ⊕ ∉

<

∈

)}

'

If ( ) ( *) then * , and = 0

If max or max


update Tabu else

*

L

st

i

i

s

z

x x

f x f x x x niter


x

TL

=

< =

= =


Tabu Search (TS)

Since the Tabu list is specified in terms of

the modifications used, we require an

to bypass the tabu status

of good solutions declared Tabu without

being recently vi

sited.

Aspiration criterion

( )

( ) ( )

may include in even if is Tabu

whenever

*

where * is the best solution found so far

z NC x z

f z f x

x

<

0

0


Let *

Let

0

Initialize


1 ; 1

Determine a su

x X

x x x

TL

iter niter


φ

∈

= =

=

= =

= + = +


such that the move

or

( ) ( *)


' := argmin{ (z NC

NC x N x

z x m m TL

f z f x

x NC x

x f∈

∈

= ⊕ ∉

<

∈

)}

'

, and = 0

If max or max

If ( ) (



* is

*) then *

z

x x

niter

iter iter niter

f x f x

niter

T

x x

L

x

=

= =

< =


Tabu Search (TS)

Update * the best solution found so farx

No monotonicity of the objective function!!!!

Stopping criteria ???

0

0

0

Initialize

Repeat u

Select

ntil stopping

an initial solution

criterion satisfied

Let *

Let

1 ;

Determine s

1

a

u

iter niter

iter iter niter

x X

nite

TL

r

x x x

φ

= =

= + = +

∈

= =

=


such that the move

or

( ) ( *)


' := argmin{ (z NC

NC x N x

z x m m TL

f z f x

x NC x

x f∈

∈

= ⊕ ∉

<

∈

and = 0

If ma

)}

'

If ( ) ( *) then * ,


else update Tabu Lis

x or

t

*

ma

i

x

s

niter

iter iter niter

z

x x

f x f x

nite

x x

r

TL

x

= =

=

< =


Tabu Search (TS)

No monotonicity of the objective function!!!!

Stopping criteria ???

( )

Stopping criteria:

Maximum number of iteration

Maximum number of successive

iterations where * does not improvef x

i

i

Selecting x'

Selecting x'

0x

Selecting x'

0x

1x

Selecting x'

0x

1x

2x

Selecting x'

0x

1x

2x

3x

Selecting x'

0x

1x

2x

3x 4

x

Selecting x'

0x

1x 2

x 3x

4x 5

x6

x

7x

Knapsack Problem


max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


The value of one and only one variable ismodified (from 0 to 1 or from 1 to 0)

to generate a new solution satisfyingconstraint

• Tabu list TL :

element introduced in the Tabu list is the pair

(index of modified variable,

current value of the modified

variable)

• Length of Tabu list = 2

• itermax = 5

nitermax = 4

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

1 2 3 42 2 3x x x x+ + + ≤

Knapsack Problem

• TL = Φ

• iter = 1

niter = 1

• Initial sol. x0 = x = [1, 0, 0, 0] ;

f(x) = 18

• N(x) f

[0, 0, 0, 0] 0

[1, 0, 1, 0] 29

[1, 0, 0, 1] 32

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Initial sol. x0 = x = x* = [1, 0, 0, 0] ;

f(x) = 18

• N(x) f

[0, 0, 0, 0] 0

[1, 0, 1, 0] 29

[1, 0, 0, 1] 32

• x' = [1, 0, 0, 1] ; f(x') = 32

• Since f(x') > f(x*) = 18, then

x* = x' , and niter := 0

• x' replaces the current sol. x and we

start a new iteration

• TL = { (4, 0)}

• Tabu list = Φ

• iter = 1

niter = 1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Current sol. x = [1, 0, 0, 1] ;

f(x) = 32

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18 <= tabu

• TL = { (4, 0)}

• iter = 2

niter = 1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Current sol. x = [1, 0, 0, 1] ;

f(x) = 32

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18 <= tabu

• x' = [0, 0, 0, 1] ; f(x') = 14

• Since f(x') < f(x*) = 32, then

x* not modified



• TL = { (4, 0), (1, 1)}

• TL = { (4, 0)}

• iter = 2

niter = 1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Current sol. x = [0, 0, 0, 1] ;

f(x) = 14

• N(x) f

[1, 0, 0, 1] 32 <= tabu

[0, 1, 0, 1] 39

[0, 0, 1, 1] 25

[0, 0, 0, 0] 0 <= tabu• TL = { (4, 0), (1, 1)}

• iter = 3

niter = 2

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Current sol. x = [0, 0, 0, 1] ;

f(x) = 14

• N(x) f

[1, 0, 0, 1] 32 <= tabu

[0, 1, 0, 1] 39

[0, 0, 1, 1] 25

[0, 0, 0, 0] 0 <= tabu

• x' = [0, 1, 0, 1] ; f(x') = 39

• Since f(x') > f(x*) = 32, then

x* = x' , and niter := 0



• TL = { (1, 1), (2, 0)}

• TL = { (4, 0), (1, 1)}

• iter = 3

niter = 2

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• Current sol. x = [0, 1, 0, 1] ;

f(x) = 39

• N(x) f

[0, 0, 0, 1] 14 <= tabu

[0, 1, 0, 0] 25

• TL = { (1, 1), (2, 0)}

• iter = 4

niter =1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

. • Current sol. x = [0, 1, 0, 1] ;

f(x) = 39

• N(x) f

[0, 0, 0, 1] 14 <= tabu

[0, 1, 0, 0] 25

• x' = [0, 1, 0, 0] ; f(x') = 25

• Since f(x') < f(x*) = 39, then

x* is not modified



• TL = { (2, 0), (4, 1)}

• TL = { (1, 1), (2, 0)}

• iter = 4

niter =1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

• TL = { (2, 0), (4, 1)}

• iter = 5

niter =2

• Stop since iter = itermax = 5

• The solution

x* = [0, 1, 0, 1]

f(x*) = 39

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Simulated Annealing (SA)

• Probabilistic technique allowing to move out of local minima

• At each iteration, a solution x' is selected randomly in a subset of the neighborhood N(x)

• This approach was already used to simulate the evolution of an unstable physical system toward a thermodynamic stable equilibrium point at a fixed temperature


0

0

0

0

Select an initial temperature

Let 0 and =

Select an

Initialize

Repeat until stopping criterion sat

isf

initial solution

Let *

ed

i

x X

x x

TP

iter niter TP P

i

x

T

∈

= =

= =

= 1 ; 1

minoriter 0 ; ( )

Repeat until minoriter = minoritermax

minoriter = minoriter + 1

Generate randomly ' (

ter iter niter niter

Value f x

x V N

+ = +

= =

∈ ⊆ )

If = ( ' ) ( ) < 0 then '

else

Generate random number (0, 1]

x

f f x f x x x

r

∆ − =

∈

If r < e then '

If ( ) ( *) then * , and 0

If ( ) then 0

If max or max

fTP x x

f x f x x x niter

f x Value niter

TP TP


α

∆−=

< = =

< =

=

= =


* is the best solution generatedx

denotes the current solution

* denotes the best solution found so far

x

x


0

0

0

0



Let *

Let 0

Initialize

Repeat until stopping criterion sa

and =

tisfied

x X

TP

x x x

iter niter TP TP

i

∈

= =

= =

= 1 ; 1

minoriter 0 ; ( )





Value f x

x V N

+ = +

= =

∈ ⊆

If = ( ' ) ( ) < 0 then '

else


)

f f x f x x x

r

x

∆ − =

∈

If r < e then '

If ( ) ( *) then * , and 0

If ( ) then 0

If max or max

fTP x x

f x f x x x niter

f x Value niter

TP TP


α

∆−=

< = =

< =

=

= =



( )

At each iteration a solution is

selected randomly in a subset

x

V N x

′

⊆


0

0

0

0



Let *

Let 0

Initialize


and =

tisfied

x X

TP

x x x

iter niter TP TP

i

∈

= =

= =

= 1 ; 1

minoriter 0 ; ( )





Value f x

x V N

+ = +

= =

∈ ⊆ )

else

Generate random num

If = ( ' ) ( ) < 0 th

ber (0, 1]

en '

f

r

x x

x

f x f x∆ − =

∈

If r < e then '

If ( ) ( *) then * , and 0

If ( ) then 0

If max or max

fTP x x

f x f x x x niter

f x Value niter

TP TP


α

∆−=

< = =

< =

=

= =



( )As long as is better than 0 , the

behavior of the procedure is similar to that

of the descent method

x x f′ ∆ <


0

0

0

0



Let *

Let 0

Initialize


and =

tisfied

x X

TP

x x x

iter niter TP TP

i

∈

= =

= =

= 1 ; 1

minoriter 0 ; ( )





Value f x

x V N

+ = +

= =

∈ ⊆ )

If = ( ' ) ( ) < 0 then '

else


x

f f x

r

f x x x∆ −

∈

=

If ( ) ( *) then * , and 0

If ( ) then 0

If < e then '

If max or max

fTP

f x f x x x niter

f x Value niter

TP TP


r x x

r

α

∆−

< = =

< =

=

=

=

=



if 0, then the that replaces

as increases and as the number of

iterations

probability

d

already comp

ecreases

leted increases

f x x

f

′∆ ≥

∆


0

0

0

0



Let *

0Let =

Initialize


and

x X

x x x

iter

TP

TP Tn Piter

i

∈

= =

= =

= 1 ; 1

minoriter 0 ; ( )





Value f x

x V N

+ = +

= =

∈ ⊆ )

If = ( ' ) ( ) < 0 then '

else


x

f f x f x x x

r

∆ − =

∈

If < then '

If ( ) ( *) then * , and 0

If ( ) then 0

If max or max

e

fTP

T

r x x

f x f x x x niter

f x Value niter

iter iter niter n e

P TP

it r

α

∆−

=

=

< = =

< =

= =




as inc as the number of

probability

decreas

iterations already

re

completed increas

ases and e

es

s

f x x

f

′∆ ≥

∆

( )

A temperature factror is used to account for

the number of iterations increasing. Thus is

multiplied by a parameter value 0,1 in

order for to decrease with the number of

iterations

TP

TP

TP

α ∈


0

0

0

0



Let *

Let 0 and =

Initia

lize


x X

TP

x x x

iter niter TP TP

i

∈

= =

= =

minoriter 0


= 1 ; 1

; ( )

Generate random


ly ' (


Value f x

x V N

+ = +

=

∈ ⊆

=

)

If = ( ' ) ( ) < 0 then '

else


x

f f x f x x x

r

∆ − =

∈

If < e then '

If ( ) ( *) then * , and 0

If ( ) then 0

If max or max

fTPr x x

f x f x x x niter

f x Value niter

TP TP


α

∆−=

< = =

< =

=

= =




as increases and as the number of

iterations

probability

d

already comp

ecreases

leted increases

f x x

f

′∆ ≥

∆

( )

A temperature factror is used to account for

the number of iterations increasing. Thus is

multiplied by a parameter value 0,1 in

order for to decrease with the number of

iterations

TP

TP

TP

α ∈

minoritermax iterations are completed

using the same temperature TP


0

0

0

0



Let *

Let 0 and =

Initia

lize


x X

TP

x x x

iter niter TP TP

i

∈

= =

= =

= 1 ; 1

minoriter 0 ; ( )





Value f x

x V N

+ = +

= =

∈ ⊆ )

If = ( ' ) ( ) < 0 then '

else


x

f f x f x x x

r

∆ − =

∈

If < e then '

, and 0

If ( ) then 0

If ( ) ( *) then

If max or max

*

fTPr x x

niter

f x Value niter

TP TP


f x f x x x

r

α

∆−=

=

< =

=

= =

< =



Update * the best solution found so farx


0

0

0

0



Let *

and =

Initialize


Let 0

x X

TP

x x

iter nit

x

er P

i

TP T=

∈

= =

=

minoriter 0 ;


minoriter = minoriter +

= 1 ;

1

Generate

1

( )

randomly ' (


Valu x

x V

e

N

f

+ = +

==

∈ ⊆ )

If = ( ' ) ( ) < 0 then '

else


x

f f x f x x x

r

∆ − =

∈

and 0

If

If < e then '

If ( ) ( *) then * ,

( ) then 0

If max or max

fTP

niter

f x Value n

r x x

f x f x x

iter


x

TP TPα

∆−

=

<

=

< =

=

=

=

=



( )

Stopping criteria:

- Maximum number of different

temperature values

- Maximum number of successive

different temperature values

where does not improvef x

Knapsack Problem


max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


The value of one and only one variable is modified (from 0 to 1 or from 1 to 0)

to generate a new solution satisfyingconstraint

2x1 + 2x2 + x3 + x4 ≤ 3

• TP0 = 20

• α = 0.5

• itermax = 3

minoritermax = 2

nitermax = 2

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

1 2 3 42 2 3x x x x+ + + ≤

Knapsack Problem

Random numbers sequence:

0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• Initial sol. x0 = x = [1, 0, 0, 0] ;

f(x) = 18 ; Value = 18

• N(x) f

[0, 0, 0, 0] 0

[1, 0, 1, 0] 29

[1, 0, 0, 1] 32

• TP = 20

• iter = 1

minoriter = 1

niter = 1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.

Random number : 0.72

• TP = 20

• iter = 1

minoriter = 1

niter = 1

• Initial sol. x0 = x = [1, 0, 0, 0] ;

f(x) = 18 ; Value = 18

• N(x) f

[0, 0, 0, 0] 0

[1, 0, 1, 0] 29

[1, 0, 0, 1] 32

[0, 0, 0, 0] [1, 0, 1, 0] [1, 0, 0, 1]

0 .33 .66 1

.72

x' = [1, 0, 0, 1] ; f(x') = 32

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


• TP = 20

• iter = 1

minoriter = 1

niter = 1

• Initial sol. x0 = x = [1, 0, 0, 0] ;

f(x) = 18 ; Value = 18

• N(x) f

[0, 0, 0, 0] 0

[1, 0, 1, 0] 29

[1, 0, 0, 1] 32

[0, 0, 0, 0] [1, 0, 1, 0] [1, 0, 0, 1]

0 .33 .66 1

.72

x' = [1, 0, 0, 1] ; f(x') = 32

• Since f(x') > f(x) = 18, then


( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• Sol. x = [1, 0, 0, 1] ;

f(x) = 32 ; Value = 18

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

• TP = 20

• iter = 1

minoriter = 2

niter = 1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


0.72, 0.83

• TP = 20

• iter = 1

minoriter = 2

niter = 1

• Sol. x = [1, 0, 0, 1] ;

f(x) = 32 ; Value = 18

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

[0, 0, 0, 1] [1, 0,0, 0]

0 0.5 1

0.83

x' = [1, 0, 0, 0] ; f(x') = 18

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.

Random number: 0.55

0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• TP = 20

• iter = 1

minoriter = 2

niter = 1

• Sol. x = [1, 0, 0, 1] ;

f(x) = 32 ; Value = 18

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

[0, 0, 0, 1] [1, 0,0, 0]

0 0.5 1

0.83

x' = [1, 0, 0, 0] ; f(x') = 18

• ∆f = 18 – 32 = - 14

• Since e - 14 / 20 ≈ 0.497 < 0.55

we keep the same current solution x

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• Sol. x = [1, 0, 0, 1] ;

f(x) = 32 ; Value = 32

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

• TP = αTP = 0.5*20 = 10

• iter = 2

minoriter = 1

niter = 1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.

Random number: 0.41

0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• TP = 10

• iter = 2

minoriter = 1

niter = 1

• Sol. x = [1, 0, 0, 1] ;

f(x) = 32 ; Value = 32

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

[0, 0, 0, 1] [1, 0,0, 0]

0 0.5 1

0.41

x' = [0, 0, 0, 1] ; f(x') = 14

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.

Random number: 0.09

0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• TP = 10

• iter = 2

minoriter = 1

niter = 1

• Sol. x = [1, 0, 0, 1] ;

f(x) = 32 ; Value = 32

• N(x) f

[0, 0, 0, 1] 14

[1, 0, 0, 0] 18

[0, 0, 0, 1] [1, 0,0, 0]

0 0.5 1

0.41

x' = [0, 0, 0, 1] ; f(x') = 14

• ∆f = 14 – 32 = - 18

• Since e - 18 /10 ≈ 0.165 > 0.09, then


( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• Sol. x = [0, 0, 0, 1] ;

f(x) = 14 ; Value = 32

• N(x) f

[1, 0, 0, 1] 32

[0, 1, 0, 1] 39

[0, 0, 1, 1] 25

[0, 0, 0, 0] 0

• TP = 10

• iter = 2

minoriter = 2

niter = 1

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.

Random number: 0.64

0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• TP = 10

• iter = 2

minoriter = 2

niter = 1

• Sol. x = [0, 0, 0, 1] ;

f(x) = 14 ; Value = 32

• N(x) f

[1, 0, 0, 1] 32

[0, 1, 0, 1] 39

[0, 0, 1, 1] 25

[0, 0, 0, 0] 0[1, 0, 0, 1] [0, 1, 0, 1] [0, 0, 1, 1] [0, 0, 0, 0]

0 0.25 0.50 0.75 1

0.64

x' = [0, 0, 1, 1] ; f(x') = 25

• Since f(x') > f(x) = 14, then


( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Knapsack Problem

max f(x) = 18x1 + 25x2 + 11x3 + 14x4

Subject to

2x1 + 2x2 + x3 + x4 ≤ 3

x1 , x2 , x3 , x4 = 0 or 1.


0.72, 0.83, 0.55, 0.41, 0.09, 0.64

• TP = 5

• iter = 3

minoriter = 0

niter = 2

• Sol. x = [0, 0, 1, 1] ;

f(x) = 25 ; Value = 32

• Stop since iter = itermax = 3

• The solution

x* = [1, 0, 0, 1]

f(x*) = 32

( ) 1 2 3 4

1 2 3 4

1 2 3 4

max 18 25 11 14

s.t. 2 2 3

, , , 0 or 1

f x x x x x

x x x x

x x x x

= + + +

+ + + ≤

=

Improving Strategies

• Intensification

• Multistart diversification strategies:

- Random Diversification (RD)

- First Order Diversification (FOD)

Intensification

• Intensification strategy used to search more extensively a

promissing region

• Two different ways (among others) of implementing:

- Temporarely enlarge the neighborhood whenever the

current solution induces a substancial improvement over

the previous best known solution

- Return to the best known solution to restart the LSM using

a temporarely enlarged neighborhood or using temporarely

shorter Tabu lists

Diversification

• The diversification principle is complementary to the

intensification. Its objective is to search more extensively the

feasible domain by leading the LSM to unexplored regions of

the feasible domain.

• Numerical experiences indicate that it seems better to apply a

short LSM (of shorter duration) several times using different

initial solutions rather than a long LSM (of longer duration).

Random Diversification (RD)

Multistart procedure using new initial solutions generated

randomly (with GRASP for instance)

First Order Diversification (FOD)

• Multistart procedure using the current local minimum x* to

generate a new initial solution

• Move away from x* by modifying the current resources of

some activities in order to generate a new initial solution in

such a way as to deteriorate the value of f as little as possible

or even improve it, if possible

Variable Neighborhood Search (VNS)

• Specify ( a priori) a set of neighborhood structures Nk, k = 1, 2,…, K



• At each major iteration,

a “local minimum” x'' is generated using some (LSM) where the initial

solution x' is selected randomly in Nk(x*), and using the neigborhood

structure Nk




a “local minimum” x'' is generated using some (LSM) where the initial solution x' is selected randomly in Nk(x*), and using the neigborhoodstructure Nk

if f(x'') < f(x*) then x'' replaces x* and

the neighborhood structure N1 is used for the next

major iteration

Justification: we assume that it is easier to apply

the LSM with neighborhood structure

N1




a “local minimum” x'' is generated using some (LSM) where the initial solution x' is selected randomly in Nk(x*), and using the neigborhoodstructure Nk

if f(x'') < f(x*) then x'' replaces x* and

the neighborhood structure N1 is used for the next

major iteration

else the neighborhood structure Nk+1 is used for the next

major iteration

Recall: Neighborhood for assignment type

problem

The elements of the neighborhood

N(x) are generated by slightly

modifying x:

N(x) = {x' : x' = x ⊕ m , m ∈ M }1, ,i n= …

( ) ( ) ( ) 1 , , , ,j j i j nx ⇒ … …

( )

( )

( ) ( ) ( )

( )( )

1 ,

Let

,

b

, ,

Each solution is obtained by selecting an activity

and modifying

e as follows: for each 1, , :

1

its resource

0 for 1, , ;

or

ij i

ij

j j i

x i n

x

x j m j j i

x j n

x N x i

j i

⇒

′

=

= = ≠

∈

=

… …

…

…

[ ]( )

( )

( ) ( )

i.e., the modification can be denoted , :

0

1

to some o

for all other ,

or

t

1 , , ,

her

,

ij i

ip

ij ij

m i p

x

x

x x i j

x j p j

p

n

=

′ =

′ =

′ =

′ ⇒ … …

Different neighborhood structures

For the assignment type problem, different neighborhood

structures Nk can be specified as follows:

each solution x’ Nk (x) is obtained by selecting k or less

different activities i and modifying their resource from

j(i) to some other pi

∈

Exchange Procedure (EP)

• Variant of (VNS) with 2 neighborhood structures where the descent method

(D) is the (LSM) used:

i) Apply the descent method with neighborhood structure N(x)

ii) Once a local minimum x* is reached, apply the descent method using an

enlarged and more complex neighborhood EN(x) to find a new solution x'

such that f(x') < f(x*).

One way to do this is to apply a truncated depth-first enumaration tree

search where x* is the root. The search is truncated to monitor the number

of modifications used (i.e., the depth of the tree or the size of the

neighborhood) and the level of deterioration tolerated before reaching x' .

One way to do this is to apply a truncated depth-first enumaration tree

search where x* is the root. The search is truncated to monitor the

number of modifications used (i.e., the depth of the tree or the size of

the neighborhood) and the level of deterioration tolerated before

reaching x' .

x*

Material from the following reference

• Jacques A. Ferland and Daniel Costa, “ Heuristic Search

Methods for Combinatorial Programming Problems”,

Publication # 1193, Dep. I. R. O., Université de Montréal,

Montréal, Canada (March 2001)

Documents

Department of Informatique and Recherche Opérationnelle ...ferland/Seminars/Local_search/1. LSM.pdf · Department of Informatique and Recherche Opérationnelle Université de Montréal