1

Escaping local optimas

• Accept nonimproving neighbors– Tabu search and simulated annealing

• Iterating with different initial solutions– Multistart local search, greedy randomized adaptive search procedure

(GRASP), iterative local search (ILS)

• Changing the neighborhood– Variable neighborhood search

• Changing the objective function or the input to the problem in a effort to solve the original problem more effectively. – Guided local search

2

Iterating with different initial solutions

• Multi-start local search– Several initial solutions– Obtain several local optimas using local search, tabu or SA

• ILS- Iterative local search– Improves the classical Multi-start local search– It does by perturbing the local optimas from the Multi-start local search and

reconsidering them as initial solutions– Perturbation

• Perturbation is a large random move of the current solution• Keep some part of the current solution and significantly change the other parts

of the current solution.• Complete change must be avoided because the perturbed solution loses the

properties of the local optima (current solution) that was derived from the most recent search history. Such an extreme change is a random restart approach.

• The length of the perturbation can be fixed or varied during the search.– Acceptance Criteria: Deterministic simple conditions such as solution is

better than previous best (tabu) or probabilistic (SA).

3

GRASP- greedy randomized adaptive search procedure

• A randomized greedy heuristic generates initial solutions – Adaptive because the greedy heuristic takes into account the

precedent solutions

• Using the initial solution perform a local search • Repeat the above steps several times (iterations)

• Greedy solution examples– TSP- selection of the nearest neighbor– Knapsack- Choosing objects that minimize wi/bi where wi is the weight

and bi is the benefit– Minimum spanning tree- choose least costly edges

4

GRASP for capacitated minimum spanning tree

• Minimum cost spanning tree• There is a root (central node) r• Initially all nodes are connected to the root (called gates) in a star

arrangement and every node is a sub-tree. Value Gj is the cost of links to the root

• Objective- find the minimum cost spanning tree such that every sub-tree has a certain capacity c where c could be the max number of nodes in a sub-tree or is a range (min and max value)– If every node has a weight, then c could be the maximum weight or a range for

the weight

• Exhaustive enumeration would be to find all possible spanning trees and select the best, which will take exponential time

• Application: networking of computers, communication between cell phone towers.

• Solution is to apply GRASP

5

GRASP for capacitated minimum spanning tree

• Start with different partial initial solutions and expand the tree

• Create a Restricted Candidate List RCL with edges that meet the capacity constraints.

• Randomly select from these edges to add to the sub-tree at each iteration– For multiple edges from node i to all j’s, (i,j) in RCL, find ti = Gj-cij and

choose the edge with the largest ti

– Add the edge only if the capacity constraint is met – If edge cij is added then delete gate Gj (link r-j) from the list of gates.

– Continue until no further pair of nodes with positive ti or not exceeding capacity constraint can be added to the sub-tree

6

GRASP for capacitated minimum spanning tree

• Max Capacity = 5

r

4

3

21

23

1

4

1

2

2

14

5 3

3

31

RCL 1-2, 2-3, 1-3, 3-4Capacity ≤ 5

2-4, 1-4 exceeds capacityCapacity 2-4 = 7Capacity 1-4 = 6

7

GRASP for capacitated minimum spanning tree

1 3 0 x

2 -1 1 x

3 -2 3 3

4 x x 2

1 2 3 4

RCL 1-2, 2-3, 1-3, 3-4Capacity ≤ 5

2-4, 1-4 exceeds capacityCapacity 2-4 = 7Capacity 1-4 = 6

For multiple edges from node i to all j’s, (i,j) in RCL, find ti = Gj-cij

Choose the edge with the largest positive ti (in red)

Node i

Node j

ti = Gj-cij

8

• Connect 1-2 remove r-2 link• Connect 3-4 remove r-4 link• Connect 3-2 remove r-2 link

• All nodes are connected. Choose the smallest gate among the leftover gates. Choose r-1

• Total cost = r-1-2-3-4 = 6• In a star type connection total cost is (r-1)+(r-2)+(r-3)+(r-4) =

13

• A regular spanning tree without capacity would be r-1-4-3-2 =5

GRASP for capacitated minimum spanning tree

9

Variable Neighborhood Search

• VNS exploits the second idea: change neighborhood structure.• VNS uses different neighborhood structures during search

• A neighborhood is substituted by another neighborhood as soon as local search can not improve the current best solution.

10

Guided Local Search

• Change the objective function with respect to an already obtained local optima

• f’(s) = f(s) + penalty function• Ex. TSP

– Create a penalty matrix P=pij for every edge (i,j). Initialize P to 0.– After a local optima is found, all future solutions having edges that are in

the local optima will be penalized. – A utility value is calculated for the edges in the new solution.– If the edge is not in the local optima then its utility is 0, otherwise its

utility uij = cij/1+pij . – Penalize the edge with the highest utility. The penalty is incremented by

1.– In TSP, the above increases the obj function for solutions that are similar

to the local optima and therefore, those solutions are not selected during the search. Thus, the search escapes local optima.• where l is a scaling parameter

11

Dispatching Rules

• See page 442 of text. • The rules are useful to obtain initial solutions