Vittorio Maniezzo - University of Bologna - Transportation
Logistics 1/65 TSP: an introduction Vittorio Maniezzo University of
Bologna
Slide 2
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 2/65 Travelling Salesman Problem (TSP) n locations given
(home of salesperson and n-1 customers). [d ij ] distance matrix
given, contains distances (Km, time, ) between each pair of
locations. The TSP asks for the shortest tour starting home,
visiting each customer once and returning home Symmetric /
asymmetric versions.
Slide 3
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 3/65 Introduction Which is the shortest tour? Difficult
to say
Slide 4
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 4/65 TSP history The general form of the TSP appears to
have been first studied by mathematicians starting in the 1930s by
Karl Menger in Vienna. Mathematical problems related to the
traveling salesman problem were treated in the 1800s by the Irish
mathematician Sir William Rowan Hamilton. The picture shows
Hamilton's Icosian Game that requires players to complete tours
through the 20 points using only the specified connections.
(http://www.math.princeton.e du/tsp/index.html)
Slide 5
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 5/65 Past practice
Slide 6
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 6/65 Current practice Actual tours of persons or vehicles
(deliveries to customers) Given: Graph of streets, where some nodes
represent customers. Find: shortest tour among customers.
Precondition: Solve shortest path problems Distance matrix
Slide 7
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 7/65 Application: PCB Minimization of time for drilling
holes in PCBs (printed circuit boards)
Slide 8
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 8/65 Application: setup costs Minimization of total setup
costs (or setup times) when setup costs are sequence dependent All
product types must be produced and the setup must return to initial
configuration i \ j12345 1018172021 2180211618 3169801920
4171620017 518 19160
Slide 9
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 9/65 Application: paint shop Minimization of total setup
costs (or setup times) when setup costs are sequence dependent All
colors must be processed i \ jWhitePinkRedBlue White0577 Pink10025
Red15705 Blue151050
Slide 10
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 10/65 Solution Methods n nodes ( n-1 )! possible tours
Some possible solution method: Complete enumeration IP: Branch and
Bound (and Cut, and Price, ) Dynamic programming Approximation
algorithms Heuristics Metaheuristics n 5 10 15 20
n!1203,628.8001,31*10^122,43*10^18
Slide 11
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 11/65 Standard Example Example with 5 customers (1 home,
4 customers) Zimmermann, W., O.R., Oldenbourg, 1990: e.g. cost of
1-2-3-4-5: 18+21+19+17+18 = 93 First: simplify by reducing costs
161918 5 17 2016174 2019 98163 181621 182 21201718 1 54321i \
j
Slide 12
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 12/65 Lower bound Subtract minimum from each row &
column Reduction constant = 17+16+16+16+16 + 1 = 82 all tours have
costs which are 82 greater, e.g. cost of 1-2-3-4-5: 1+5+3+0+2 = 11
[= 93 - 82] Optimal tour does not change (see [Toth, Vigo, 02]).
03225 0 4014 33 8203 105 22 3301 1 54321i \ j
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 14/65 A wider scope A combinatorial optimization problem
is defined over a set C = {c 1, , c n } of basic components. A
solution of the problem is a subset S C ; F 2 C is the subset of
feasible solutions, (a solution S is feasible iff S F). z: 2 C is
the cost function, the objective is to find a minimum cost feasible
solution S , i.e., to find S F such that z(S) z(S), S F. Failing
this, the algorithm anyway returns the best feasible solution
found, S * F.
Slide 15
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 15/65 TSP as a CO example TSP is defined over a weighted
undirected complete graph G=(V,E,D), where V is the set of
vertices, E is the set of edges and D is the set of edge weights.
The component set corresponds to E (C=E), F corresponds to the set
of Hamiltonian cycles in G z(S) is the sum of the weights
associated to the edges belonging to the solution S.
Slide 16
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 16/65 LP formulation is a binary LP (IP) which is almost
identical to the assignment problem: plus conditions to avoid short
cycles LP-Formulation
Slide 17
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 17/65 Optimal solution of LP without conditions to avoid
short cycles = solution of assignment problem short cycles 1-3-1
and 2-5-4-2 conditions to avoid short cycles are needed Example for
short cycles 12345
Slide 18
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 18/65 Conditions to avoid short cycles Several
formulations in the literature e.g. Dantzig-Fulkerson-Johnson
exponentially many!
Slide 19
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 19/65 Example: avoid short cycles Above example with n =
5 cities: must consider subsets Q: {1, 2}; {1, 3}; {1, 4}; {1, 5};
{2, 3}; {2, 4}; {2, 4}; {3, 4}; {3, 5}; {4, 5} constraint for
subset Q = {1, 2} and V-Q = {3, 4, 5} x 13 + x 14 + x 15 + x 23 + x
24 + x 25 1 n = 6 subsets (constraints) exponentially many!
Slide 20
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 20/65 Heuristics for the TSP Vittorio Maniezzo University
of Bologna
Slide 21
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 21/65 Computational issues The size of the instances met
in real world applications rules out the possibility of solving
them to optimality in an acceptable time. Nevertheless, these
instances must be solved. Thus the need to look for suboptimal
solutions, provided that they are of acceptable quality and that
they can be found in acceptable time.
Slide 22
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 22/65 Heuristic algorithms How to cope with
NP-completeness: small instances; polynomial special cases;
approximation algorithms guarantee to find a solution within a
known gap from optimality; probabilistic algorithms guarantee that
for instances big enough, the probability of getting a bad solution
is very small; heuristic algorithms: no guarantee, but
historically, on the average, these algorithms have the best
quality/time trade off for the problem of interest.
Slide 23
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 23/65 heuristics: focus on solution structure Simplest
heuristics exploit structural properties of feasible solutions in
order to quickly come to a good one. They belong to one of two main
classes: constructive heuristics or local search heuristics.
Slide 24
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 24/65 Constructive heuristics 1. Sort the components in C
by increasing costs. 2. Set S*= and i=1. 3. Repeat If (S* c i is a
partial feasible solution) then S* = S* c i. i=i+1. Until S*
F.
Slide 25
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 25/65 Constructive heuristics A constructive approach can
yield optimal solutions for certain problems, eg. the MST. In other
cases it could be unable to construct a feasible solution. TSP:
order all edges by increasing edge cost, take the least cost one
and add to it increasing cost edges, provided they do not close
subtours, until a Hamiltonian circuit is completed. More involved
constructive strategies give rise to well- known TSP heuristics,
like the Farthest Insertion, the Nearest Neighbor, the Best
Insertion or the Sweep algorithms.
Slide 26
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 26/65 Nearest Neighbor Heuristic 0. Start from any city;
e.g. the first one 1.Find nearest neighbor (to the last city) not
already visited 2. repeat 1 until all cities are visited. Then
connect last city with starting point Ties can be broken
arbitrarily. Starting from different starting points gives
different solutions! Some will be bad some will be good. GREEDY the
last connections tend to be long and improvement heuristics should
be used afterwards
Slide 27
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 27/65 Best Insertion Heuristic 0.Select 2 cities A, B and
start with short cycle A-B-A 1.Insert next city (not yet inserted)
in the best position in the short cycle 2.Repeat 1. until all
cities are visited Using different starting cycles and different
choices of the next city different solutions are obtained Often
when symmetric: starting cycle = 2 cities with maximum distance
next city = maximizes min distance to inserted cities
Slide 28
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 28/65 An approximation algorithm In some cases, it is
possible to construct solutions which are guaranteed to be not too
bad. Approximation algorithms provide a guarantee on worst possible
value for the ratio (z h z*)/z*. TSP hypothesis: it is always
cheaper to go straight from a node u to another node v instead of
traversing an intermediate node w (triangular inequality).
Slide 29
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 29/65 Approx-TSP An approx. Algorithm for the TSP with
triangular inequality is: Approx-TSP-Tour(G,w) 1select a root
vertex r V 2construct a MST T for G rooted in r 3let L be the list
of vertices visited in a preorder tree walk of T 4return the
hamiltonian cycle H which visits all vertices in the order L
Slide 30
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 30/65 Approx-TSP: example a b c h d e fg
Slide 31
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 31/65 Approx-TSP: example a b c h d e fg
Slide 32
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 32/65 Approx-TSP: example a b c h d e fg
Slide 33
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 33/65 Approx-TSP: example a b c h d e fg
Slide 34
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 34/65 Approx-TSP: performance Theorem Approx-TSP-Tour is
an approximation algorithm with a ratio bound of 2 for the
traveling salesman problem with triangle inequality. Proof Because
the optimal tour solution must have one more edge than the MST of
T, so c(MST) c(T*). c(Tour) = 2c(MST) Because of the assumption of
triangle inequality, c(H) c(Tour). Therefore, c(H) 2c(T*). Since
the input is a complete graph, so the implementation of Prims
algorithm runs in O(V 2 ). Therefore the total time complexity of
Approx-TSP-Tour also is O(V 2 ).
Slide 35
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 35/65 Local Search: neighborhoods The neighborhood of a
solution S, N(S), is a subset of 2 C defined by a neighborhood
function N : 2 C 2 2 c. Often only feasible solutions considered,
thus neighborhood functions N : F 2 F. The specific function used
has a deep impact on the algorithm performance and its choice is
left to the algorithm designer.
Slide 36
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 36/65 Local search 1. Generate an initial feasible
solution S. 2.Find S' N(S), such that z(S')=min z(S^), S^ N(S).
3.If z(S') < z(S) then S=S' go to step 2. 4. S* = S. The update
of a solution in step 3 is called a move from S to S'. It could be
made to the first improving solution found.
Slide 37
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 37/65 Local search There are problems for which a local
search approach guarantees to find an optimal solution (ex. the
simplex algorithm). For TSP, two LS are 2-opt and 3-opt, which take
a solution (a list of n vertices) and exhaustively swap the
elements of all pairs or triplets of vertices in it. More
sophisticated neighborhoods give rise to more effective heuristics,
among which Lin and Kernighan [LK73].
Slide 38
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 38/65 Local search heuristics 2 opt (arcs): try all
inversions of some subsequence if improvement start again from
beginning 3 opt (arcs): try all shifts of some subsequence to
different positions if improvement start again from beginning 2-opt
(nodes): swap every pair of nodes in the permutation representing
the solution. if improvement start again from beginning 3-opt
(nodes): swap every triplet of nodes in the permutation
representing the solution. if improvement start again from
beginning
Slide 39
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 39/65 Metaheuristcs for the TSP (some of them) Vittorio
Maniezzo University of Bologna, Italy
Slide 40
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 40/65 Metaheuristics: focus on heuristic guidance Simple
heuristics can perform very well, but can get trapped in poor
quality solutions (local optima). New approaches have been
presented starting from the mid '70ies. They are algorithms which
manage the trade-off between search diversification, when search is
going on in bad regions of the search space, and intensification,
aimed at finding the best solutions within the region being
analyzed. These algorithms have been named metaheuristics.
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 42/65 Simulated Annealing Simulated Annealing (SA) [AK89]
modifies local search in order to accept, in probability, worsening
moves. 1.Generate an initial feasible solution S, initialize S* = S
and temperature parameter T. 2.Generate S N(S). 3.If z(S') z(S)) S*
= S else accept to set S=S' with probability p = e
-(z(S')-z(S))/kT. 4.If ( annealing condition ) decrease T. 5.If
not( end condition ) go to step 2.
Slide 43
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 43/65 SA example: ulysses 16 Simulated annealing
trace
Slide 44
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 44/65 Tabu search Tabu Search (TS) [GL97] escapes from
local minima by moving onto the best solution of the neighborhood
at each iteration, even though it is worse than the current one. A
memory structure called tabu list, TL, forbids to return to already
explored solutions. 1.Generate an initial feasible solution S, set
S* = S and initialize TL = . 2.Find S' N(S), such that z(S')=min
{z(S^), S^ N(S), S^ TL}. 3. S=S', TL=TL {S}, if (z(S*) > z(S))
set S* = S. 4.If not( end condition ) go to step 2.
Slide 45
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 45/65 TS example: ulysses 16 Tabu Search trace
Slide 46
Vittorio Maniezzo - University of Bologna - Transportation
Logistics 46/65 GRASP GRASP (Greedy Randomized Adaptive Search
Procedure) [FR95] restarts search from another promising region of
the search space as soon as a local optimum is reached. GRASP
consists in a multistart approach with a suggestion on how to
construct the initial solutions. 1.Build a solution S (=S* in the
first iteration) by a constructive greedy randomized procedure
based on candidate lists. 2.Apply a local search procedure starting
from S and producing S'. 3.If z(S')