MIP-based heuristics (Matheuristics)Hybrid meta-heuristic methods for combinatorial optimization

Tatjana Davidovic

Mathematical Instituteof the Serbian Academy of Sciences and Arts

http://www.mi.sanu.ac.rs/˜tanjad(tanjad@mi.sanu.ac.rs)

February, 2012

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 1 / 12

Matheuristics for 0-1 Mixed Integer Programming

What are Matheuristics?

Matheuristic: short from math-heuristic

Hybridisation of metaheuristics and mathematical programmingtechniques

New solutions generated by manipulating the mathematical model ofthe input problem

Soyster’s algorithm for pure 0-1 integer programming problems from1978 was one of the first matheuristics

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 2 / 12

Matheuristics for 0-1 Mixed Integer Programming

The 0-1 Mixed integer programming problem (0-1 MIP)

max{ν(ξ) = cT ξ | ξ ∈ X}, (1)

whereX = {ξ ∈ R|N| | Aξ ≤ b; ξj ∈ {0, 1} for j ∈ B; ξj ∈ Z+

0 for j ∈ G; ξj ≥0 for j ∈ C} is the feasible set,N = C ∪ G ∪ B is the set of indices of all variables,C is the set of indices of continuous variables,G is the set of indices of general integer variables andB is the set of indices of binary variables,C ∩ G = ∅, C ∩ B = ∅, G ∩ B = ∅, B 6= ∅.

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 3 / 12

Matheuristics for 0-1 Mixed Integer Programming

Distance in the 0-1 MIP Solution Space

S – Solution space of 0-1 MIP problem P;

S – Solution space of LP relaxation LP(P);

Definition

The distance between x ∈ S and y ∈ S is defined asδ(x , y) =

∑j∈B | xj − yj | and can be linearised as (Fischetti & Lodi,

2003):

δ(x , y) =∑j∈B

xj(1− yj) + yj(1− xj).

When x , y ∈ {0, 1}n, δ(x , y) is equivalent to Hamming distance.

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 4 / 12

Matheuristics for 0-1 Mixed Integer Programming

Local Branching (Fischetti & Lodi, 2003)

Procedure LocBra(P, x ′, k∗)(1) Initialisation. Set proceed ←true, k ← k∗;(2) Main step.

while (proceed) doAdd cuts: ctx ≥ ctx ′, δ(x ′, x) ≤ k ;Set x ′′ ← MIPSOLVE(P, x ′);switch status do

case “optSolFound”: reverse last pseudo-cut intoδ(x ′, x) ≥ k + 1; x ′ ← x ′′; k ← k∗;

case “feasibleSolFound”: replace last pseudo-cut withδ(x ′, x) ≥ 1; x ′ ← x ′′; k ← k∗;

case “provenInfeasible”: reverse last pseudo-cut intoδ(x ′, x) ≥ k + 1; k ← k + dk∗/2e;

case “noFeasibleSolFound”: delete last pseudo-cutδ(x ′, x) ≤ k; k ← k − dk∗/2e;

endend

(3) Output. return x ′;

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 5 / 12

Matheuristics for 0-1 Mixed Integer Programming

Variable Neighbourhood Search Framework

N1(x)

f

x

x’

f(x)

x

N (x)k

Global minimum

Local minimum

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 6 / 12

Matheuristics for 0-1 Mixed Integer Programming

Variable Neighbourhood BranchingPseudo-code (Hansen & Mladenovic & Urosevic, 2006)

Procedure VNB(P, kmin, kstep, kmax , kvnd)1 Set proceed ← true; Set solutionLimit = 1;2 x ′ ← MIPSolve(P, solutionLimit);3 solutionLimit ←∞; x∗ ← x ′;4 while (proceed) do5 Q ← P; x ′′ ← VND-MIP(Q, kvnd , x

′);6 if (ctx ′′ < ctx∗) then7 x∗ ← x ′′; k ← kmin;8 else k ← k + kstep;9 x ′ ← Shake(P, x∗, k , kstep, kmax)

//If no feasible solutions are found, return current (infeasible) solution.

10 if (x ′ = x∗) then break;11 Update proceed ;12 endwhile13 return x∗;

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 7 / 12

Matheuristics for 0-1 Mixed Integer Programming

Local and VN BranchingModel-specific Parameter Adjustment

Local Branching

k∗ = b0.20|B|cVN Branching

kmax = b0.5|B|ckmin = kstep = b0.05|B|cAlternatively, kmax (and kmin by analogy) could be recomputed for eachnew incumbent solution x∗ as the objective value of the LP relaxationof the following problem:

max{δ(x∗, x) | x ∈ X}

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 8 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsBasic Idea

Let x be an optimal solution of LP(P) and p =| B |.Let δj =| xj − x j |, j ∈ B, with (xj) ordered so that δ1 ≤ δ2 ≤ . . . ≤ δp.

x

δ

x0

x1

x2

xk…

xk+k

step

xk+2k

step

xn

xn−k

step

xn−2k

step…

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 9 / 12

Matheuristics for 0-1 Mixed Integer Programming

VNDS for 0-1 MIPsPseudo-code

VNDS-MIP(P, d , x , kvnd )1 Find an optimal solution x of LP(P);2 Set proceed1← true, proceed2← true;3 while (proceed1)4 δj ←| xj − x j |, j ∈ B; index variables xj , j ∈ B,

so that δ1 ≤ δ2 ≤ . . . ≤ δp , p =| B |;5 Set nd ←| {j ∈ B | δj 6= 0} |, kstep ← [nd/d ], k ← p − kstep ;6 while (proceed2 and k > 0)7 Fix values of x1, . . . , xk ;8 x ′ ← MIPSOLVE(P, x);9 Release x1, . . . , xk ;

10 if (cT x ′ > cT x) then11 Q ← P; x ← VND-MIP(Q, kvnd , x

′); break;12 else13 if (k − kstep > p − nd ) then kstep ← max{[k/2], 1};14 Set k ← k − kstep ;15 Update proceed1 and proceed2;16 endwhile17 endwhile18 return x .

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 10 / 12

Matheuristics for 0-1 Mixed Integer Programming

Some applications

Standard benchmarks for testing 0-1 MIP solvers: the 29 data sets:

7 MIPLIB-3.0 instances1 network design instance2 crew scheduling instances5 railway crew scheduling instance1 nesting instance2 telecommunication network design instances2 rolling stock and line planning instances5 lot-sizing instances4 railway line planning instances

Barge container ship routing problem (20 random instances ofdifferent size).

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 11 / 12

Matheuristics for 0-1 Mixed Integer Programming

Summary of Experimental Results

0-1 MIP Benchmarks

VNDS VNB LB CPLEX RINS

average gap (%) 0.654 3.120 14.807 32.052 20.173gap ranks 2.43 3.02 3.43 3.45 2.67

average time (s) 6883 8103 5846 11632 11606time ranks 2.74 2.78 2.28 3.69 3.52

Ship routing and scheduling

CPLEX LB VNB VNDS

Average obj. 22533.70 22346.05 22800.50 22057.70

Average time 1518.34 517.03 1369.42 508.77

Tatjana Davidovic (MI SANU) MIP-based heuristics (Matheuristics) February, 2012 12 / 12