49
Introduction SOC Relaxations Complexity Row Layout Conclusion Conic Optimization: An Exciting Present and a Promising Future Miguel F. Anjos Professor and Canada Research Chair Various parts are joint work with A. Engau (U. Colorado-Denver) B. Ghaddar (U. Waterloo; now IBM) P. Hungerländer (Klagenfurt U.) A. Vannelli (U. Guelph) J.C. Vera (Tilburg U.) EUROPT 2014 – Université de Perpignan, France – July 11, 2014

Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Conic Optimization:An Exciting Present and a Promising Future

Miguel F. AnjosProfessor and Canada Research Chair

Various parts are joint work withA. Engau (U. Colorado-Denver)

B. Ghaddar (U. Waterloo; now IBM)P. Hungerländer (Klagenfurt U.)

A. Vannelli (U. Guelph)J.C. Vera (Tilburg U.)

EUROPT 2014 – Université de Perpignan, France – July 11, 2014

Page 2: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Conic OptimizationThe problem of optimizing a linear or convex quadratic function overthe intersection of an affine space and a pointed closed convex cone:

(P) inf 〈c, x〉 (D) sup bT y

s.t. 〈ai , x〉 = bi , i = 1, . . . ,m s.t.m∑

i=1yiai + s = c

x ∈ K s ∈ K∗

where the dual cone K∗ is defined as

K∗ := {y ∈ <n : 〈x,y〉 ≥ 0 ∀x ∈ K}.

If K = Rn+ then we have linear optimization (LO)

If K = SOCn = {x ∈ Rn+1 : x0 ≥√

x21 + . . .+ x2

n} then we havesecond-order cone optimization(SOCO)If K = Sn

+ then we have (positive) semidefinite optimization (SDO)

These cones are self-dual: K = K∗ .

Page 3: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

“LO ( SOCO ( SDO”

The SOC constraint can be rewritten as

x20 − x2

1 − . . .− x2n ≥ 0, x0 ≥ 0

and is therefore equivalent to the positive semidefinite constraintx0 x1

x0 x2x0 x3

. . ....

x1 x2 x3 · · · x0

� 0

where � 0 denotes positive semidefiniteness.

Hence, SOCO is a special case of SDO, andLO is a special case of SOCO.

Page 4: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Trending Now...1 Increasing degree of the functions describing the models:

LinearQuadraticPolynomial

2 Increasing complexity of the convex cones underlying the models:Non-negative orthantSecond-order and semidefinite conesCopositive and completely positive cones

3 Increasing variety of algorithms to compute with these cones:Ellipsoid methodInterior-point methodsFirst-order methods; bundle methods; Lagrangian methodsCombinations with branch-and-bound, cutting planes

4 Increasing array of applications:Control theoryCombinatorial optimizationFinanceEngineeringPolynomial optimization

Page 5: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Trending Now...1 Increasing degree of the functions describing the models:

LinearQuadraticPolynomial

2 Increasing complexity of the convex cones underlying the models:Non-negative orthantSecond-order and semidefinite conesCopositive and completely positive cones

3 Increasing variety of algorithms to compute with these cones:Ellipsoid methodInterior-point methodsFirst-order methods; bundle methods; Lagrangian methodsCombinations with branch-and-bound, cutting planes

4 Increasing array of applications:Control theoryCombinatorial optimizationFinanceEngineeringPolynomial optimization

Page 6: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

A Success Story: The Max-Cut ProblemGiven a graph G = (V ,E) and weights wij for all edges (i , j) ∈ E , findan edge-cut of maximum weight, i.e. find a set S ⊆ V s.t. the sum ofthe weights of the edges with one end in S and the other in V \ S ismaximum.

Max-Cut is in fact equivalent to quadratic unconstrained binaryoptimization (QUBO), and hence has many applications.

Page 7: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Integer Linear Optimization (ILO) Formulation

maxn∑

i=1

n∑j=i+1

wijyij

s.t. yij + yik + yjk ≤ 2,1 ≤ i < j < k ≤ nyij − yik − yjk ≤ 0,1 ≤ i < j ≤ n, k 6= i , jyij ∈ {0,1},1 ≤ i < j ≤ n

where

yij =

{1 if edge ij is cut0 otherwise,

yij = yji , and wij denotes the weight of edge ij .

This formulation is the basis for a highly successful branch-and-cutalgorithm for solving spin glass problems in physics (Liers, Jünger,Reinelt and Rinaldi (2005)).

Page 8: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Quadratic Formulation of Max-CutWhereas the ILO formulation is edge-based, we use a node-basedquadratic formulation.

Let the vector v ∈ {−1,+1}n represent any cut in the graph viathe interpretation that the sets {i |vi = +1} and {i |vi = −1} specifythe partition.Then max-cut may be formulated as:

maxn∑

i=1

n∑j=i+1

wij

(1−vi vj

2

)s.t. v2

i = 1, i = 1, . . . ,n.

If wij = wji and wii = 0 (without loss of generality), then:

n∑i=1

n∑j=i+1

wij

(1− vivj

2

)= vT Qv with Q symmetric.

Page 9: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

The Basic Semidefinite Relaxation of Max-Cut

Consider the change of variable X = vvT , v ∈ {±1}n.Then Xij = vivj , vT Qv = 〈Q, vvT 〉 = 〈Q,X 〉, and max-cut is equivalentto

max 〈Q,X 〉s.t. diag(X ) = e

rank(X ) = 1X � 0,

where e denotes the vector of ones, and X � 0 denotes that X issymmetric positive semidefinite.

Removing the rank constraint, we obtain the basic SDO relaxation ofmax-cut:

max 〈Q,X 〉s.t. diag(X ) = e

X � 0.

Page 10: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Max-Cut Solvers Available On the WebLO-based Spin Glass Server (Liers, Jünger, Reinelt and Rinaldi (2005):

http://www.informatik.uni-koeln.de/spinglass/

An SDO-based cutting-plane framework is a key ingredient of themax-cut solver Biqmac (Rendl, Rinaldi and Wiegele (2007)):

http://biqmac.uni-klu.ac.at/

A similar approach with the addition of a quadratic regularization termto the semidefinite relaxation is the basis of the solver BiqCrunch(Krislock, Malick and Roupin (2012)):

http://lipn.univ-paris13.fr/BiqCrunch/

Page 11: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Selected Extensions

This basic relaxation of max-cut is also the basis for successfulsolution approaches to other problems, including:

Max-k -cut problems (Ghaddar, A. and Liers (2007); A., Ghaddar,Hupp, Liers, Wiegele (2013))Min-bisection problems (Armbruster, Helmberg, Fügenschuh andMartin (2011))k -clustering probllems (Krislock, Malick and Roupin (2014))Single-row facility layout problems (A., Kennings, Vannelli (2005);A. and Vannelli (2008); A. and Yen (2009); Hungerländer andRendl (2012))Multi-row facility layout problems (Hungerländer and A. (2012,2014))

Page 12: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Algorithmic Framework: Cutting-Plane Algorithm

Solve initial relaxation

Look for validinequalities;

if none found, stopotherwise

Add and/or removeinequalities;then resolve

repeat

For max-cut one can use:

Initial Relaxationmax 〈Q,X 〉s.t. diag(X ) = e

X � 0

Triangle Inequalities

Xij + Xik + Xjk ≥ −1−Xij + Xik + Xjk ≥ −1

Xij − Xik + Xjk ≥ −1Xij + Xik − Xjk ≥ −1

Page 13: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

General Research Objective:Improve the Performance of this Framework

Solve initial relaxation

Look for validinequalities;

if none found, stopotherwise

Add and/or removeinequalities;then resolve

repeat

Today: Three Aspects1 SOC relaxations for

binary quadraticoptimization

2 Complexity of handlinglarge numbers ofinequalities

3 Another success story:Application to rowlayout problems

Page 14: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

SOC Relaxations for Binary Quadratic Optimization

This is joint work with B. Ghaddar and J.C. Vera.

B. Ghaddar, J.C. Vera, and M.F. Anjos.Second-Order Cone Relaxations for Binary Quadratic PolynomialPrograms.SIAM Journal on Optimization, 21(1), 2011, 391-414.

Page 15: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Polynomial Optimization PerspectiveA general polynomial optimization (PO) problem has the form:

sup f (x)s.t. gi(x) ≥ 0, i = 1, . . . ,m

The problem can be equivalently written in the form:

inf λs.t. λ− f (x) ≥ 0 ∀x ∈ S := {x : gi(x) ≥ 0, i = 1, . . . ,m}

which we re-write as

inf λs.t. λ− f (x) ∈ Pd(S)

wherePd(S) = {p(x) ∈ Rd [x ] : p(s) ≥ 0 for all s ∈ S}

is the cone of polynomials of degree ≤ d that are non-negative over S.

Page 16: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

A General Recipe for Relaxations of POThe set

Pd(S) = {p(x) ∈ Rd [x ] : p(s) ≥ 0 for all s ∈ S}

is in general a very complex object.

It is always a convex cone.In most cases the decision problem for Pd(S) is NP-hard.Hence the condition λ− f (x) ∈ Pd(S) is NP-hard in general.We relax it to λ− f (x) ∈ K for a suitable K ⊆ Pd(S).Then

inf λs.t. λ− f (x) ∈ K

provides an upper bound for the original problem.The choice of the convex cone K is key for the quality of theresulting bounds.

Page 17: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Building Blocks for Conic Representations - IConsider the following example:

p(x , y) = 8− 2x − x2 − 12xy + 2y2

S = {(x , y) : x2 + y2 ≤ 1}Observe that

p(3,0) < 0, butp(x , y) ≥ 0 for all (x , y) ∈ S, since

p(x , y) = (1− 2x + 2y)2 + (1 + x − 2y)2 + 6(1− x2 − y2).

Theorem (S-lemma)

Let B := {x : ‖x‖2 ≤ 1}. Then

p(x) ∈ P2(B) iff p(x) = α(x) + c(1− ‖x‖2)

where α(x) is a degree-2 sum-of-squares (SOS) and c ≥ 0.

(see the survey of Pólik and Terlaky (2007)).

Page 18: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Applying the S-lemmaHow to find α(x) and c?Every degree-2 SOS has the form

α(x) =[

1 xT]

M[

1x

]with M � 0.

Therefore proving non-negativity of p(x , y) reduces to a semidefinitefeasibility problem: 8 −1 0

−1 −1 −60 −6 2

= M + c

1 0 00 −1 00 0 −1

, with M � 0, c ≥ 0.

CorollaryOptimizing a polynomial of degree 2 over the unit ball can be cast as aconic (semidefinite) optimization problem.

Page 19: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Building Blocks for Conic Representations - IIp(x , y) = 5− 4x + 3yS = {(x , y) : x2 + y2 ≤ 1}

Observe that p(x , y) ≥ 0 for all (x , y) ∈ S since

p(x , y) = 5 + (−4,3)[

xy

]≥ 5− ‖(−4,3)‖ = 0

TheoremLet B = {x : ‖x‖ ≤ 1} then

p(x) ∈ P1(B) iff p(x) = aT[

1x

],a ∈ L = {(uo,u) : ‖u‖ ≤ uo}.

CorollaryOptimizing a polynomial of degree 1 over the unit ball can be cast as aconic (SOC) optimization problem.

Page 20: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

What about P3(B)?

Theorem (Nesterov (2003))The decision problem over P3(B) is NP-hard.

In summary:Let B = {x : ‖x‖ ≤ 1}. The decision problem p(x) ∈ Pd(B)

is easy for d = 1,2;is NP-hard for d ≥ 3.

Standard Polynomial Optimization ApproachGenerate hierarchy of tractable approximations of Pd(S):

K1 ⊆ K2 ⊆ · · · ⊆ Pd(S)

Page 21: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

First Example: The Max-Cut Problem

Consider the well-known formulation of max-cut:

max 12∑

i<j wij(1− xixj)

s.t. x ∈ {−1,1}

The only constraints are the binary constraints. We recast it as:

min λ

s.t. λ− 12∑

i<j wij(1− xixj) ∈ P2 ({−1,1}n)

Page 22: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Hierarchy of Relaxations

Degree-r Relaxation

(Pr ) min λ

s.t. λ− 12

∑i<j

wij(1− xixj) = α(x) +∑

i

fi(x)(1− x2i )

α(x) is SOS of degree ≤ rfi(x) is polynomial of degree ≤ r − 2

Theorem

(P1)∗ ≥ (P2)

∗ ≥ · · · ≥ (Pn)∗ = (P)∗.

Page 23: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Second Example: Quadratic Knapsack Problem (QKP)

max q(x) =[1 x

]Q[1x

]s.t. wT x ≤ c

x ∈ {−1,1}n

Generalization of the classical linear knapsack problemIntroduced by Gallo, Hammer and Simeone (1980)Known to be NP-hard

Note that QKP has the binary constraints plus one linear constraint.

PO Reformulationmin λ

s.t. λ− q(x) ∈ P2({−1,1}n ∩ {x : c − wT x ≥ 0}

).

Page 24: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Basic Semidefinite Relaxation

The semidefinite relaxation HRW4 of Helmberg-Rendl-Weismantel(2000) is equivalent to relaxing P2({−1,1}n ∩ {x : c − wT x ≥ 0}) as:

min λ

s.t. λ− q(x) = α(x) +∑

ici(1− x2

i )

+(c − wT x){∑

id−i (1− xi) +

∑i

d+i (1 + xi)

}where α(x) is a degree-2 SOS, ci are free, and d+

i ,d−i ≥ 0.

One (n + 1)× (n + 1) semidefinite matrix2n non-negative varsn free vars∼ n2 constraints

Page 25: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

A New Result on SOC Representation

Observationx ∈ {−1,1}n ⇒ ‖x‖2 = n

This leads to the following result:

Lemma

If f (x) is a polynomial of degree 1 and B′ := {x : ‖x‖2 = n}, then

f (x) ∈ P1(B′) if and only if f (x) = f T(√

nx

)with f ∈ Ln+1, where Ln+1 is the second-order cone.

Page 26: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Basic SOCO-SDO relaxationBasic SOCO-SDO relaxation

min λ

s.t. λ− q(x) = α(x) +∑

ici(1− x2

i )

+(c − wT x)f (x) +∑

i(1− xi)g−i (x) +∑

i(1 + xi)g+i (x)

whereα(x) is a degree-2 SOS, ci are free, and f (x),g+

i (x),g−i (x) ∈ P1(B′).One (n + 1)× (n + 1) semidefinite matrix2n + 1 vectors in Ln

n free vars∼ n2 constraints

Theorem

HRW4 ∗ ≥ SOCO-SDO ∗ ≥ QKP ∗

Page 27: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

A Pure SOC Relaxation for QKPTo verify how much SOCO alone can do, we remove the SDO part:

Basic SOCO-SDO relaxation

min λ

s.t. λ− q(x) = α(x) +∑

ici(1− x2

i )

+(c − wT x)f (x) +∑

i(1− xi)g−i (x) +∑

i(1 + xi)g+i (x)

whereα(x) is a degree-2 SOS, ci are free, and f (x),g+

i (x),g−i (x) ∈ P1(B′).

One (n + 1)× (n + 1) semidefinite matrix2n + 1 vectors in Ln

n free vars∼ n2 constraints

Page 28: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Comparison of Upper Bounds for the QKP

n_d_rnd HRW4 SOCO-SDO SOCO50_10_50 2412.5 2353.9 2846.150_30_50 11485.6 11433.2 12050.950_50_50 23863.0 23846.1 23851.050_70_50 32626.5 32571.1 32575.150_90_50 17682.6 17671.0 17672.860_10_60 7216.0 7188.7 7410.160_30_60 26530.8 26496.5 26502.760_50_60 13895.5 13871.4 14396.660_70_60 56583.5 56561.2 56561.460_90_60 62015.6 62009.0 62009.070_10_70 4109.6 4036.7 5104.270_30_70 20275.1 20208.6 21826.870_50_70 45573.2 45507.1 45752.870_70_70 1882.8 1631.6 1737.970_90_70 32914.0 32857.3 32876.1

Page 29: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Triangle Inequalities Relaxation

Any relaxation for the QKP can be improved using the triangleinequalities:

x ∈ {−1,1}n ⇒

xixj + xjxk + xkxi ≥ −1,xixj − xjxk − xkxi ≥ −1,−xixj + xjxk − xkxi ≥ −1,−xixj − xjxk + xkxi ≥ −1.

Page 30: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Improved SOCO Relaxation

SOCO-4 Relaxation

min λ

s.t. λ− q(x) =∑

ici(1− x2

i )

+(c − wT x)f (x) +∑

i(1− xi)g−i (x) +∑

i(1 + xi)g+i (x)

+∑

i<j<k d±,±ijk (xixj ± xjxk ± xkxi + 1)

where α(x) is a degree-2 SOS, ci are free,f (x),g+

i (x),g−i (x) ∈ P1(B′), and d±,±ijk ≥ 0.

2n + 1 vectors in Ln

n free variables∼ n2 constraints∼ n3 nonnegative variables

Page 31: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Comparison of Upper Bounds for the QKP (ctd)

n_d_rnd HRW4 SOCO-SDO SOCO SOCO-SDO-4 SOCO-450_10_50 2412.5 2353.9 2846.1 2316.1 2316.150_30_50 11485.6 11433.2 12050.9 11403.2 11403.250_50_50 23863.0 23846.1 23851.0 23815.1 23815.850_70_50 32626.5 32571.1 32575.1 32555.5 32555.550_90_50 17682.6 17671.0 17672.8 17651.5 17651.660_10_60 7216.0 7188.7 7410.1 7170.7 7170.860_30_60 26530.8 26496.5 26502.7 26488.8 26488.960_50_60 13895.5 13871.4 14396.6 13844.5 13845.760_70_60 56583.5 56561.2 56561.4 56552.6 56552.660_90_60 62015.6 62009.0 62009.0 62009.0 62009.070_10_70 4109.6 4036.7 5104.2 3957.4 3958.170_30_70 20275.1 20208.6 21826.8 20190.0 20190.070_50_70 45573.2 45507.1 45752.8 45488.1 45488.270_70_70 1882.8 1631.6 1737.9 1619.7 1619.670_90_70 32914.0 32857.3 32876.1 32837.2 32837.2

Page 32: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Comparison of Computational Time (sec)

n_d_rnd HRW4 SOCO-SDO SOCO SOCO-SDO-4 SOCO-450_10_50 121.1 122.4 10.9 755.3 386.950_30_50 122.8 134.6 11.2 688.4 326.950_50_50 116.5 122.7 13.7 727.5 320.350_70_50 94.1 139.5 15.3 601.0 260.150_90_50 99.9 128.4 14.0 723.6 381.060_10_60 540.8 670.4 25.8 3769.0 1336.060_30_60 572.3 644.9 26.1 5420.0 1796.060_50_60 564.4 703.3 25.2 3597.0 1344.060_70_60 540.7 868.2 28.3 3645.0 1502.060_90_60 556.2 520.0 23.3 1815.0 450.270_10_70 2074.0 2295.0 43.8 12460.0 2192.770_30_70 2124.0 2360.0 48.5 9209.0 1747.070_50_70 2107.0 2806.0 56.2 10260.0 3212.070_70_70 2453.0 1631.6 46.4 11780.0 4226.070_90_70 2137.0 2716.0 58.4 9840.0 2825.0

Page 33: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

In SummaryWorking in the PO framework and for binary quadratic optimization, weobtained

A tight SOCO-SDO-based relaxation.A cheaper pure SOCO relaxation that is competitive withSDO-relaxations.The impact of these relaxations was computationallydemonstrated on the QKP.

It is also possible to generate polynomial cuts:

Instead of using the classical approach to PO of approximatinguniformly the whole feasible setapply a local approach, i.e., locally approximate the feasible set byadding valid polynomial inequalities.

Such a cut generating procedure was proposed by Ghaddar, Vera andA. (2011) for general PO, and also specialized for binary quadraticoptimization.

Page 34: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Complexity of an Integrated Interior-Point Method withCutting Planes

This is joint work with A. Engau.

A. Engau and M.F. Anjos.A Primal-Dual Interior-Point Algorithm for Linear Programming withSelective Addition of Inequalities.Cahier du GERAD G-2011-44, 2011.

Page 35: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Handling a Large Number of Inequalities

The question of how to add inequalities efficiently has receivedattention for some time.

Among the contributions in this area:Column-generation and cutting-plane methods (see e.g. Mitchell(2003), Mitchell (2009));Dual approaches, such as augmented Lagrangian relaxations(see e.g. Güler (1992), Evtushenko (2005));Build-up or build-down approaches for constraint-reduction inlinear optimization (see e.g Dantzig-Ye (1991), Kaliski-Ye (1993),den Hertog-Roos-Terlaky (1994)),and more recently in convex quadratic optimization(Jung-O’Leary-Tits (2012)) and semidefinite optimization (Park(2011)).

Page 36: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Handling a Large Number of Inequalities (ctd)We restrict our attention here to the class of interior-point methods(IPMs) that combine a build-up cutting-plane approach with thedynamic addition of inequalities to solve linear and/or semidefiniteoptimization problems, such as in:

Helmberg-Rendl (1998)Mitchell (2000)Gruber-Rendl (2003)Engau-A.-Vannelli (2012)Engau-A.-Bomze (2013)

See the chapter by Engau:Recent Progress in Interior-Point Methods: Cutting-Plane Algorithmsand Warm Startsin theHandbook on Semidefinite, Conic and Polynomial Optimization.

Page 37: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Integrated Algorithm (Engau, A. and Vannelli (2012))

Start from the initial relaxation without any cuts P(X ) ≥ q

min C • Xs.t. AX = b, X � 0PIX − r = qI , r = ξ, ξ ≥ 0

max bT y + qTI z

s.t. S = C −AT y − PTI z � 0

z = ψ, ψ ≥ 0

Key Steps:Stops the interior-point algorithm after a few steps;Add and remove inequalities;Warmstart the interior-point algorithm by turning negativevariables into free variablesand adding the requirement that the free variables be equal tonew non-negative variables;Use an infeasible interior-point method.

Page 38: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

What About Polynomiality?

The well-known ideas in this algorithm (and in others too) are:to try to predict and add relevant inequalities before they areviolated, andto resume the algorithm from the current iterate.

While the computational performance of these algorithms is welldocumented in the literature,we know of no supporting theoretical analysis or proof of convergenceand worst-case complexity for such methods.

Main objective: Obtain insights into the conditions under which analgorithm of this kind is polynomial or may be exponential.

Page 39: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

General Worst-Case Complexity Result

Theorem (Engau and A. (2011))Under reasonable assumptions, an algorithm of this type finds anε-optimal solution in

O(((κ+ τ + 1)/ε)`(n + `)3/2eθ/11) iterations

where n is the number of variables in the primal,θ = O(`/

√n + `) and

` is the number of inequalities (out of L) that are added to the problem.

In particular, if ` = O(√

n) or ` ≤ L = O(√

n), then θ = O(1) and theiteration bound is polynomial: O(((κ+ τ + 1)/ε)`(n + `)3/2).

Moreover, we are not able to affirm polynomial time complexity if largenumbers of inequalities must be added very close to optimality.

This agrees with well-known observations in practice about IPMs.

Page 40: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Single- and Multi-Row Facility Layout

This is joint work with P. Hungerländer.

P. Hungerländer and M.F. Anjos.A Semidefinite Optimization Approach to Space-Free Multi-RowFacility Layout.Cahier du GERAD G-2012-03, revised May 2014.

P. Hungerländer and M.F. Anjos.A Semidefinite Optimization-Based Approach for Global Optimizationof Multi-Row Facility Layout.Cahier du GERAD G-2014-45, 2014.

Page 41: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

The Single-Row Facility Layout Problem (SRFLP)

The SRFLP consists in finding an optimal linear placement ofdepartments with varying dimensions on a straight line:

AGV

M4M3M2M1

An instance of the problem consists of:n 1-D departments {1, . . . ,n}with positive lengths `1, . . . , `n

and (usually non-negative) pairwise connectivities cij .

We seek a placement of the departments so as to minimize the totalweighted sum of the center-to-center distances between all pairs offacilities.

Page 42: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

The Multi-Row Facility Layout Problem (MRFLP)

Similar to the SRFLP but with two of more rows available to place thedepartments:

AGV

M5M4

M1 M2 M3

Page 43: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Global Optimization of the SRFLPThe SRFLP can be modeled based on the quadratic formulation ofmax-cut (A., Kennings and Vannelli (2005)):

min∑i<j

[cij∑

k 6=i,j`k

(1−Rki Rkj

2

)]+∑i<j

12cij(`i + `j)

s.t. RijRjk − RijRik − RikRjk = −1 for all triples i < j < k

R2ij = 1 for all pairs i < j

where for each pair i , j of departments:

Rij :=

{1, if facility i is to the right of facility j−1, if facility i is to the left of facility j .

Using the same algorithmic framework as for max-cut,A. and Vannelli (2008) solved instances with up to 30 departmentsto global optimality, andHungerländer and Rendl (2013) computed global optima for up to42 departments.

Page 44: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

What about the MRFLP?

There are three additional modeling issues that arise in the MRFLP butnot in the SRFLP:

1 Expressing the center-to-center distance between departmentsassigned to different rows;

2 Assigning each department to exactly one row;3 Handling the possibility of empty space between departments.

If empty space is not allowed then we have the space-free MRFLP.The SDO model from single-row can be extended to this problemwhile retaining the quality of the bounds (Hungerländer and A.(2012)).

Handling space is seemingly more tricky; all the models in theliterature use continuous variables to model the spaces betweendepartments, and that significantly weakens the bounds.

Page 45: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Modeling Spaces in the MRFLP

Theorem

If all the department lengths are integer, then there is always anoptimal solution to the MRFLP on the half grid.

Corollary

If all the department lengths are integer, then for each instance of theMRFLP we obtain an equivalent instance by adding spacingdepartments of length 0.5 such that the length of each row becomesequal to

∑ni=1 `i .

Lemma

If all departments have the same length `, then spaces of size ` aresufficient to preserve an optimal solution.

Page 46: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Global Optimization of the MRFLP

Many computational results, too little time...

Putting all these tools together gives, to the best of our knowledge,the first global optimization approach for multi-row layout that isapplicable beyond the double-row case.

Page 47: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Time to wrap up...

Page 48: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Concluding Thoughts

Conic optimization continues to be an exciting research area.

Initially much attention was given to SDO; now other cones aregetting more attention, for example:

mixed-integer SOCOcopositive and completely positive cones

There remains a need to improve the performance of algorithmsfor conic optimization:

1 Solve very large problems (large variables and many constraints):bundle methods, partial Lagrangians, etc.

2 Better tools to exploit the structure of the problem.

The use of conic optimization is expanding to more and moreapplications, and I am confident that it will remain fruitful for yearsto come.

Page 49: Conic Optimization: An Exciting Present and a Promising ...smai.emath.fr/spip/IMG/pdf/europt2014_-_anjos.pdf · Miguel F. Anjos Professor and Canada Research Chair Various parts are

Introduction SOC Relaxations Complexity Row Layout Conclusion

Read More About It...

For papers, references, questions, you are welcome to contact me:

[email protected]

Thank you for your attention and enjoy the rest of conference.