LCoP Part I – Introduction...Nemirovski A., Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, Society for Industrial and Applied Mathematics:

LCoP Part I – Introduction

Shu-Cherng Fang

Department of Industrial and Systems EngineeringGraduate Program in Operations Research

North Carolina State University

Conic Programming 3 / 25

Introduction

Content• Linear Conic Programs• Applications• Duality Theory and Algorithms• References


Linear Conic Program

Min C •Xs.t. A •X = B

X ∈ K(LCoP)

where K is a closed, convex cone; A, B and C are in the space ofinterests with • being an appropriate linear operator.


K = Rn+ (First Orthant)

When K = Rn+ = {x ∈ Rn|xi ≥ 0, i = 1, ..., n}, LCoP becomes LP.

Min cTxs.t. Ax = b

x ∈ Rn+

(LP)

where A ∈ Rm×n, b ∈ Rm and c ∈ Rn.

Equivalently,

Min cTxs.t. aTi x = bi, i = 1, . . . ,m

x ≥Rn+

0(LP)

where ai ∈ Rn, bi ∈ R and c ∈ Rn.


K = Rn+

Figure:R1+ Figure:R2

+ Figure:R3+


K = Ln (Lorentz Cone/Second OrderCone)

When K = Ln = {x ∈ Rn|√x21 + · · ·+ x2n−1 ≤ xn}, LCoP becomes

SOCP.Min cTxs.t. Ax = b

x ∈ Ln(SOCP)

where A ∈ Rm×n, b ∈ Rm and c ∈ Rn.

Equivalently,

Min cTxs.t. aTi x = bi, i = 1, . . . ,m

x ≥Ln 0(SOCP)

where ai ∈ Rn, bi ∈ R and c ∈ Rn.


K = Ln

Figure: L2 Figure: L3


K = Sn+ (Positive Semidefinite Cone)

When K = Sn+ = {X ∈ Rn×n|X = XT � 0}, LCoP becomes SDP.

Min C •Xs.t. Ai •X = bi, i = 1, ...,m

X ∈ Sn+(SDP)

where C,A1, ..., Am are given n× n symmetric matrices and b1, ..., bm aregiven scalars, and

M •X =∑i,j

MijXij = tr(MTX).

Equivalently,

Min C •Xs.t. Ai •X = bi, i = 1, ...,m

X � 0(SDP)


K = Sn+

S2+ =

{(x, y, z) ∈ R3|

[x yy z

]� 0.

}⇐⇒ x ≥ 0, z ≥ 0, xz ≥ y2.


Application of SOCP – I

Torricelli Point ProblemThe problem is proposed by Pierre de Fermat in 17th century. Giventhree points a, b and c on the R2 plane, find the point in the plane thatminimizes the total distance to the three given points. The solutionmethod was found by Torricelli, hence know as Torricelli point.

SOCP Formulation

Min t1 + t2 + t3

s.t.

[x− at1

]∈ L3,

[x− bt2

]∈ L3,

[x− ct3

]∈ L3

Question:C =? A =? X =? · · ·


Application of SOCP – II

Robust Portfolio DesignAssume returns r are known within an ellipsoid

E = {r = r + κΣ1/2u : ‖u‖2 ≤ 1}.

where r is the expected return, Σ is the empirical covariance matrix,0 < κ < 1 is a given constant.

robust counterpart: (optimize the worst case)

maxω

minr∈E{rTω : eTω = 1, ω ≥ 0}.


SOCP Formulation

Notice thatminr∈E

rTω

= min‖u‖2≤1

{rTω + κuT Σ1/2ω}

= rTω − κ‖Σ1/2ω‖2

Robust portfolio problem is an SOCP

Max rTω − κ‖Σ1/2ω‖2s.t. eTω = 1, ω ≥ 0

⇐⇒

Max ts.t. eTω = 1, ω ≥ 0[

κΣ1/2ωrTω − t

]∈ Ln+1

Question:C =? A =? X =? · · ·


Application of SDP – I

Correlation Matrix Verification

Consider three random variables A, B and C. By definition, theircorrelation coefficients ρAB , ρAC and ρBC are valid if and only if 1 ρAB ρAC

ρAB 1 ρBC

ρAC ρBC 1

� 0

Suppose we know from some prior knowledge (e.g. empirical results ofexperiments) that −0.2 ≤ ρAB ≤ −0.1 and 0.4 ≤ ρBC ≤ 0.5. What are thesmallest and largest values that ρAC can take?


SDP Formulation

The problem can be formulated as the following problem:

Min/Max ρAC

s.t. −0.2 ≤ ρAB ≤ −0.1

0.4 ≤ ρBC ≤ 0.5

ρAA = ρBB = ρCC = 1 ρAA ρAB ρAC

ρAB ρBB ρBC

ρAC ρBC ρCC

� 0


SDP Formulation

In order to formulate the problem as in standard form, we handle theinequality constraints by augmenting the variable matrix and introducingslack variables, for example

0 12 0 0 0 0 0

12 0 0 0 0 0 00 0 0 0 0 0 00 0 0 1 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0

•

ρAA ρAB ρAC 0 0 0 0ρAB ρBB ρBC 0 0 0 0ρAC ρBC ρCC 0 0 0 0

0 0 0 s1 0 0 00 0 0 0 s2 0 00 0 0 0 0 s3 00 0 0 0 0 0 s4

= ρAB + s1 = −0.1


SDP Formulation

X =

1 ρAB ρAC 0 0 0 0ρAB 1 ρBC 0 0 0 0ρAC ρBC 1 0 0 0 00 0 0 s1 0 0 00 0 0 0 s2 0 00 0 0 0 0 s3 00 0 0 0 0 0 s4

, C =

0 0 12

0 0 0 00 0 0 0 0 0 012

0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0

A1 =

0 12

0 0 0 0 012

0 0 0 0 0 00 0 0 0 0 0 00 0 0 1 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0

A2 =?, A3 =? and A4 =?


Other Applications - QCQP =⇒ SOCP

The popularity of SOCP is also due to the fact that it is a generalized formof convex QCQP (Quadratically Constrained Quadratic Programming).Specifically, consider the following QCQP:

Min xTA0x+ 2bT0 x+ c0s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m

where A0 � 0, Ai � 0 for i = 1, . . . ,m.Note that

t ≥n∑

i=1

x2i ⇐⇒

∣∣∣∣∣∣∣∣∣

∣∣∣∣∣∣∣∣∣

x1...xn

(t− 1)/2

∣∣∣∣∣∣∣∣∣

∣∣∣∣∣∣∣∣∣2

≤ t+ 1

2⇐⇒

x1...xn

(t− 1)/2(t+ 1)/2

∈ Ln+2


Other Applications - QCQP =⇒ SOCP

Therefore, for each i = 1, . . . ,m

xTAix+ 2bTi x+ ci ≤ 0⇐⇒

A1/2i x

−1/2− bTi x− ci/21/2− bTi x− ci/2

∈ Ln+2

QCQP can be equivalently written as

Min u

s.t.

A1/20 x

−1/2− bT0 x+ u/2− c0/21/2− bT0 x+ u/2− c0/2

∈ Ln+2

A1/2i x

−1/2− bTi x− ci/21/2− bTi x− ci/2

∈ Ln+2, i = 1, . . . ,m.


Other Applications - SOCP =⇒ SDP

Ln+1 can be easily embedded into Sn+1+ by observing the fact that[

xt

]∈ Ln+1 ⇐⇒

[t xT

x tIn

]∈ Sn+1

+

Based on this, we will focus on the theorems and algorithms for SDP. Butthis does not mean that SOCP is useless or we should transform SOCPto SDP in any case.


Duality Theory of LP

Theorem (Weak Duality Theorem of LP)If x is primal feasible and y is dual feasible, then cTx ≥ bT y.

Theorem (Strong Duality Theorem of LP)

• If either LP or LD has a finite optimal solution, then so does the otherand they achieve the same optimal objective value.

• If either LP or LD has an unbounded objective value, then the otherhas no feasible solution.

How about the duality theorems of LCoP?


Algorithms for LP

Simplex Method for LP

• Starting from one vertex• Check whether current vertex is optimal or not. If yes, stop.

Otherwise, go to the next step.• Move to a neighboring vertex, go to the previous step.

The complexity of simplex method is not polynomial.

Polynomial-time Algorithms

• Ellipsoid Method• Karmarkar’s Projective Scaling Algorithm• Affine Scaling Algorithm: Primal, Dual and Primal-Dual

How about the algorithms for LCoP?


References

Books• Bertsekas D.P., Nedic A. and Ozdaglar A.E., Convex Analysis and

Optimization, Athena Scientific: Belmont, MA USA 2003• Boyd S. and Vandenberghe L., Convex Optimization, Cambridge University

Press: Cambridge, UK 2004• Fang S. and Puthenpura S., Linear Optimization and Extensions: Theory

and Algorithms, Prentice-Hall Inc.: Englewood Cliffs, NJ USA 1993• Nemirovski A., Lectures on Modern Convex Optimization: Analysis,

Algorithms, and Engineering Applications, Society for Industrial and AppliedMathematics: Philadelphia, PA USA 2001

• Renegar J., A Mathematical View of Interior-point Methods in ConvexOptimization, Society for Industrial and Applied Mathematics: Philadelphia,PA USA 2001

• Handbook of Semidefinite Programming: Theory, Algorithms, andApplications, edited by Wolkowicz H., Saigal R. and Vandenberghe L.,Kluwer Academic Publisher: Norwell, MA USA 2000


References

• Rockafellar R.T., Convex Analysis, Princeton University Press:Princeton, NJ USA 1970

• Wright. S., Primal-Dual Interior-Point Methods, Society for Industrialand Applied Mathematics: Philadelphia, PA USA 1997

Others

• Ye Y., Linear Conic Programming, lecture notes online: http://www.stanford.edu/class/msande314/sdpmain.pdf

• Todd M.J., Semidefinite Programming, lecture notes online:http://people.orie.cornell.edu/~miketodd/cornellonly/or637/or637.html

A very popular general purpose SDP solver, SeDuMi, of Jos F. Sturm canbe found in: http://sedumi.ie.lehigh.edu/


http://www.stanford.edu/class/msande314/sdpmain.pdf

http://www.stanford.edu/class/msande314/sdpmain.pdf

http://people.orie.cornell.edu/~miketodd/cornellonly/or637/or637.html

http://people.orie.cornell.edu/~miketodd/cornellonly/or637/or637.html

http://sedumi.ie.lehigh.edu/

Documents

LCoP Part I – Introduction...Nemirovski A., Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, Society for Industrial and Applied Mathematics: