PICOS: A python interface to conic optimization solverspage.math.tu-berlin.de/~sagnol/papers/ISMP_picos.pdf · an interface to several optimization solvers: (currently, CVXOPT, SCIP

PICOS: A python interface to conicoptimization solvers

Guillaume SAGNOL

Zuse Institut Berlin (ZIB)

ISMP 2012

Outline

1 Intro and Motivation

2 Tutorial

3 Examples

Outline

2 Tutorial

3 Examples

What is PICOS ?

PICOS isa python packagean interface to several optimization solvers: (currently,CVXOPT, SCIP (zibopt), CPLEX, MOSEK, SMCP)a user-friendly modelling languageparticularly suited for SDP and SOCP

PICOS is nota solvera stand-alone interface

What is PICOS ?

PICOS isa python packagean interface to several optimization solvers: (currently,CVXOPT, SCIP (zibopt), CPLEX, MOSEK, SMCP)a user-friendly modelling languageparticularly suited for SDP and SOCP

PICOS is nota solvera stand-alone interface

A “Hello world” example

Find the largest integer smaller than 5.2

Import the package and instanciate a problem>>> import picos as pic>>> prob = pic.Problem()

Add a scalar, integer variable>>> x = prob.add_variable(’x’,1, vtype=’integer’)

Add a constraint, and set the objective value>>> prob.add_constraint(x>> prob.set_objective(’max’,x)

A “Hello world” example (continued)

Display the problem>>> print prob---------------------optimization problem (MIP):1 variables, 1 affine constraintsx : (1, 1), integer

maximize xsuch thatx < 5.2

---------------------

Solve the problem (with the ZIB optimization suite)>>> sol = prob.solve(solver=’zibopt’,verbose=0)

Display the solution>>> print x #optimal value of x5.0

---------------------

Motivation

Personal work on Optimal Experimental Design:many SOCP and SDP.No equivalent for YALMIP under Python.Transforming optimization problems to a canonicalform is painful and time-consuming

Goal of PICOSQuick implementation of new modelsBenchmark between solversEducation

Motivation

Personal work on Optimal Experimental Design:many SOCP and SDP.No equivalent for YALMIP under Python.Transforming optimization problems to a canonicalform is painful and time-consuming

Goal of PICOSQuick implementation of new modelsBenchmark between solversEducation

Features

Interfaced solvers and types of problems PICOS canhandle:

cvxopt (LP, SOCP, SDP, GP)smcp (LP, SOCP, SDP)mosek (LP, MIP, (MI)SOCP, convex QCQP, MIQP)cplex (LP, MIP, (MI)SOCP, convex QCQP, MIQP)zibopt (soplex + scip : LP, MIP, MIQP, general QCQP).

http://abel.ee.ucla.edu/cvxopt/http://abel.ee.ucla.edu/smcp/http://www.mosek.comhttp://www.ibm.com/software/integration/optimization/cplex-optimizer/http://zibopt.zib.de/http://soplex.zib.de/http://scip.zib.de/

Outline

2 Tutorial

3 Examples

Variables

Scalars, vectors, matrices, list and dict of variables>>> t = prob.add_variable(’t’,1) #a scalar>>> x = prob.add_variable(’x’,4) #a column vector>>> Y = prob.add_variable(’Y’,(2,4)) #a matrix>>> Z = []>>> for i in range(5): # a list of 5 matrices... Z.append( prob.add_variable(’Z[%d]’%i ,(4,2)) )

Attributes vtype,size,name and value.>>> Y# variable Y:(2 x 4),continuous #>>> x.vtype’continuous’>>> x.vtype=’integer’>>> x# variable x:(4 x 1),integer #>>> x.size(4, 1)>>> Z[2].name’Z[2]’

Variables

Scalars, vectors, matrices, list and dict of variables>>> t = prob.add_variable(’t’,1) #a scalar>>> x = prob.add_variable(’x’,4) #a column vector>>> Y = prob.add_variable(’Y’,(2,4)) #a matrix>>> Z = []>>> for i in range(5): # a list of 5 matrices... Z.append( prob.add_variable(’Z[%d]’%i ,(4,2)) )

Attributes vtype,size,name and value.>>> Y# variable Y:(2 x 4),continuous #>>> x.vtype’continuous’>>> x.vtype=’integer’>>> x# variable x:(4 x 1),integer #>>> x.size(4, 1)>>> Z[2].name’Z[2]’

Affine Expressions

Expressions are formed with variables andoverloaded operators

>>> Z[0]+Z[3]# (4 x 2)-affine expression: Z[0] + Z[3] #>>> Z[0]+Y.T #note the transposition property .T# (4 x 2)-affine expression: Z[0] + Y.T #

Now, assume that A[0],...,A[4] are 2x4-matrices>>> A[0] * Z[0] + A[4] * Z[4]# (2 x 2)-affine expression: [ 2 x 4 MAT ]*Z[0]

+ [ 2 x 4 MAT ]*Z[4] #

For a nicer display, declare A as a parameter>>> A = pic.new_param(’A’,A)>>> A[0] * Z[0] + A[4] * Z[4]# (2 x 2)-affine expression: A[0]*Z[0] + A[4]*Z[4] #

Affine Expressions

+ [ 2 x 4 MAT ]*Z[4] #

Affine Expressions

+ [ 2 x 4 MAT ]*Z[4] #

Overloaded operatorsMultiplications

>>> A[0] * Z[0]# (2 x 2)-affine expression: A[0]*Z[0] #>>> Z[0] * A[0]# (4 x 4)-affine expression: Z[0]*A[0] #>>> A[1] * Z[0] * A[2]# (2 x 4)-affine expression: A[1]*Z[0]*A[2] #>>> 5*Y# (2 x 4)-affine expression: 5*Y #>>> ( A[3] | Y )#matrix scalar product: (A|B)=trace(A*B.T)# (1 x 1)-affine expression: 〈 A[3] | Y 〉 #

Slicing>>> Y[1,:] #2d row of Y# (1 x 4)-affine expression: Y[1,:] #>>> x[-1] #last element of x# (1 x 1)-affine expression: x[-1] #

Concatenation>>> ( ( A[0].T*A[0] & x ) // #vertical (//)... ( x.T & t ) ) #and horizontal (&) concat.# (5 x 5)-affine expression: [A[0].T*A[0],x;x.T,t] #

Abstract sums

You can take the advantage of python syntax tocreate sums of affine expressions:

>>> sum([A[i]*Z[i] for i in range(5)])# (2 x 2)-affine expression: A[0]*Z[0] + A[1]*Z[1] +

A[2]*Z[2] + A[3]*Z[3] + A[4]*Z[4] #

To avoid long string representations, use the functionpicos.sum:

>>> pic.sum([A[i]*Z[i] for i in range(5)],... ’i’, #string representation of the iterator... ’[5]’) #set to which the iterator belongs# (2 x 2)-affine expression: Σ_{i in [5]} A[i]*Z[i] #

Summing over several indices is possible>>> ij_pairs=[(0,1),(2,3),(4,2),(0,3),(1,0)]>>> pic.sum([A[i][:,j]-x[j] for i,j in ij_pairs],... (’i’,’j’),... ’pairs’)# (2 x 1)-affine expression: Σ_{i,j in pairs} A[i][:,j]

- |x[j]| #

Abstract sums

A[2]*Z[2] + A[3]*Z[3] + A[4]*Z[4] #

- |x[j]| #

Abstract sums

A[2]*Z[2] + A[3]*Z[3] + A[4]*Z[4] #

- |x[j]| #

objective function

We define the objective function of a problem with thefunctionset_objective(’max’|’min’|’find’,Expression)

>>> prob.set_objective(’max’,( A[0] | Y )-t)>>> print prob---------------------optimization problem (MIP):59 variables, 0 affine constraints

Z : list of 5 variables, (4, 2), continuoust : (1, 1), continuousY : (2, 4), continuousx : (4, 1), integer

maximize 〈 A[0] | Y 〉 -tsuch that[]

---------------------

Constraints

Linear (in)equalities are formed with the syntax

Expression1 [’>’|’>> (1|x) < 2 #sum of the x[i] less or equal than 2# (1x1)-affine constraint: 〈 |1| | x 〉 < 2.0 #

>>> pic.sum([A[i]*Z[i] for i in range(5)],’i’,’[5]’) == 0... #A 2x2 equality. The RHS is the all-zero matrix# (2x2)-affine constraint: Σ_{i in [5]} A[i]*Z[i] = |0| #

Quadratic constraints

Quadratic constraints are entered in the same fashion.For example,

t2 ≥ 2t − 1 + x1x2can be formed as:

>>> t**2 > 2*t - 1 + x[1]*x[2]#Quadratic constraint -t**2 + 2.0*t -1 + x[1]*x[2] < 0 #

WarningAt this stage, convexity of the constraint is notcheckedOnly the scalar quadratics are handled

Another example:

[t ,1] A1 x ≤ 3〈1,Y 〉>>> (t & 1) * A[1] * x < 3*(1|Y)#Quadratic constraint [t,1]*A[1]*x -(3.0*〈 |1| | Y 〉 )< 0 #

Quadratic constraints

Quadratic constraints are entered in the same fashion.For example,

t2 ≥ 2t − 1 + x1x2can be formed as:

>>> t**2 > 2*t - 1 + x[1]*x[2]#Quadratic constraint -t**2 + 2.0*t -1 + x[1]*x[2] < 0 #

WarningAt this stage, convexity of the constraint is notcheckedOnly the scalar quadratics are handled

Another example:

[t ,1] A1 x ≤ 3〈1,Y 〉>>> (t & 1) * A[1] * x < 3*(1|Y)#Quadratic constraint [t,1]*A[1]*x -(3.0*〈 |1| | Y 〉 )< 0 #

Second Order Cone Constraints

Euclidean (resp. Frobenius) norm of an affine expression:

abs(Expression)

Two types of SOC constraints can be entered withPICOS:

(i) Ice-cream cone constraint ‖x‖ ≤ t ;(ii) Rotated cone constraint ‖x‖2 ≤ tu, t ≥ 0,

where t and u are scalar affine expressions and x is amultidimensional affine expression.

WarningWhen a constraint of the form abs(x)**2 < t*u ispassed, it is automatically interpreted as a constraint oftype (ii), i.e., it is implicitely assumed that t isnonnegative.

abs(Expression)

Second Order Cone Constraints: Examples

>>> abs(x) < (2|x-1)# (4x1)-SOC constraint: ||x|| < 〈 |2.0| | x -|1| 〉 #

>>> abs(Y+Z[0].T) < t+4# (2x4)-SOC constraint: ||Y + Z[0].T|| < t + 4. #

>>> abs(Z[1][:,0])**2 < (2*t-3)*(x[2]-x[-1])# (4x1)-Rotated SOC constraint:

||Z[1][:,0]||^2 < ( 2.0*t -3.0)( x[2] -(x[-1])) #

>>> 1 < (t-1)*(x[2]+x[3]) #1 is understood as ||[1]||**2# (1x1)-Rotated SOC constraint:

1.0 < ( t -1.0)( x[2] + x[3]) #

Semidefinite Constraints

Linear matrix inequalities (LMI) can be passed to picoswith the operators ’’.For example, the LMI

3∑i=0

xiATi Ai � AT4 A4

>>> pic.sum([x[i]*A[i].T*A[i] for i in range(4)],... ’i’,’0...3’... ) >> A[4].T*A[4]# (4x4)-LMI constraint Σ_{i in 0...3} x[i]*A[i].T*A[i]

� A[4].T*A[4] #

I It is possible to create symmetric matrix variables withthe option vtype=symmetric

I Note the difference betweenX>0 (elements of X are nonnegative)X>>0 (matrix X is positive semidefinite)

3∑i=0

xiATi Ai � AT4 A4

� A[4].T*A[4] #

3∑i=0

xiATi Ai � AT4 A4

� A[4].T*A[4] #

Adding constraints to the problem

Single constraints can be added with

add_constraint(Constraint)

For example>>> for i in range(1,5):... prob.add_constraint(Z[i]==Z[i-1]+Y.T)>>> print prob---------------------[...]

maximize 〈 A[0] | Y 〉 -tsuch thatZ[1] = Z[0] + Y.TZ[2] = Z[1] + Y.TZ[3] = Z[2] + Y.TZ[4] = Z[3] + Y.T

---------------------

To access a constraint:>>> prob.get_constraint(2) #constraints are numbered from 0# (4x2)-affine constraint: Z[3] = Z[2] + Y.T #

Single constraints can be added with

add_constraint(Constraint)

---------------------

Single constraints can be added withadd_constraint(Constraint)

---------------------

Groupping constraints

Lists of constraints can be added with

add_list_of_constraints(Constraints,iterator,set)

>>> prob.remove_all_constraints()>>> prob.add_constraint(Y>0) #a single constraint>>> prob.add_list_of_constraints([Z[i]==Z[i-1]+Y.T... for i in range(1,5)],... ’i’,’1...4’)>>> print prob---------------------[...]

maximize 〈 A[0] | Y 〉 -tsuch thatY > |0|Z[i] = Z[i-1] + Y.T for all i in 1...4

---------------------

Now, prob.get_constraint((1,)) will return our list ofconstraints, and>>> prob.get_constraint((1,2)) #or prob.get_constraint(3)# (4x2)-affine constraint: Z[3] = Z[2] + Y.T #

Lists of constraints can be added with

add_list_of_constraints(Constraints,iterator,set)

---------------------

write a problem to a file

It is possible to write the problem into a file with

write_to_file()

A simple exampleimport picos as picprob = pic.Problem()y = prob.add_variable(’y’,1, vtype=’integer’)x = prob.add_variable(’x’,1)prob.add_constraint(x>1.5)prob.add_constraint(y-x>0.7)prob.set_objective(’min’,y)

#now write the problem to a .lp file...prob.write_to_file(’helloworld.lp’)

It is possible to write the problem into a file with

write_to_file()

A simple exampleimport picos as picprob = pic.Problem()y = prob.add_variable(’y’,1, vtype=’integer’)x = prob.add_variable(’x’,1)prob.add_constraint(x>1.5)prob.add_constraint(y-x>0.7)prob.set_objective(’min’,y)

#now write the problem to a .lp file...prob.write_to_file(’helloworld.lp’)

\* file helloworld.lp generated by picos*\Minimizeobj : 1 ySubject Toin0 : -1 y+ 1 x

Solving a problem

Problems are solved with the function

Problem.solve(**options)

Some useful options:solver=None [’cvxopt’|’zibopt’|’mosek’|’cplex’|’smcp’]

verbose = 1 [0|1|2]

tol = 1e-8

gaplim = 1e-4

maxit = None

timelimit = None

Optimal variables can be accessed withVariable.value

Optimal dual variables can be accessed withConstraint.dual

Outline

2 Tutorial

3 Examples

MAXCUT SDP

maximizeX∈S|V |

14〈L,X 〉

subject to diag(X ) = 1X � 0

G = load_some_graph()N = G.number_of_nodes()

maxcut = pic.Problem()X=maxcut.add_variable(’X’,(N,N),’symmetric’)L=pic.new_param(’L’,1/4.*nx.laplacian(G)) #Laplacian

maxcut.add_constraint(pic.tools.diag_vect(X)==1)maxcut.add_constraint(X>>0) #X positive semidefinitemaxcut.set_objective(’max’,L|X)

maxcut.solve()

MAXCUT SDP

maximizeX∈S|V |

14〈L,X 〉

subject to diag(X ) = 1X � 0

G = load_some_graph()N = G.number_of_nodes()

maxcut = pic.Problem()X=maxcut.add_variable(’X’,(N,N),’symmetric’)L=pic.new_param(’L’,1/4.*nx.laplacian(G)) #Laplacian

maxcut.add_constraint(pic.tools.diag_vect(X)==1)maxcut.add_constraint(X>>0) #X positive semidefinitemaxcut.set_objective(’max’,L|X)

maxcut.solve()

Exact A-optimal design: MISOCP

Given observationmatrices A1, . . . ,As,allocate N trials tothe s experiments.

mint∈Rsn∈Ns

Z1,...,Zs∈Rli×m

∑i

ti

s.t.s∑

i=1

AiZi = I

∀i ∈ [s], ‖Zi‖2F ≤ ni ti ,s∑

i=1

ni = N.

exdesign=pic.Problem()A=pic.new_param(’A’,load_list_of_obs_mat())s=len(A)m=A[0].size[0]N =pic.new_param(’N’,20) #number of trialsI =pic.new_param(’I’,cvx.spdiag([1.]*m))Z=[exdesign.add_variable(’Z[%d]’%i,A[i].T.size) for i in range(s)]n=exdesign.add_variable(’n’,s, vtype=’integer’)t=exdesign.add_variable(’t’,s)

#define the constraints and objective functionexdesign.add_list_of_constraints(

[abs(Z[i])**2

mint∈Rsn∈Ns

Z1,...,Zs∈Rli×m

∑i

ti

s.t.s∑

i=1

AiZi = I

∀i ∈ [s], ‖Zi‖2F ≤ ni ti ,s∑

i=1

ni = N.