Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Extended Mathematical Programming: Structure andSolution
Michael C. Ferris, Steven Dirkse, Jan Jagla, Alex Meeraus
University of Wisconsin, Madison
FOCAPO 2008: July 1, 2008
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 1 / 25
Mathematical programming: modeling
Optimization models improve understanding of underlying systemsand facilitate operational/strategic improvements
Modeling systems enable application interfacing, prototyping ofoptimization capability
Data (collection) remains bottleneck in many applicationsI Tools interface to databases, spreadsheets, Matlab
Problem format is old/traditional
minx
f (x) s.t. g(x) ≤ 0, h(x) = 0
I Support for integer, sos, semicontinuous variablesI Limited support for logical constructsI Support for complementarity constraints
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 2 / 25
Complementarity Problems (MCP)
p represents prices, x represents activity levels
System model: given prices, (agent) i determines activities xi
Gi (xi , x−i , p) = 0
x−i are the decisions of other agents.
Walras Law: market clearing
0 ≤ S(x , p)− D(x , p) ⊥ p ≥ 0
Key difference: optimization assumes you control the complete system
Complementarity determines what activities run, and who produceswhat
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 3 / 25
World Bank Project (Uruguay Round)
24 regions, 22 commoditiesI Nonlinear complementarity
problemI Size: 2200 x 2200
Short term gains $53 billion p.a.I Much smaller than previous
literature
Long term gains $188 billion p.a.I Number of less developed
countries loose in short term
Unpopular conclusions - forcedconcessions by World Bank
Region/commodity structure notapparent to solver
Application: Uruguay Round• World Bank Project with
Harrison and Rutherford• 24 regions, 22 commodities
– 2200 x 2200 (nonlinear)• Short term gains $53 billion p.a.
– Much smaller than previous literature
• Long term gains $188 billion p.a.– Number of less developed
countries loose in short term• Unpopular conclusions – forced
concessions by World Bank
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 4 / 25
MPEC: complementarity constraints
minx ,s
f (x , s)
s.t. g(x , s) ≤ 0,0 ≥ s ⊥ h(x , s) ≤ 0
g , h model “engineering” expertise: finite elements, etc
⊥ models complementarity, disjunctions
Complementarity “⊥” constraints available in AMPL and GAMS
NLPEC: use the convert tool to automatically reformulate as aparameteric sequence of NLP’s
Solution by repeated use of standard NLP softwareI Problems solvable, local solutions, hard
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 5 / 25
Use of complementarity
Pricing electricity markets and optionsVideo games: model contact problems
I Friction only occurs if bodies are in contactStructure design
I how springy is concreteI optimal sailboat rig design
Computer/traffic networksI The price of anarchy measures difference between “system optimal”
(MPEC) and “individual optimization” (MCP)
What else can we model via CP?
min (G (x),H(x)) ≤ y
min(F 1(x),F 2(x), . . . ,Fm(x)
)= 0
kth-largest(F 1(x),F 2(x), . . . ,Fm(x)
)= 0
Switch on/off: g(x)h(x) ≤ 0, h(x) ≥ 0
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 6 / 25
So what’s my point?
Can solve practical models due to availability of good software (e.g.PATH)
Design (optimization) under our control
Uncertainties treated via “scenarios”
Competition/market effects beyond control of designer treated bycomplementarity
Complementarity facilitates modeling of competition, nonsmoothnessand “switching”
Large scale models involving complementarity now solvable
Do you (or should you) care?
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 7 / 25
EMP(i): Embedded models
Model has the format:
minx
f (x , y)
s.t. g(x , y) ≤ 0 (⊥ λ ≥ 0)
H(x , y , λ) = 0 (⊥ y free)
Difficult to implement correctly, particularly when multipleoptimization models present
Can do automatically - simply annotate equations
EMP tool automatically creates an MCP
∇x f (x , y) + λT∇g(x , y) = 0
0 ≤ −g(x , y) ⊥ λ ≥ 0
H(x , y , λ) = 0
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 8 / 25
Recall: World Bank Example
Application: Uruguay Round• World Bank Project with
Harrison and Rutherford• 24 regions, 22 commodities
– 2200 x 2200 (nonlinear)• Short term gains $53 billion p.a.
– Much smaller than previous literature
• Long term gains $188 billion p.a.– Number of less developed
countries loose in short term• Unpopular conclusions – forced
concessions by World Bank
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 9 / 25
Walras meets WardropIncreasing LaborDemand
Increasing
Housing
What is the effect on housing prices of increasing capacity on the red arcs?
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 10 / 25
Features
We buy a house to “optimize” some measureI Price driven by marketI We compete against each other
Driver’s choose routes to “optimize” travel timeI Choices affect congestionI Your choice affects me!
Production processes are “optimized”
But the road designer does not control any of these!
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 11 / 25
Simplified AGE model
(P) : miny≥0
cT y
s.t. Ay ≥ d (⊥ p ≥ 0)
(C ) : maxd≥0
u(d)
s.t. pTd ≤ I
In equilibrium, the optimal demand d from (C) will be the demand in(P), and the sales price p in (C) will be the marginal price onproduction from (P)
Complementarity conditions of (P) and (C) have both primal anddual variables
Optimization models linked by variables and multipliers
Equilibrium problem solvable as a complementarity problem
Can add “other features” such as taxation, transportation, tolls.
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 12 / 25
The Hollywood perspective: game theory
Nash Games: x∗ is a Nash Equilibrium if
x∗i ∈ arg minxi∈Xi
`i (xi , x∗−i , q),∀i ∈ I
x−i are the decisions of other players.
Quantities q given exogenously, or via complementarity:
0 ≤ H(x , q) ⊥ q ≥ 0
EMP reformulates automatically for appropriate solvers
Applications: Discrete-Time Finite-State Stochastic Games.Specifically, the Ericson & Pakes (1995) model of dynamiccompetition in an oligopolistic industry.
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 13 / 25
Key point: models generated correctly solve quicklyHere S is mesh spacing parameter
S Var rows non-zero dense(%) Steps RT (m:s)
20 2400 2568 31536 0.48 5 0 : 0350 15000 15408 195816 0.08 5 0 : 19100 60000 60808 781616 0.02 5 1 : 16200 240000 241608 3123216 0.01 5 5 : 12
Convergence for S = 200 (with new basis extensions in PATH)
Iteration Residual
0 1.56(+4)1 1.06(+1)2 1.343 2.04(−2)4 1.74(−5)5 2.97(−11)
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 14 / 25
Security Constrained Optimal Power Flow
least cost energy dispatch (uses prices and quantities)
subject to: physical grid constraintsI power flow equations (Ohm’s law)I line and bus voltage limits
and contingency constraints
and “generators dispatch to optimize profits” at given prices(supply/price bid curves)
Leader/follower game: Stackleberg
Supply chains with “market leader”
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 15 / 25
EMP(ii): Heirarchical models
Bilevel programs:
minx ,y
f (x , y)
s.t. g(x , y) ≤ 0,y solves min
sv(x , s) s.t. h(x , s) ≤ 0
Model as:model bilev /deff,defg,defv,defh/;plus empinfo: bilevel y min v defh
EMP tool automatically creates the MPEC
Note that heirarchical structure is available to solvers fordecomposition approaches
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 16 / 25
EMP(iii): Other new types of constraints
range constraints L ≤ Ax − b ≤ U
indicator constraints
disjunctive programming
soft constraints
rewards and penalties
robust programming (probability constraints, stochastics)
f (x , ξ) ≤ 0,∀ξ ∈ U
conic programming aTi x − bi ∈ Ki
Some constraints can be reformulated easily, others not!
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 17 / 25
CVaR constraints: mean excess dose (radiotherapy)VaR, CVaR, CVaR+ and CVaR-
Loss
Fre
qu
en
cy
1111 −−−−αααα
VaR
CVaR
Probability
Maximumloss
Move mean of tail to the left!
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 18 / 25
Example: Robust Linear Programming
Data in LP not known with certainty:
min cT x s.t. aTi x ≤ bi , i = 1, 2, . . . ,m
Suppose the vectors ai are known to be lie in the ellipsoids
ai ∈ εi := {ai + Piu : ‖u‖2 ≤ 1}
where Pi ∈ Rn×n (and could be singular, or even 0).Conservative approach: robust linear program
min cT x s.t. aTi x ≤ bi , for all ai ∈ εi , i = 1, 2, . . . ,m
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 19 / 25
Robust Linear Programming as SOCP/ENLP
The constraints can be rewritten as:
bi ≥ sup{
aTi x : ai ∈ εi
}= aT
i x + sup{
uTPTi x : ‖u‖2 ≤ 1
}= aT
i x +∥∥∥PT
i x∥∥∥
2
Thus the robust linear program can be written as
min cT x s.t. aTi x +
∥∥∥PTi x∥∥∥
2≤ bi , i = 1, 2, . . . ,m
min cT x +m∑
i=1
ψC (bi − aTi x ,PT
i x)
where C represents the second-order cone. Our extension allows automaticreformulation and solution (as SOCP) by Mosek or Conopt.
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 20 / 25
EMP(iv): Extended nonlinear programs
minx∈X
f0(x)+θ(f1(x), . . . , fm(x))
Examples of different θ
least squares, absolute value, Huber functionSolution reformulations are very differentHuber function used in robust statistics.
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 21 / 25
Choices for θ
infx∈X
f0 + θ[f (x)], θ(u) = supy∈Y{yTu − k(y)}
θ is convex with values in (−∞,+∞]; may be nonsmooth
L2: k(u) = 14λu2, Y = (−∞,+∞)
L1: k(u) = 0, Y = [−ρ, ρ]
L∞: k(u) = 0, Y = ∆, unit simplex in Rm+
Linear-quad (Huber 1981): k(u) = 14λu2, Y = [−ρ, ρ]
Second order cone constraint: k(y) = 0, Y = C ◦
The new feature here is implementation and solution within theGAMS modeling language framework, which produces a tool usablewithout advanced knowledge in convex analysis and withoutcumbersome “hand tailoring” to accommodate different penalizations[Ferris, Dirkse, Jagla, and Meeraus 2008]
This makes the theoretical benefits accessible to users from a widevariety of different fields (examples later)
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 22 / 25
Solution Procedures
Solution uses reformulation. One way (using e.g. PATH):
0 ∈ ∇xL(x , u) + NX (x)0 ∈ −∇uL(x , u) + NY (u)
NX (x) is the normal cone to the closed convex set X at x .
EMP: allows “annotation” of constraints to facilitate library ofdifferent θ functions to be applied
EMP tool automatically creates an MCP
Available!
To do: extend solvers to exploit X and Y beyond simple bound sets
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 23 / 25
Conclusions
Model is clearer, structure available to solver
Large scale complementarity problems reliably solvable
Complementarity constraints within optimization problems
Extended Mathematical Programming available within a modelingsystem
System can easily formulate and solve second order cone programs,risk measures, robust optimization, soft constraints via piecewiselinear penalization (with strong supporting theory)
Embedded optimization models automatically reformulated forappropriate solution engine
Enhance library of (implemented) θ functions (what do you want?)
Exploit structure in solvers
Extend application usage of complementarity solvers
Extend complementarity solvers to VI solvers
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 24 / 25
The good, the bad and the ugly
Formulate extensions, convert automatically, provide structure, solvewith specific solvers, underlying theory
Uncertainty, global properties, large-scale issues
Reformulations may be suboptimal, grid solution strategies may notbe reproduceable, requires “knowledgeable” user (for annotations),needs exercising!
Michael Ferris (University of Wisconsin) EMP FOCAPO 2008: July 1, 2008 25 / 25