Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
The Gurobi Optimizer
Bob Bixby
Outline
Gurobi Introduction◦ Company◦ Products◦ Benchmarks
Gurobi Technology◦ Rethinking MIP◦ MIP as a bag of tricks
2© 2010 Gurobi Optimization8-Jul-11
Incorporated July, 2008 Founders: Zonghao Gu, Ed Rothberg, Bob
Bixby Product Releases:◦ Version 1.0: May 2009 Performance roughly equal to CPLEX 11.0◦ Version 2.0: October 2009◦ Version 3.0: April 2010◦ Version 4.0: November 2010◦ Version 4.5: April 2011
Gurobi Optimization
3
The Gurobi Products Gurobi Solvers◦ Mixed-Integer Programming (MIP and MIQP) Deterministic, parallel
◦ Linear and Quadratic Programming Dual and primal simplex Parallel Barrier
APIs◦ Simple command-line interface◦ Python interactive interface◦ C, C++, Java, .NET, Python callable libraries◦ All standard modeling languages
Commercial and Academic Licenses◦ Licensing options and pricing available at
www.gurobi.com
4
Company Business Model
Focus on math programming solvers Our business is helping our customers succeed with
math programming solutions Provide the best possible support from people who
know math programming Be a flexible partner Licensing models that meet user needs Clear, upfront pricing Be the technology leader Build the best available math programming
technology
5
Expanding the Reach of Optimization
6
Expanding the Reach of Optimization Multi-core Parallelism at no extra charge◦ Gurobi was the first
Free Trial Licenses◦ Automated download and license generation◦ 500 variable by 500 constraint size limit
Academic Program◦ Free single-user licenses for academics Automated download and license generation
Pay-as-you-go Licenses◦ Amazon EC2◦ Pay-by-the-day licenses
7
Academic Licenses
Over 8400 free academic licenses issued Over 60% outside the US Over 50% outside the OR community
8
MIP PerformanceBenchmarks
9
Version-to-version improvements:(Geometric mean runtime over ~800 models in our internal model set that take more than 100s to solve)◦ Gurobi 1.0 -> 2.0: 2.2X◦ Gurobi 2.0 -> 3.0: 2.9X (6.4X)◦ Gurobi 3.0 -> 4.0: 1.3X (8.3X)◦ Gurobi 4.0 -> 4.5: 1.8X (14.9X)
Continued improvement in our ability to solve difficult MIP models
MIP Performance –Gurobi Internal Test Set
10
MIP/MIQP Performance –Mittelmann Benchmarks
11
Gurobi 4.5 vs. Competition: Solve times(> 1.0 means Gurobi faster)
BenchmarkCPLEX XPRESS
P=1 P=4 P=12 P=1 P=4 P=12MIPLIB 2010 1.42X 1.19X 1.19X 1.01X 1.21X 1.16XFeasibility 3.57X - - 8.16X - -Infeasibility - - 2.62X - - 3.10XPathological - - 0.90X - - 1.04XMIQP - 2.43X - - 2.22X -
Gurobi Technology
13
Basic MIP Technology:A Fresh Look
14
Start from a clean slate Things have changed:◦ Two particular examples: “Sub-MIP” as a pervasive approach Ubiquitous parallel processing
Building From the Ground Up
15
Key recent insight for heuristics:◦ Can use MIP solver recursively◦ Solve a related model: Hopefully smaller and simpler◦ Examples: Local cuts [Applegate, Bixby, Chvatal & Cook, 2001] Local branching [Fischetti & Lodi, 2003] RINS [Danna, Rothberg, Le Pape, 2005] Solution polishing [Rothberg, 2007]
Sub-MIP As A New Paradigm
16
Rethinking MIPTree Search
17
Each node in branch-and-bound is a new MIP
Branch-and-Bound
Original model, plus several variable fixings
Can view search tree as a tree-of-trees
The nature of a sub-MIP can change dramatically
18
Gurobi MIP search tree manager built to handle multiple related trees◦ Can transform any node into the root node of a
new tree Maintains a pool of nodes from all trees◦ No need to dedicate the search to a single sub-
tree
Tree-of-Trees
19
Each tree has its own relaxation and its own strategies...◦ Presolved model for each subtree◦ Cuts specific to that subtree◦ Pseudo-costs for that subtree only◦ Symmetry detection on that submodel◦ Etc.
Captures structure that is often not visible in the original model
Tree-of-Trees
Heuristics
20
Summary of Heuristics
5 heuristics prior to solving root LP◦ 5 different variable orders, fix variables in this order
15 heuristics within tree (9 primary, several variations)◦ RINS, RINS diving, rounding, fix and dive (LP), fix and
dive (Presolve), fix and dive (simple), Lagrangian approach, pseudo costs, Hail Mary (set objective to 0)
3 solution improvement heuristics◦ Applied whenever a new integer feasible is found
Key tool:◦ Bound strengthening
21
MIP is a bag of tricks!
22
Improving a MIP Solver
Improvements can be plotted on two axes:
Generality
SpeedupBig speedups onlots of models
Modest speedupson lots of models
Big speedups ona few models
23
New Ideas
Generality
Speedup
Experiencegenerally keeps you away from here
Nine out of tenideas end up here
24
Gurobi 2.0
Generality
Dual Simplex improvements
Zero-half cuts
SOS improvements
Node files
Parallel improvements
Speedup
25
Gurobi 3.0
Generality
Speedup
BarrierNetwork cuts
Symmetry
Cut tuning
SubMIP cutsNew presolve red.
26
Disjoint Subtrees
Disjoint Subtrees
Basic principle of branching:◦ Feasible regions for child nodes after a branch should
be disjoint Not always the case Simple example – integer complementarity: x ≤ 10 b y ≤ 10 (1-b) x, y non-negative ints, x ≤ 10, y ≤ 10, b binary Branch on b: x=y=0 feasible in both children
28
Removing Overlap
Simplest way to remove overlap:◦ Modify variable bound in one subtree b=0 child: x = 0, 10 ≥ y ≥ 0 b=1 child: y = 0, 10 ≥ x ≥ 1
29
Overlap present in several models◦ 35 out of 510 models in our test set
Performance impact can be huge◦ Model neos859080 goes from 10000+ seconds
to 0.01s Median improvement for affected models
is ~1%
Performance Impact
30
A Presolve Reduction
A Presolve Reduction
Consider◦ a x + b y = c◦ x, y are integer variables◦ a, b and c are integers, a > 1◦ Assume gcd(a,b) = 1 Otherwise a Euclidean reduction is possible
◦ Observation: Then x(mod b) and y(mod a) are constants. Reduction◦ Substitute y = a z + d (d is easy to compute).◦ z has a smaller search space than y
General application◦ Can easily be extended to general “all integer”
constraints.
32
Pseudo-Cost Adjustment
33
What’s a good branching variable?◦ Superb: fractional variable infeasible in both
branch directions◦ Great: infeasible in one direction◦ Good: both directions move the objective
Expensive to predict which branches lead to infeasibility or big objective moves◦ Strong branching Truncated LP solve for every possible branch at
every node Rarely cost effective◦ Need a quick estimate
Pseudo-Costs
34
Use historical data to predict impact of a branch:◦ Record costx = obj / x for each branch Need a scheme for infeasible branches too◦ Store results in a pseudo-cost table Two entries per integer variable Average (or max) down cost Average (or max) up cost
◦ Use table to predict cost of a future branch
Pseudo-Costs
35
What do you do when there is no history?◦ E.g., at the root node
Initialize pseudo-costs [Linderoth & Savelsbergh, 1999]◦ Always compute up/down cost (using strong
branching) for new fractional variables Initialize pseudo-costs for every fractional
variable at root Reliability branching [Achterberg, Koch, &
Martin, 2002]◦ Don’t rely on historical data until pseudo-cost
for a variable has been recomputed r times
Pseudo-Cost Initialization
36
Gurobi 4.5 adjusts pseudo-costs in several ways◦ Implied pseudo-cost bounds◦ Ancestor adjustment
Pseudo-Cost Adjustment
37
Consider constraint:◦ ∑ xi = 1
Computed objective bounds:◦ x1=1 -> obj ≥ 100◦ x2=0 -> obj ≥ 110
Stronger bound for x1=1:◦ x1=1 -> x2=0 -> obj ≥ 110
In general:◦ If x=a -> y=b, then objx=a ≥ objy=b
Often violated:◦ Strong branching uses an iteration limit
Implied Pseudo-Cost Bounds
38
Consider ∑ xi = 1 again Computed objective bounds:◦ x1=0 -> obj ≥ 100◦ xj=1 -> obj ≥ 110 for all j != 1
Stronger bound for x1=0:◦ x1=0 -> xj=1 for some j != 1 -> obj ≥ 110
In general:◦ If xi=a -> xj=b for some j != i◦ Then objxi=a ≥ min(objxj=b)
Implied Pseudo-Cost Bounds
39
40
Objective move isn’t entirely the result of most recent branch variable
Ancestor Adjustment
Depends on ancestors as well
Particularly true for infeasible nodes
Adjust pseudo-costs for ancestor branches
41
Need an adjustment strategy
Ancestor Adjustment
Empirically, adjustment should decrease as you move up the tree
Our approach: Exponential
backoff ½ for parent, ¼ for
grandparent, etc.
Product Roadmap
Projected Gurobi Roadmap
Version 4.X – November 2011◦ LP & MIP performance release
Version 5.0 – April 2012◦ SOCP and MISOCP◦ MIP performance enhancements
Version 6.0◦ New features under discussion◦ MIP performance enhancements
43
Thank You