Improvements to Benders' decomposition: systematic ...dinamico2.unibg.it/icsp2013/doc/ps/3-Sara Lumbreras.pdf · July 2013 Improvements to Benders' decomposition: systematic classification

July 2013

Improvements to Benders' decomposition: systematic classification

and performance comparison in a Transmission Expansion Planning

problem

Sara Lumbreras & Andrés Ramos

2 Instituto de Investigación Tecnológica

Escuela Técnica Superior de Ingeniería ICAI

Sara Lumbreras, July 2013

Motivation

A quick introduction

Master problem improvement techniques

Subproblem improvement techniques

Case Study: Transmission Expansion Planning

Results

Conclusions

Agenda

Motivation




It was first proposed in 1962

In its 50 years of history it has been applied to very diverse fields:

Scheduling

Routing (e.g. traveling salesman) & vehicle assignment

Computer network design

Capacity allocation in telecommunications networks

Manufacturing system design

Portfolio optimization

Specially, in Power systems

Generation, Transmission & Distribution Expansion Planning [Pereira et al, ‘A

decomposition approach to automated generation/ transmission expansion planning’, 1985]

Hydrothermal co-ordination

Unit Commitment

Many improvements to the basic strategy have been proposed in an uncoordinated way, so they are not easily accessible or related to the cases where they can be useful

Benders, ‘Partitioning procedures for solving mixed-variables programming problems’, 1962

Motivation

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:

TEPResults Conclusions




This work aims at filling this gap:

Classifies the improvements to Benders’ decomposition that appear in the literature

Proposes new improvements to add to the ones in the literature

Links these methodologies to the cases where they can be useful

Compares their performance in a particularly relevant case study based on Transmission Expansion Planning

Given its characteristics this problem has been extensively solved with Benders’ decomposition

Motivation (II)

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:






Benders’ decomposition is applied to a two-stage stochastic (linear) problem

First-stage decisions (also known as investment decisions) are taken before an uncertain event occurs

Second-stage decisions (also known as operation decisions or recourse decisions) are adjusted after the uncertainty has been revealed

Second-stage scenarios can be solved independently

Both stages are coupled through the tender constraints

First-stage decisions “complicate” the resolution of the problem


1st stage 2nd stage Stochastic

scenarios

Constraints of the 1st stage

Constraints that link both

stages (tender constraints)

Constraints of the 2nd stage

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Benders’ decomposition divides the two-stage stochastic linear programming problem in two parts (master problem & subproblems) that are solved iteratively until convergence

The process builds an increasingly accurate piecewise linear approximation of the recourse problem (Benders’ cuts)

A quick introduction (cont’ed)

lower

bound

upper

bound

Complete problem

Master problem

Subproblem (solved independently for each scenario)

iterations

current iteration

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Feasibility cuts

When a certain 1st-stage solution is infeasible in the subproblem, a feasibility cut is generated with the aim of eliminating solutions that do not abide the constraints

Usually, the sum of infeasibilities is minimized

The resulting feasibility cut has the form:

A quick introduction (cont’ed)

infeasibility

1lδ =

0lδ =

optimality cut

feasibility cut

For uniform notation

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





This method is likely to yield time savings in the following cases:

First-stage variables complicate the resolution of the problem

Decoupling the resolution of 2nd stage scenarios is specially important

Computational time taken by this method is related to the number of complicating variables

Therefore, it is likely to yield time savings if the number of complicating variables is small

The master problem and the subproblem have different natures (and hence it would be convenient to solve them with different methods, e.g. NLP / MIP vs LP)

Both conditions apply to TEP

Scope of Benders’ decomposition

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Improvement techniques respond to the need to reduce either:

Master problem solution time (solution time can be long because of size, integral variables or a large number of cuts), or

Subproblem solution time (solution time can be long because of a large number of scenarios or 2nd stage conditions)

Guides for their suitability will be provided

However, the practical benefit achieved will have to be assessed on a case-by-case basis

Most of them involve trade-offs that must be assessed individually

A case study based on Transmission Expansion Planning will be presented

Classification of improvement techniques

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


Master problem improvements




Master problem relaxations

Binary variables complicate the resolution, so solving the LP relaxation is much quicker

It yields valid Benders’ cuts

Linear-first approaches solve first the relaxed problem

Recently, [Lumbreras & Ramos, ‘Optimal design of an offshore wind farm applying

decomposition strategies’, 2012] proposed a progressive discretization of variables to improve convergence (semi-relaxed cuts)

Improvements that modify the solution technique (I)

upper bound

lower bound

variable

discretization

convergence

Can be useful if the

linearized problem can

be solved much quicker

and the integrality gap is

small

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Sub-optimal master problem solutions

Early terminations of the master problem might improve convergence

Using any feasible solution [Fortz & Poss, An improved Benders decomposition applied to a

multi-layer network design problem, 2008]

Rounding linearized solutions [Costa et al, ‘Accelerating Benders decomposition with heuristic master solutions’, 2012]

(feasibility must be checked)

Alternative strategies to find master proposals

Non-classical optimization techniques (e.g., metaheuristics) can be applied to find near-optimal solutions in affordable times

[Poojari & Beasley,’ Improving Benders’ decomposition using a Genetic Algorithm’, 2009]

Improvements that modify the solution technique (II)

gap

time

Can be useful if the sub-

optimal solution is not far

from the optimal one and is

obtained in a reduced time

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Box-step method (introduction of additional constraints):

Ideally they are conditions that should be met at the optimum, so they only eliminate not useful zones of the feasible region

From expert opinion

Data mining

If the additional constraint is active at the optimal solution the constraint should be relaxed by an specified amount (the step) and the problem resolved.

Use of a more suitable solution technique

E.g. Constraint programming and logic-based methods [Benoist et al, ‘Constraint programming contribution to Benders’ decomposition’, 2002]

In many cases, complex problems include many logical constraints that make use of auxiliary binary variables

This greatly complicates the problem

There are techniques that have been specially developed for these problems, where the logical constraints are included explicitly (e.g. LOGMIP)

Improvements that modify the solution technique (III)

Can be useful if there is a technique that

is better suited to the specific problem

Can be useful if these constraints

are available

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Extracting non-dominated cuts / Pareto-optimal cuts [Sherali & Fraticelli, ‘On generating maximal nondominated Benders cuts’, 2011]

A cut or constraint dominates another if any evaluation of first stage decisions is larger than or equal to the previous one

Generating covering cuts [Saharidis et al, Accelerating benders method using covering cut bundle generation, 2010]

Generating cuts so that they carry the maximum amount of information possible

They include the maximum possible number of 1st-stage variables

Removing inactive cuts [Marin & Salmeron, ‘Electricity capacity expansion under uncertain demand: decomposition approaches’, 1998]

Or dynamically defining the master problem so that only the cuts that are likely to be active constraints are taken into account

Improvements that modify Benders’ cuts (I)

Can be useful if there

are too many cuts and

most are dominated

Can be useful if there are too many cuts

Can be useful if there are too many cuts

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Minimal Infeasible Subsystems (MIS) can be used to modify the way feasibility cuts are calculated

[Saharidis & Ierapitrou, ‘Improving benders decomposition using maximum feasible subsistem (MFS) cut generation, 2009]

Instead of minimizing the sum of infeasibilities the problem minimizes the number of equations that are infeasible

This enables faster convergence in some cases

Conversely, if most of the solutions are infeasible, it is possible to keep a maximum feasible set to derive optimality cuts to better guide the search

Modifications to Benders’ cuts (II)

Can be useful if most of

the cuts are feasibility cuts

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


Subproblem improvements




Subtree partition [Birge & Loveaux, ‘A multicut algorithm for stochastic linear programs’, 1988]

The second stage scenarios can be arranged differently

In general, the most efficient arrangement cannot be known beforehand (tradeoff between the accuracy of the cuts and solution time for the master problem)

Scenario structure design (I)

ws: wind scenario

ss: system state

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Scenario aggregation: the second stage corresponding to the scenarios with the highest impact on the final design are added to the master problem

[Cerisola & Ramos, ‘Node aggregation in stochastic nested benders decomposition applied to hydrothermal coordination’,2000]

The master problem proposes solutions that are closer to the optimal

Convergence speed can be increased

Scenario structure design (II)

Can be useful if one of the

scenarios has a much

higher impact on the final

solution

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Solution technique modifications

Bunching: if the 2nd stage scenarios are similar we can solve only one scenario and calculate the others using the calculated sensitivities

[Birge & Loveaux, ‘Introduction to Stochastic Programming’, 1988]

Application of specific algorithms [Marin & Salmeron, ‘Electricity Capacity Expansion Models Under Uncertain Demand. Decomposition Approaches’, 1988]

Or even a series of increasingly accurate versions of the subproblem (as long as they have increasing values of the o.f.) so that time is not wasted in the first few iterations

[Romero & Monticelli, ‘A hierarchical decomposition approach for transmission network expansion planning’, 1994]

Sub-optimal subproblem solutions (Zakeri’s cuts) [Zakeri et al, ‘Inexact Cuts in Benders Decomposition’, 1999]

Any infeasible solution in the subproblem will give a valid cut (can use IPM)

Can be useful if there are

many similar scenarios being

calculated

Can be useful if there is an specific more efficient technique available

Can be useful if solving the subproblem

until optimality, even for one scenario, is

computationally very expensive

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


Case Study: Transmission

Expansion Planning




The power system in a nutshell

Supply is composed by generators of different technologies with different operation costs

Demand has to be served instantaneously (there is no possibility for storage)

The transmission network enables (and constrains) the physical transactions (and the operation outcome)

Flows follow Kirchhoff’s laws, so bilateral transactions do not have a physical meaning

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Transmission Expansion Planning

The Transmission Expansion Planning (TEP) problem consists in deciding the optimal transmission investments (lines or otherwise) that should be added to the existing transmission network in order to minimize the total investment and operation costs (generation costs and reliability).

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Simplified formulation

The TEP problem can be formulated as MIP in a centralized, cost-minimization framework

Minimize the sum of investment and operation costs

Subject to Kirchhoff’s laws

Respecting generation and transmission limits

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


min InvC OpC+ijc ijc

ijc CL

InvC ic x∈

= ∑

( )i i i

i

OpC g cg pns pnlt= +∑

:jic ijc i i i i

jic ijc

f f g d pns σ∈

− + = −∑ ∑

i j

ijc

ijc

f ijc ELX

θ θ−= ∈

( )

( )

1 :

1 :

i j

ijc ijc ijc ijc

ijc

i j

ijc ijc ijc ijc

ijc

f M xX

f M xX

ijc CL

θ θρ

θ θρ

+

−

−− ≥− −

−− ≤ −

∈

:

:

( )

ijc ijc ijc ijc

ijc ijc ijc ijc

f f x

f f x

ijc CL EL

π

π

+

−

≥−

≤

∈ ∪

, 0i i i ii

g g g pns d≤ ≤ ≤ ≤




Benders’ decomposition applied to TEP

The master problem takes the form:

And the subproblem:

2

2 2

min

( )

( ) ( )

1, ...,

ijc ijcijc CL

l lT lT

ijc ijc ijcijc

lT lT l

ijc ijc ijc ijc ijc

c x P

Z f

M x x

l n

ω ω

ω

ω ω ω ω

ω ω

π π

ρ ρ

∈ ∈Ω

+ −

+ −

+ Θ

Θ ≥ + − +

− −=

∑ ∑

∑

2,min ( )i i

n

i i ig pns

i

Z g cg pns pnltω ω

ω ω ω

ω

= +∑

:jic ijc i i i i

jic ijc

f f g d pnsω ω ω ω ω ωσ− + = −∑ ∑

i j

ijc

ijc

f ijc ELX

ω ω

ωθ θ−

= ∈

( )

( )

1 :

1 :

i j

ijc ijc ijc ijc

ijc

i j

ijc ijc ijc ijc

ijc

f M xX

f M xX

ijc CL

ω ω

ω ω

ω ω

ω ω

θ θρ

θ θρ

+

−

−− ≥− −

−− ≤ −

∈

:

:

( )

ijc ijc ijc ijc

ijc ijc ijc ijc

f f x

f f x

ijc CL EL

ω ω ω

ω ω ω

π

π

+

−

≥−

≤

∈ ∪

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


, 0i i i ii

g g g pns d≤ ≤ ≤ ≤




The ratio of time spent solving the master problem and the subproblem is the most important factor in the decision

In all cases, time spent in the subproblem is less than 12%

Master problem modification techniques are more appropriate.

Selection of the improvement techniques

Master Subproblem Decomposition

Case Eq Var D var Time (s) Eq Var Time (s) It. Time

(s)

Time per it.

Garver 14 14 9 0.6 15 45 0.4 5 5.1 1.0

46N 162 82 79 1.0 208 359 0.0 161 162.6 1.0

46N 100

scenarios

7402 181 79 9.7 208 359 1.2 75 820.6 10.9

87N 124 155 152 2.9 181 748 0.0 123 364.5 3.0

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Are relaxations potentially interesting?

The largest case study, with 152 discrete variables in the master problem, is solved in 2.9s for MIP and 0.1s in a relaxed version (85% savings)

Master problem relaxations should be explored

- Linear first

- Semi-relaxed cuts

Are suboptimal master solutions interesting?

We examine the tolerance-performance curve

Values around 2.5% seem interesting

Selection of the improvement techniques (cont’ed)

0

20

40

60

80

100

120

140

160

180

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%

Iterations

Solution tim

e (s)

Relative optimality tolerance

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





Selection of the improvement techniques (cont’ed)

Other master problem modification techniques

Box-step techniques could be interesting in general in problems where an additional constraint can be easily imposed. In this case we can limit investment. Values of 85%, 100% and 115% of optimal investment were used

Alternative solution techniques (other than MIP) are not available for this problem

Modifications to Benders cuts are not interesting in this case

The number of generated cuts is not excessively large, so cover cuts or cut removal are not indicated

The subproblem is feasible in all cases, so MIS is not applicable

ij ijij

c x MaxInv∈

≤∑C

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


Results




Results

Only the largest case study is reported for the sake of clarity

Semi-relaxed cuts and suboptimal master solutions offer consistent savings that can be above 50% of solution time

Linear-first approaches and box-step methods are not able to provide consistent results

87N Time (s) % Savings

Benders' decomposition 364.5

Linear first 374.7 -3%

Semi relaxed cuts 162.4 55%

Suboptimal master εMaster = 5%

εStep = 0.025%

106.5 71%

Suboptimal master εMaster = 2.5%

εStep = 0%

232.5 36%

Suboptimal master Highest tolerance 237.1 35%

Boxstep Initial maximum investment

= 85% optimal

193.0 47%


= 100% optimal

135.8 63%


= 115% optimal

551.3 -51%

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


Conclusions




Benders’ decomposition is a key tool in stochastic optimization which divides the problem in:

A master problem that optimizes the first stage and a piecewise linear approximation of the second stage costs

A subproblem which optimizes the second stage and creates the piecewise approximation by means of primal and marginal information

The method can bring substantial benefits when:

First-stage variables complicate the resolution of the problem

Master problem and subproblems have a different nature

Conclusions (I)

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





A slow master problem can be accelerated:

Modifying the solution method

Relaxations

Sub-optimal solutions

Box-step

Use of a more suitable technique

Modifying the calculation of cuts:

Nondominated cuts / Pareto optimal cuts

Covering cuts

Removing inactive cuts

Minimal Infeasible Subsystems

Conclusions (II)

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:





A slow subproblem can be accelerated:

Selecting the most suitable structure for the scenario tree

Scenario aggregation

Bunching

Application of specific algorithms

Sub-optimal solutions

A case study based on Transmission Expansion Planning has demonstrated

How to select the most promising improvement techniques for a particular problem

Results show that for this case the semi-relaxed cuts (proposed by the authors) and sub-optimal master resolution were able to consistently offer savings above 50%

Conclusions (III)

MotivationA quick

introduction

Master problem

techniques

Subproblem

techniques

Case Study:


Thank you

Instituto de Investigación Tecnológica

Santa Cruz de Marcenado, 26

28015 Madrid

Tel +34 91 542 28 00

Fax + 34 91 542 31 76

[email protected]

www.upcomillas.es

Documents

Improvements to Benders' decomposition: systematic ...dinamico2.unibg.it/icsp2013/doc/ps/3-Sara Lumbreras.pdf · July 2013 Improvements to Benders' decomposition: systematic classification