Dantzig-Wolfe Decomposition · Thomas Stidsen 3 DTU-Management / Operations Research Dantzig-Wolfe...

1Thomas Stidsen

DTU-Management / Operations Research

Dantzig-Wolfe Decomposition– Changing a problem ...

Thomas Stidsen

thst@man.dtu.dk

DTU-Management

Technical University of Denmark

2Thomas Stidsen

OutlineGeneral Background for Dantzig-Wolfe

Mathematical background

A 2-dim example

Multi Commodity Flow problem (the lastexercise)

Block-Angular Structure

3Thomas Stidsen

Dantzig-WolfeHistorically:

Dantzig-Wolfe decomposition was invented byDantzig and Wolfe 1961.

The method is so closely connected to columngeneration that they in some aspects may beconsidered to be identical.

Dantzig-Wolfe and Column-Generation is one ofthe most used methods for practical problems.

Notice that column generation and Dantzig-Wolfe

are used interchangeably ...

4Thomas Stidsen

From Appendix A we haveGiven the convex set X = {x|Ax ≤ b} can berepresented by the extreme points and extremerays of the convex set (Minkowski-Weyl’s Theorem):

X = {x|x =∑

p λp · xp +∑

r δr · xr}

where∑

p λp = 1, λp ≥ 0, δr ≥ 0 and where

p ∈ {1, . . . , P}, r ∈ {1, . . . , R} See Appendix A, p.

5Thomas Stidsen

Extreme raysGood news: In theory we need extreme rays torepresent arbitrary polyhedrons, but in reality weonly encounter this in Dantzig-Wolfe decompositionvery seldom. I have never seen a Dantzig-Wolfedecomposition where an extreme ray wasnecessary. Hence we will from here on simplyassume that we can just use extreme points.

6Thomas Stidsen

The importance of a good polytope

1 2 3 4

If we can achieve the smallest polytope in the figure,we can solve MIP problems with LP solvers !!!

7Thomas Stidsen

Given a Linear programMin:

A1x ≥ b1

A2x ≥ b2

x ≥ 0

8Thomas Stidsen

Changing representationDantzig-Wolfe decomposition is actually aboutchanging representation from a set of constraints toa set of extreme points. The good question is now,how many constraints should be replaced byextreme points ?

All: You tried that, in the first exercise, it doesnot really help ...

None: Well that does not change the problem !!!

Some: Yes, but why should that help ?

9Thomas Stidsen

Visualized

1 2 3 4

10Thomas Stidsen

The mathematical formulationMax:

x + 2y

s.t.:−x + y ≤

x + y ≤9

−x − y ≤ −1

−x − y ≤ −3

−x + y ≤ 1 (A2)

2x + y ≤ 13 (A2)

−x − 3y ≤ −7 (A2)

11Thomas Stidsen

Lets look at A1

1 2 3 4

12Thomas Stidsen

Change representationNow we represent part, the A2 part, of theconstraints with the convex combination of extremepoints and extreme arrays:X = {x|x =

p λp · xp}

where the extreme points p define the setX = {x|A2x ≥ b2}, BUT WHICH EXTREMEPOINTS ????

13Thomas Stidsen

Improving the bound of A1: A1’

−x + y ≤5

x + y ≤9

−x − y ≤ −1

−x − y ≤ −3

x, y ∈ R+(A1) OR x, y ∈ Z+(A1′)

14Thomas Stidsen

The LP improvement !

Objective

LP gap improvement

1 2 3 4

15Thomas Stidsen

InsertionMin:

cT (∑

λp · xp +∑

δr · xr)

A2(∑

λp · xp +∑

δr · xr) ≥ b2

λp = 1

λp, δr ≥ 0

Notice: xp and xr are now constants and λp and δr

are now the variables

16Thomas Stidsen

Insertion without extreme raysMin:

cT (∑

λp · xp)

A2(∑

λp · xp) ≥ b2

λp = 1

λp ≥ 0

17Thomas Stidsen

Insertion without extreme rays, matrix versionAssume that:

The A2 matrix is a m2 by n matrix, i.e. m2 rowsand n x variables.

We replace the values of the x variables withthe column vector of variables λp.

We have a new matrix X with n rows and pcolumns

18Thomas Stidsen

The transformed problemMin:

A2X λp ≥ b2

1 λp = 1

λp ≥ 0

19Thomas Stidsen

WhereGenerally: x = Xλp.

The matrix AX = A2X is a matrix with m2 rowsand p columns (corresponding to the newvariables.

Each matrix element in the new matrix can befound as axm2,p = A1(m1)X(p)

A(m2): The m2’th row of the A2 matrix

X(p): The p’th column of the X matrix

This means that we can calculate for each addedcolumn the new resulting column in the masterproblem

20Thomas Stidsen

How can we find the extreme points of A1 ?We need two things:

Satisfy the constraints of A1 or A′1 (otherwise it

will not be an extreme points)

Calculate the reduced costs of the extremepoint in the A1 or A′

1 constraints, based on theoriginal costs and the dual variables from theA2 part.

21Thomas Stidsen

The sub-problemMin:

(c − π · A2)x − α

A1 · x ≥ b2

x, y ∈ R+(A1) OR x, y ∈ Z+(A′1)

22Thomas Stidsen

The sub-problemStopping criteria, if we cannot generate an extremepoint:(c − π · A2)x − α < 0Stopping criteria, if we cannot generate an extremeray:(c − π · A2)x < 0If we cannot generate either an extreme point or anextreme ray then we have the optimal solution.Of course, for maximization problems, the reducedprofits for extreme points and extreme rays isrequired to be positive.

23Thomas Stidsen

The column generation algorithmEnsure feasibility of the master problemrepeat

SOLVE min{cTx|A2X λp ≥ b, 1 λp = 1, λp ≥ 0}get πSOLVE

min{cred = (c − π · A1)x − α|A1 · x ≥ b1,x, y ∈ R+(A1)ORx, y ∈ Z+(A′

1)}calculate new column data:

cp = cTx

axm2,p = A2(m2)X(p)

until cred ≥ 0

24Thomas Stidsen

Guideline when decomposingThe guideline when performing Dantzig-Wolfedecomposition:

Find a problem which has Non-Integral property

Find a way to solve the sub-problem fast.

Examples of the combinatorial sub-problems: Theknapsack problem and the constrained shortestpath problem.

Much more about this in the next lecture !

25Thomas Stidsen

The Block-Angular StructureWe can use the structure in the following way:

The variables are linked together by the A1

matrix.

If we seperate the matrixes and are able tosolve the problems seperatly, we can solve theproblem by solving the subproblem for thematrix’es A′

1 and A′′2

26Thomas Stidsen

The Block-Angular StructureSuppose the optimisation problem has the followingstructure Given two convex sets, in two dimensions:

A2’’

27Thomas Stidsen

The Block-Angular StructureNotice that the two sub-problems share theprices π ...

Sometimes it is beneficial to simply use astandard MIP solver.

You have already experienced the division intoseveral sub-problems in the multi-commodityflow exercise

28Thomas Stidsen

Multi-Commodity flowMaximize the flow AE and BE through anetwork with limited capacity.

An example could be: A producer of naturalgas, production at A and B. Supply thecustomer, who does not care where the gascomes from.

Actually this problem can be solved by the maxflow algorithm (how ?). This formulation waschosen in order to make the problem simple.

We will only consider simple paths.

Orientation is tricky !

29Thomas Stidsen

Multi-Commodity flow: The network

��

Demand1: A −> E

Demand2: B −> E

30Thomas Stidsen

AE matrix

AB 1 1 1 1AC 1 1 1BC 1 1 1BD 1 1 1CD 1 1 1CE 1 1 1DE 1 1 1 1

31Thomas Stidsen

BE matrix

AB 1 1AC 1BC 1 1BD 1 1CD 1 1 1CE 1 1 1DE 1 1 1

32Thomas Stidsen

c and bcp = 1

AB 1AC 1BC 1BD 4CD 1CE 3DE 3

33Thomas Stidsen

The link-path problemMax:

cT (xAEp + xBE

AAExAEp + ABExBE

p ≤ b

xAEp , xBE

p ≥ 0

How did we get here ?

34Thomas Stidsen

The arc-flow formulationMax: cT (yAE + yBE)

xAE(ij) −

xAE(ji) =

yAE i = A

−yAE i = E

0 otherwise

xBE(ij) −

xBE(ji) =

yBE i = B

−yBE i = E

0 otherwise

xAE(ij) + xAE

(ji) + xBE(ij) + xBE

(ji) ≤ b ∀{ij}

yAE, yBE, xAE(ij), x

BE(ij) ≥ 0

35Thomas Stidsen

CommentsThe arc-flow constraints correspond to apolytope (A1 · x = b1) which is replaced with thepath variables

Notice that each path variable corresponds to anumber of arc flow variables

We have two separate subproblems ...

We can easily calculate an arc-flow solutionbased on our paths ...

36Thomas Stidsen

Integer SolutionsAs I mentioned the last time, there are severalmethods:

Rounding up, not always possible ....

Solve the LP problem with column generation,and after the column generation algorithm hasfinished, the MIP model is solved using astandard solver.

Use standard branching in the original space

Branch and price, hard and quite timeconsuming ...

37Thomas Stidsen

Branch and Price: Branching ProblemIf we branch on the transformed variables, we havebig problems in "down branch", because thatvariable is generated again.

SSS SSSSS

x1=1x1=0

111110101100011010001000

111001S

38Thomas Stidsen

Branch and Price: Branching in the original variabWe can perform standard branching, but we have todo it in another variable space ! This I will talk moreabout in the next lecture.

39Thomas Stidsen

BranchingSo called Ryan-Foster branching can be applied(for set-partitioning problems)Min:

s.t.:∑

ai,j · xj = 1 ∀i

xj ∈ {0, 1}

Further, notice that ai,j ∈ {0, 1}

40Thomas Stidsen

Ryan-Foster BranchingThe “end rule”: Each row will (in the optimal integersolution) be “covered” by EXACTLY one column(variable). When we have a fractional solution, thisend rule is broken: At least two rows will be coveredby two variables. So called Ryan-Foster branchingcan be applied (for set-partitioning problems)

41Thomas Stidsen

Ryan-Foster BranchingFinding the branching cuts, find two variables whichare fractional and which share at least one “cover”.

x1 x2==

0.3 0.7

42Thomas Stidsen

The branching rulesGiven two rows i1 and i2 in one branch requireai1,j′ = ai2,j′ for all variables, i.e. rows i1 and i2 arecovered by the same variables, and in the otherbranch: ai1,j′ 6= ai2,j′, i.e. rows i1 and i2 are coveredby different variables.

Dantzig-Wolfe Decomposition · Thomas Stidsen 3 DTU-Management / Operations Research Dantzig-Wolfe...

Documents

Dantzig Wolfe Decomposition - Université catholique … · Contents 1 Algorithm Description [Infanger, Bertsimas] 2 Examples [Bertsimas] 3 Application of Dantzig-Wolfe in Stochastic

Stephen A. Dantzig. Softbox Lighting Techniques

Dantzig-Wolfe Decomposition and Branch-and-Price … · Department of Computer Science & Software Engineering ... elling language Zinc ... To deal with a possibly exponential number

Interview with George B. Dantzig: The Father of Linear ... 1… · An Interview with George B. Dantzig: The Father of Linear Programming ... An Interview with George B. Dantzig:

Introduction - Amsterdam Optimizationamsterdamoptimization.com/pdf/minlp.pdf · [18], Benders Decomposition [16], Column Generation [17] and Dantzig-Wolfe De-composition [19] algorithms

A Stabilized Structured Dantzig-Wolfe Decomposition Methodpages.di.unipi.it/frangio/papers/S2DW.pdf · 2012. 12. 8. · than that of the LP relaxation [3,8{10,21,22]. On the other

Dantzig–Wolfe Decomposition and Plant-wide MPC Coordination (2007)

Z historie lineárního programování-George B. Dantzig (1914-2005)

A Stabilized Structured Dantzig- Wolfe Decomposition Methodpages.di.unipi.it/frangio/papers/CIRRELT-2010-02.pdf · 2010. 1. 13. · A Stabilized Structured Dantzig-Wolfe Decomposition

Running headline : Dantzig selector fox Cox’s modelstats.lse.ac.uk/fryzlewicz/dscox/SurvDantzig-final.pdfRunning headline : Dantzig selector fox Cox’s model Anestis Antoniadis,

A class of Dantzig–Wolfe type decomposition methods for ...pages.cs.wisc.edu/~solodov/lss11DW.pdfA class of Dantzig–Wolfe type decomposition methods 179 Note that this problem

Dantzig Fulkerson

1 Dantzig-Wolfe Decomposition. 2 Outline block structure of a problem representing a point by extreme points Revised Simplex to the extreme point

The Constrained Dantzig Selector with Enhanced Consistency

Dantzig-Wolfe Decomposition for Cost Partitioning

Automatic Dantzig–Wolfe reformulation of mixed integer

The Dantzig selector: Statistical estimation when p is

A class of Dantzigâ€“Wolfe type decomposition methods for

Column Generation, Dantzig-Wolfe, Branch-Price-and-Cut · 2020. 9. 30. · Reminder: Vertex Coloring Problem Data G= (V;E) undirected graph Goal color all vertices such that adjacent

Operations Research Manaemoriam: George Dantzigpucrs.br/ciencias/viali/graduacao/po_2/literatura/historia/Dantzig... · Title: Operations Research Mana...emoriam: George Dantzig Author: