Discrete Op+mizaon - Inria · Fondements de l’op-misa-on discrète h7ps://project.inria.fr/2015ma2827/ Material from P. Van Hentenryck’s course • Linear Programming • Mixed

DiscreteOp+miza+on

MA2827Fondementsdel’op-misa-ondiscrète

h7ps://project.inria.fr/2015ma2827/MaterialfromP.VanHentenryck’scourse

•  LinearProgramming

•  MixedIntegerProgram

•  Examples(TSP,Knapsack)

Outline

Whatisalinearprogram?


•  nvariables,mconstraints•  Variablesarenon-nega+ve•  Inequalityconstraints


•  Whataboutmaximiza+on?– solve

•  Whatifavariablecantakenega+vevalues?–  replacexi by

•  Equalityconstraints?– usetwoinequali+es

•  Variablestakingintegervalues?– mixedintegerprogramming

Geometricalview

•  First,convexsets

Geometricalview

•  First,convexsets

Geometricalview

•  Convexcombina+ons

Geometricalview

•  Convexsets– AsetSinRnisconvexifitcontainsalltheconvexcombina+onsofthepointsinS

•  Theintersec+onofconvexsetsisconvex– Proof?

Geometricalview

•  Ahalfspaceisaconvexset

•  Polyhedron:intersec+onofsetofhalfspaces(alsoconvex).Iffinite,polytope

GeometricalviewHyperplanes

13

x1

x2

Tuesday, 11 June 13

Geometricalview

x1

x2

�4.0x1 + 2.5x2 �0.25

Hyperplanes

14

Tuesday, 11 June 13

Geometricalview

Geometricalview

x1

x2

�3.0x1 � 6.0x2 �43.50

Hyperplanes

15

Tuesday, 11 June 13

Geometricalview

Geometricalview

x1

x2

1.5x1 � 4.5x2 3.00

Hyperplanes

16

Tuesday, 11 June 13

Geometricalview

x1

x2

�4.0x1 + 2.5x2 �0.25�3.0x1 � 6.0x2 �43.501.5x1 � 4.5x2 3.001.7x1 � 1.7x2 16.674.8x1 + 1.2x2 82.332.0x1 + 3.5x2 65.75

�3.0x1 + 5.0x2 29.50

Hyperplanes

21

Tuesday, 11 June 13

Geometricalview3D Hyperplanes

23

Tuesday, 11 June 13


24

Tuesday, 11 June 13


25

Tuesday, 11 June 13

GeometricalviewHyperplane, Facets, and Vertices

26

Tuesday, 11 June 13


26

Facet

Tuesday, 11 June 13


26

FacetVertex

Tuesday, 11 June 13

Geometricalview3D Constraints

27

Tuesday, 11 June 13

Geometricalview3D Constraints

28

Tuesday, 11 June 13

GeometricalviewGeometry of Linear Programs‣Every point in a polytope is a convex

combination of its vertices

29

Tuesday, 11 June 13

Geometricalview

x1

x2

Geometry of Linear Programs‣Every point in a polytope is a convex


29

Tuesday, 11 June 13

Geometricalview

x1

x2



29

Tuesday, 11 June 13

Geometricalview

x1

x2



29

Tuesday, 11 June 13

Geometricalview

x1

x2



29

Tuesday, 11 June 13

Geometricalview

x1

x2



29

Tuesday, 11 June 13

Geometricalviewmin c1x1 + . . .+ cnxn

subject to

a11x1 + . . .+ a1nxn b1

. . .

am1x1 + . . .+ amnxn bm

xi � 0 (1 i n)

Why I Love These Vertices

30

‣Theorem: At least one of the points where the objective value is minimal is a vertex.

Tuesday, 11 June 13

Geometricalview

x1

x2


31

Tuesday, 11 June 13

Geometricalview

x1

x2

c1x1 + . . .+ cnxn = b


31

Tuesday, 11 June 13

Geometricalview

x1

x2

c1x1 + . . .+ cnxn = b


31

Tuesday, 11 June 13

Geometricalview

x1

x2

c1x1 + . . .+ cnxn = bmin


31

Tuesday, 11 June 13

Geometricalview

x1

x2



31

Tuesday, 11 June 13

Geometricalview

x1

x2



31

Tuesday, 11 June 13

Geometricalview

x1

x2



31

Tuesday, 11 June 13

Geometricalview

x1

x2



31

c1x1 + . . .+ cnxn = b

⇤

Tuesday, 11 June 13

GeometricalviewWhy I Love These Vertices Now in 3D!

32

Tuesday, 11 June 13

GeometricalviewWhy I Love These Vertices Now in 3D!

33

Tuesday, 11 June 13

Geometricalview

•  Theorem:Atleastoneofthepointswheretheobjec+vevalueisminimalisavertex

•  Proof?

Geometricalview

Geometricalview

•  Howtosolvealinearprogram?– Enumerateallthever+ces

– Selecttheonewiththesmallestobjec+vevalue

Algebraicview

•  Thesimplexalgorithm– Amoreintelligentwayofexploringthever+ces

•  InventedbyG.Dantzig

•  Exponen+alworst-case,butworkswellinprac+ce

Simplexalgorithm

•  Outline1.  Anop+malsolu+onisatavertex2.  Avertexisabasicfeasiblesolu+on(BFS)3.  YoucanmovefromoneBFStoaneighboringBFS4.  YoucandetectwhetheraBFSisop+mal5.  FromanyBFS,youcanmovetoaBFSwithbe7er

acost

Basicfeasiblesolu+on(BFS)min c1x1 + . . .+ cnxn

subject to

a11x1 + . . .+ a1nxn b1

. . .

am1x1 + . . .+ amnxn bm

xi � 0 (1 i n)

Linear Programs

7

Tuesday, 11 June 13

Basicfeasiblesolu+on(BFS)

a11x1 + . . .+ a1nxn = b1

. . .

am1x1 + . . .+ amnxn = bm

xi � 0 (1 i n)

Algebraic View: BFS

Goal: How to find solutions to linear systems

8

Tuesday, 11 June 13


x1 = b1 +nX

i=m+1

a1i xi

. . .

xm = bm +nX

i=m+1

ami xi

Algebraic View: BFS

Basic Solution

9

Tuesday, 11 June 13






a11x1 + . . .+ a1nxn = b1

. . .

am1x1 + . . .+ amnxn = bm

xi � 0 (1 i n)

Algebraic View: BFS

Goal: How to find solutions to linear systems

10

Tuesday, 11 June 13



x1 = b1 +nX

i=m+1

a1i xi

. . .

xm = bm +nX

i=m+1

ami xi

Feasible if 8i 2 1..m : bi � 0

Algebraic View: BFS

Basic Feasible Solution

12

Tuesday, 11 June 13


•  Selectmvariables– Willbethebasicvariables

•  Re-writethemusingonlynon-basicvariables– Gaussianelimina+on

•  Ifalltheb’sarenon-nega+ve– WehaveaBFS


•  Butwedonothaveequali+es


•  Butwedonothaveequali+es

Slackvariables


•  Re-writetheconstraintsasequali+es– Withslackvariables

•  Selectmvariables– Willbethebasicvariables

•  Re-writethemusingonlynon-basicvariables– Gaussianelimina+on

•  Ifalltheb’sarenon-nega+ve– WehaveaBFS

Naïvealgorithm

•  Generateallbasicfeasiblesolu+ons–  i.e.,selectmbasicvariables,performGaussianelim.–  testwhetheritisfeasible

•  SelecttheBFSwiththebestcost

•  But, n!m!(n−m)!

Canweexplorethisspacemoreefficiently?

Simplexalgorithm


acost

Simplexalgorithm

•  Localsearchalgorithm

•  MovefromBFStoBFS

•  Guaranteedtofindaglobalop+mum– Duetoconvexity

Keyidea:Howdowedothismoveopera+on?

Simplexalgorithm

Simplexalgorithm

•  MovetoanotherBFS(localsearch)– Selectanon-basicvariablewithnega+vecoeff.(enteringvariable)

– Replaceabasicvariablewiththisselec+on(leavingvariable)

– PerformGaussianelimina+on

Simplexalgorithm

Simplexalgorithmx3 = 1 � 3x1 � 2x2

x4 = 2 � 2x1 + x6

x5 = 3 � x1 + x6

From BFS to BFS

9

Tuesday, 11 June 13

Simplexalgorithmx3 = 1 � 3x1 � 2x2

x4 = 2 � 2x1 + x6

x5 = 3 � x1 + x6

From BFS to BFS

10

Tuesday, 11 June 13

Simplexalgorithm

Simplexalgorithm

Simplexalgorithm

Simplexalgorithm

Simplexalgorithm

MovetoanotherBFS:•  Selecttheenteringvariablexe– Non-basicvariablewithnega+vecoeff.

•  Selecttheleavingvariablexltomaintainfeasibility•  ApplyGaussianelimina+on–  i.e.,eliminatexe fromthetherighthandside

•  Pivot(e,l)

Simplexalgorithm


acost

Simplexalgorithm

•  ABFSisop+maliftheobjec+vefunc+on,aherelimina+ngallthebasicvariablesis

Simplexalgorithm

•  Example

Simplexalgorithm

•  Example

Simplexalgorithmmin 6 � 3x1 � 3x2

subject to

x3 = 1 � 3x1 � 2x2

x4 = 2 � 2x1 + x2

x5 = 3 + x1 � 3x2

From BFS to BFS

19

Tuesday, 11 June 13


subject to

x3 = 1 � 3x1 � 2x2

x4 = 2 � 2x1 + x2

x5 = 3 + x1 � 3x2

From BFS to BFS

19

Tuesday, 11 June 13


subject to

x3 = 1 � 3x1 � 2x2

x4 = 2 � 2x1 + x2

x5 = 3 + x1 � 3x2

From BFS to BFS

19

Tuesday, 11 June 13

Simplexalgorithm

Simplexalgorithm

Simplexalgorithm


acost

Simplexalgorithm

•  Localmoveimprovestheobjec+vevalue•  InaBFS,assumethat– b1>0,b2>0,…,bm>0–  thereexistsanenteringvariableewithce<0

•  Then,themovepivot(e,l)isimproving

Simplexalgorithm

•  If– b1,b2,…,bmarestrictlyposi+ve– objec+vefunc+onisboundedbelow

•  Algorithmterminateswithanop+malsolu+on

Simplexalgorithm

•  Selec+ngtheleavingvariable

•  Noleavingvariable!

Simplexalgorithm

•  Basicsolu+on

•  WhatifIincreasethevalueofx1– Solu+onremainsfeasible,but– Valueofobjec+vefunc+ondecreases

Simplexalgorithm

•  Anotherissue:Whatifsomebibecomeszero?

•  Leavingvariable:x2

Simplexalgorithm


acost

Needtofindanewwaytoguaranteetermina+on

Simplexalgorithm

•  Termina+on– Blandrule:selectalwaysthefirstenteringvariablelexicographically

– Lexicographicpivo+ngrule:break+eswhenselec+ngtheleavingvariable

– Pertuba+onmethods

Simplexalgorithm

•  HowdoIfindmyfirstBFS?

•  Introducear+ficialvariables

Simplexalgorithm

•  EasyBFS

•  Butwrtanotherproblem

Simplexalgorithm

•  Two-phasestrategy1.  FirstfindaBFS(ifoneexists)

2.  Then,findop+malBFS

Simplexalgorithm

1.  FindaBFS

•  Feasibleiftheobjec+vevalueis0–  i.e.,allyi’sare0,andthereisaBFSwithoutyi(almostalways)

Simplexalgorithm

Documents

Discrete Op+mizaon - Inria · Fondements de l’op-misa-on discrète h7ps://project.inria.fr/2015ma2827/ Material from P. Van Hentenryck’s course • Linear Programming • Mixed