EE 553Integer Programming

J. McCalley

1

Solving IPsMinimize f(x1, x2, x3)Subject to h(x1, x2, x3)=c g(x1, x2, x3)≤b x1, x2, x3 are binary (1 or 0)

2

How to solve it?

1. Check every possible solution (exhaustive enumeration),A huge computational burden.

2. Solve it as an LP and then round to closest binary value.Under such an approximation, optimality cannot guaranteed.

Two immediate ideas:

Solving IPs: Tree search methods

3

• Tree-search methods conceptualize the problem as a huge tree of solutions, and then they try to do some smart things to avoid searching the entire tree.

• Most popular IP (and MIP) solver today is a tree-search method called branch and bound. CPLEX uses this method, in combination with cutting planes.

• We will study the branch and bound method.

●

● ●

● ● ● ●

● ● ● ● ● ● ● ●

Solving IPs: Branch & Bound

4

Solving IPs: Branch & Bound

5

Definitions•Predecessor problem•Successor problemProblem Pj is predecessor to Problem Pk and Problem Pk is successor to Problem Pj if:

they are identical with the exception that one continuous-valued variable in Pj is constrained to be integer in Pk.

●

● ●

● ● ● ●

● ● ● ● ● ● ● ●

Pj

Pk

Example

6

0,

5

40710

s.t.

1217 max

21

21

21

21

0

xx

xx

xx

xx

P

Solution (as an LP) using CPLEX yields

333.68,333.3,667.1

:

21

0

xx

SolutionP

“Zeta”

0,

1

5

40710

s.t.

1217 max

21

1

21

21

21

1

xx

x

xx

xx

xx

P

Pose a problem P1 to be exactly like P0 except that we will constrain x1≤1.

What do you expect the value of x1 to be in the optimal solution?Because the solution without the constraint x1≤1 wanted 1.667 of x1, we can be sure that the solution with the constraint x1≤1 will want as much of x1 as it can get, i.e., it will want x1=1.Solution (as an LP) using CPLEX yields

0.65,0.4,0.1: 211 xxSolutionP

It worked: x1 did in fact become integer. In fact, x2 became integer as well, but this is by coincidence.

Example

7

Is the P1 solution we obtained, which by chance is a feasible solution to IP1, also optimal to IP1?

Let’s consider the same problem but with (x1,x2) constrained to be integers.

integers. ,

0,

5

40710

s.t.

1217 max

21

21

21

21

21

1

xx

xx

xx

xx

xx

IP

●

● ●

● ● ● ●

● ● ● ● ● ● ● ●

x1<1333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

P1 has to be as good as or better than its successors because these will be more heavily constrained, so we need not look at P1 successors.

2P

But what about P2? What is P2?

Example

8

●

● ●

● ● ● ●

● ● ● ● ● ● ● ●

x1<1333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

P2 is the P0 problem with x1>2.It is the remaining part of the space we need to search.

x1>2

0,

2

5

40710

s.t.

1217 max

21

1

21

21

21

2

xx

x

xx

xx

xx

P

Use CPLEX to solve P2. (see solution on the tree). Observe P2 is infeasible!(infeasible to the IP, that is, because x2 is non-integer).

284.68

,857.2,0.2

:

21

2

xx

SolutionP

So can we draw any useful conclusion from this observation?

YES!But why?

1. The P2 solution, 68.284, is better than our best feasible solution so far, 65.0 (65.0 establishes a lower boundsolution is no less than 65).2. Although P2 is infeasible, we can add constraints and find feasible solutions in successor problems.3. We are not sure any of those successor problems will be “better than our best” (of 65), but because 68.284>65, we know it is worth trying.

Example

9

●

● ●

● ● ● ●

● ● ● ● ● ● ● ●

x1<1333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP x1>2

284.68

,857.2,0.2

:

21

2

xx

SolutionP

Yes, Why? (In more detail…) Compare the objective function value of P2, 68.8286, with the objective function value of P1, 65. Since we are maximizing, the objective function of P2 is better. But the P2 solution is not feasible. But we can constrain x2 so that we get a feasible solution. Whether that feasible solution will have better objective function value we do not know. What we do know is, because the objective function value of P2 (68.8286) is better than the objective function value of P1 (65), it is worthwhile to check it. Although the objective function value of successor problems to P2 can only get worse (lower), they might be better than P1, and if we can find a successor (or a successor’s successor,…) that is feasible, it might be better than our best current feasible solution, which is P1.

Example

10

●

● ●

● ● ● ●

● ● ● ● ● ● ● ●

x1<1333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP x1>2

284.68

,857.2,0.2

:

21

2

xx

SolutionP

Question: What if the P2 solution would have been 64?Would you have searched its successor nodes?

NO! Why not?

Because successor nodes, even if feasible, would necessarily be more constrained than P2 and therefore no better than its solution. Since we already have a feasible solution of 65, and P2 successors could be no better than 64, there is no use searching them. Again, the value of 65 establishes a lower bound on the problem solution.

Example

11

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

But because P2 solution is better than our current best feasible solution, we should pursue P2 successor problems.

Our choices are x2<2 and x2>3. Let’s try x2<2, using P3.

So what constraint should we add to P2?

0,

2

2

5

40710

s.t.

1217 max

21

2

1

21

21

21

3

xx

x

x

xx

xx

xx

P

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

The P3 solution is not feasible. Should we branch further?YES!

1. The P3 solution, 68.2, is better than our best feasible solution so far, 65.0.

2. Although P2 is infeasible, we can add constraints and find feasible solutions in successor problems.3. We are not sure any of those successor problems will be “better than our best” (of 65), but because 68.2>65, we know it is worth trying.

Example

12

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

Our choices are x1<2 and x1>3. Let’s try x1<2, using P4.

So what constraint should we add to P3?

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

Should we branch further?

No! Why?

• The P4 solution is feasible! And so we will not find another better feasible solution that is successor to P4.

• The P4 objective is 58, worse than our best (65). So P4 and any further successor nodes are of no interest.

0,

2

2

5

40710

s.t.

1217 max

21

2

1

21

21

21

4

xx

x

x

xx

xx

xx

P

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

Two reasons, either one of which is enough:

Example

13

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

We impose x1>3, using P5.

Now what? We have to decide on going back to P3 or P2. Choose P3.

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

Should we branch further?

Yes! Why?

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP0,

2

3

5

40710

s.t.

1217 max

21

2

1

21

21

21

5

xx

x

x

xx

xx

xx

P

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

1. The P5 solution, 68.1429, is better than our best feasible solution so far, 65.0.

2. Although P5 is infeasible, we can add constraints and find feasible solutions in successor problems.3. We are not sure any of those successor problems will be “better than our best” (of 65), but because 68.1429>65, we know it is worth trying.

Example

14

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

Should we branch further?

Yes! Why?

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

1. The P6 solution, 68.1, is better than our best feasible solution so far, 65.0.

2. Although P6 is infeasible, we can add constraints and find feasible solutions in successor problems.3. We are not sure any of those successor problems will be “better than our best” (of 65), but because 68.1>65, we know it is worth trying.

Our choices are x2<1 and x2>2. Let’s try x2<1, using P6.

So what constraint should we add to P5?

0,

1

3

5

40710

s.t.

1217 max

21

2

1

21

21

21

6

xx

x

x

xx

xx

xx

P

x2<1 x2>2

1.68

0.1,3.3

:

21

6

xx

SolutionP

Example

15

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

Should we branch further?

No! Why?

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

Our choices are x1<3 and x1>4. Let’s try x1<3, using P7.

So what constraint should we add to P6?

x2<1 x2>2

1.68

0.1,3.3

:

21

6

xx

SolutionP

0,

1

3

5

40710

s.t.

1217 max

21

2

1

21

21

21

7

xx

x

x

xx

xx

xx

P

0.63

0.1,0.3

:

21

7

xx

SolutionPx1<3 x1>4

• The P7 solution is feasible! We will not find another better successor to P7

• P7 objective is 63, worse than our best (65), so P7’s successor nodes are of no interest.

Two reasons, either one of which is enough:

Now let’s try here.

Example

16

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

Should we branch further?

No! Why?

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

So now add the x1>4 constraint to P6, to obtain P8.

x2<1 x2>2

1.68

0.1,3.3

:

21

6

xx

SolutionP

0.63

0.1,0.3

:

21

7

xx

SolutionPx1<3 x1>4

• The P8 solution is feasible! We will not find another better successor to P8

Note the P8 objective is 68, which is better than our best (65)! So P8 solution becomes our new best, i.e., it becomes our new lower bound on the solution. That is, the objective at the solution must be at least 68.

0,

1

4

5

40710

s.t.

1217 max

21

2

1

21

21

21

8

xx

x

x

xx

xx

xx

P

0.68

0,0.4

:

21

8

xx

SolutionP

Example

17

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

x2<1 x2>2

1.68

0.1,3.3

:

21

6

xx

SolutionP

0.63

0.1,0.3

:

21

7

xx

SolutionPx1<3 x1>4

0.68

0,0.4

:

21

8

xx

SolutionP

Question: Do we need to check the other branch to P5 and P2?

Answer: Yes! Why?

Because the objective value for P5 and P2 at greater than our bound of 68, so a successor node could be better than 68 as well.

Example

18

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

x2<1 x2>2

1.68

0.1,3.3

:

21

6

xx

SolutionP

0.63

0.1,0.3

:

21

7

xx

SolutionPx1<3 x1>4

0.68

0,0.4

:

21

8

xx

SolutionP

0,

3

2

5

40710

s.t.

1217 max

21

2

1

21

21

21

9

xx

x

x

xx

xx

xx

P

P9: infeasibleto the LP

0,

2

3

5

40710

s.t.

1217 max

21

2

1

21

21

21

10

xx

x

x

xx

xx

xx

P

P10: infeasible to the LP

Central ideas to branch & bound

19

Branch, force integrality on one variable by adding a constraint to an LP-relaxation:

Use LP-relaxation to decide how to branch. Each branch adds a constraint to previous LP-relaxation to enforce integrality on one variable that was not integer in the predecessor solution.

Bound, continue branching only if the objective function value of the current solution is better than the objective function value of the best feasible solution obtained so far:

Maintain the best feasible solution obtained so far as a bound on tree-paths that should still be searched. • If any tree node has an objective value less optimal than the identified bound,

no further searching from that node is necessary, since adding constraints can never improve an objective.

• If any tree node has an objective value more optimal than the identified bound, then additional searching from that node is necessary.

Using CPLEX to solve MIPS directly

20

maximize 17 x1 + 12 x2subject to 10 x1 + 7 x2 <= 40 x1 + x2 <= 5 Bounds 0<= x1 <= 1000 0<= x2 <= 1000Integer x1 x2end

1. Created CPLEX source within a text file called mip.lp as follows:

2. Used WinSCP to port the file to server (linux-7.ece.iastate.edu).4. Called cplex.5. Typed read mip.lp to read in problem statement.6. Typed mipopt to call the MIP-solver. The result was as follows…

Using CPLEX to solve MIPS directly

21

maximize 17 x1 + 12 x2subject to 10 x1 + 7 x2 <= 40 x1 + x2 <= 5 Bounds 0<= x1 <= 1000 0<= x2 <= 1000Integer x1 x2end

1. Created CPLEX source within a text file called mip.lp as follows:

7. Typed display solution variables -

The result was:

Variable Name Solution Valuex1 4.000000All other variables in the range 1-2 are 0.

Depth vs breadth

22

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

Recall this point in our procedure:The question can be posed like this:Depth: Do we continue from P3, requiring x1≤2, for example? or Breadth: Do we go back to P2 to examine its other branch, x2≥3?

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

For high-dimensional IPs, it is usually the case that feasible solutions are more likely to occur deep in a tree than at nodes near the root. Finding multiple feasible solutions early in B&B is important because it tightens the bound (in the above example, it increases the bound), and therefore enables termination of branching at more nodes (and therefore decreases computation). One can see that this will be the case if we consider the bound before we find a feasible solution: the bound is infinite! (-∞ for maximization problems and +∞ for minimization problems).

Branching variable selection

23

For high-dimensional IPs, is there any benefit for selecting one non-integer variable over another when branching?

In our problem, except initially, there was never a decision to make in this way because there was never more than one non-integer variable.

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

x2<1 x2>2

1.68

0.1,3.3

:

21

6

xx

SolutionP

0.63

0.1,0.3

:

21

7

xx

SolutionPx1<3 x1>4

0.68

0,0.4

:

21

8

xx

SolutionP

P9: infeasible

P10: infeasible

A rich research question.For specific problems, you can pre-specify an ordering of the variables required to be integer. Good orderings become apparent based on experience with running the algorithm or based on physical understanding, e.g., largest unit.

Mixed integer problems

24

Our example was a pure integer problem. What about a MIP?

x1<1

333.68

,333.3,667.1

:

21

0

xx

SolutionP

0.65

,0.4,0.1

:

21

1

xx

SolutionP

x1>2 284.68

,857.2,0.2

:

21

2

xx

SolutionP

x2<2 x2>3

2.68

,0.2,6.2

:

21

3

xx

SolutionP

x1<2 x1>3

0.58

0.2,0.2

:

21

4

xx

SolutionP

1429.68

4286.1,0.3

:

21

5

xx

SolutionP

x2<1 x2>2

1.68

0.1,3.3

:

21

6

xx

SolutionP

0.63

0.1,0.3

:

21

7

xx

SolutionPx1<3 x1>4

0.68

0,0.4

:

21

8

xx

SolutionP

P9: infeasible

P10: infeasible

The solution to IP2 is obtained as soon as we solve P1 and P2.

integer.

0,

5

40710

s.t.

1217 max

1

21

21

21

21

2

x

xx

xx

xx

xx

IP

Conclusion: We can easily solve MIP within our LP-relaxation branch and bound scheme by simply allowing the non-integer variables to remain relaxed.