Optimization - Lecture 5, Part 1

M. Pawan Kumar

http://www.robots.ox.ac.uk/~oval/

Slides available online http://mpawankumar.info

• Integer Linear Programming

• Duality

• Integer Polyhedron

• Totally Unimodular Matrices

Outline

Linear Program

s.t. A x ≤ b

maxx cTx

Integer Linear Program

s.t. A x ≤ b

maxx cTx

x is an integer vector

Every element of x is an integer

Integer Linear Program

s.t. A x ≤ b

maxx cTx

x ∈ Zn

Every element of x is an integer

Example

4x1 – x2 ≤ 8

maxx x1 + x2

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0

s.t.

x ∈ Zn

Example

x1 ≥ 0

x2 ≥ 0

Example

4x1 – x2 = 8

x1 ≥ 0

x2 ≥ 0

Example

4x1 – x2 ≤ 8

x1 ≥ 0

x2 ≥ 0

Example

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

x1 ≥ 0

x2 ≥ 0

Example

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0

Example

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0

x ∈ Zn

maxx cTx

(2,6)

(3,4)

(2,0)(0,0)

(0,1)

Example

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0

x ∈ Zn

maxx cTx

(2,6)

(3,4)

(2,0)(0,0)

(0,1)

Why? True in general?

Example

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0

x ∈ Zn

maxx cTx

(2,6)

(3,4)

(2,0)(0,0)

(0,1)

Example

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0.5

x2 ≥ 0

x ∈ Zn

maxx cTx

(2,6)

(3,4)

(2,0)(0,0)

(0,1)

Example

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0.5

x2 ≥ 0

x ∈ Zn

maxx cTx

(2,6)

(3,4)

(2,0)(0.5,0)

(0.5,2.25)

ILP are generally NP-hard

Integer Linear Program

Many combinatorial optimization problems

“Easy” ILP = Easy optimization problem

• Integer Linear Programming

• Duality

• Integer Polyhedron

• Totally Unimodular Matrices

Outline

s.t. A x ≤ b

maxx cTx

Question

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤≥=

s.t. A x ≤ b

maxx cTx

Answer

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤

s.t. A x ≤ b

maxx cTx

Question

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤

s.t. A y = c

miny bTy

≤≥=

s.t. A x ≤ b

maxx cTx

Answer

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤

s.t. A y = c

miny bTy

=

s.t. A x ≤ b

maxx cTx

Question

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤

s.t. A y = c

miny bTy

=s.t. A y = c

miny bTy

y ∈ Zm≤ ≥ =

s.t. A x ≤ b

maxx cTx

Answer

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤

s.t. A y = c

miny bTy

=s.t. A y = c

miny bTy

y ∈ Zm≥

s.t. A x ≤ b

maxx cTx

Duality Relationship

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤

s.t. A y = c

miny bTy

=s.t. A y = c

miny bTy

y ∈ Zm≥

≤

s.t. A x ≤ b

maxx cTx

Duality Relationship

x ∈ Zn

s.t. A x ≤ b

maxx cTx≤

s.t. A y = c

miny bTy

=s.t. A y = c

miny bTy

y ∈ Zm≥

≤ Strict inequality for some problems

• Integer Linear Programming

• Duality

• Integer Polyhedron

• Totally Unimodular Matrices

Outline

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0(2,6)

(3,4)

(2,0)(0,0)

(0,1)

All the vertices of P are integer vectors

P

Integer Polyhedron

Integer polytope is a bounded integer polyhedron

4x1 – x2 ≤ 8

2x1 + x2 ≤ 10

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0(2,6)

(3,4)

(2,0)(0,0)

(0,1)

P

Integer Polyhedron

Integer polyhedron need not be bounded

All the vertices of P are integer vectors

4x1 – x2 ≤ 8

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0(2,6)

(3,4)

(2,0)(0,0)

(0,1)

P

Integer Polyhedron

Integer polyhedron need not be bounded

All the vertices of P are integer vectors

4x1 – x2 ≤ 8

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0

Integer polyhedron need not be bounded

P

Integer Polyhedron

(2,0)(0,0)

(0,1)

All the vertices of P are integer vectors

4x1 – x2 ≤ 8

5x1 - 2x2 ≥ -2

x1 ≥ 0

x2 ≥ 0

Are all polyhedra integer polyhedra?

P

Question

(2,0)(0,0)

(0,1)

All the vertices of P are integer vectors

NO

4x1 – x2 ≤ 8

5x1 - 2x2 ≥ -2

x1 ≥ 0.5

x2 ≥ 0

Are all polyhedra integer polyhedra?

P

Question

(2,0)(0,0)

(0,1)

All the vertices of P are integer vectors

NO

4x1 – x2 ≤ 8

5x1 - 2x2 ≥ -2

x1 ≥ 0.5

x2 ≥ 0

Are all polyhedra integer polyhedra?

Question

(2,0)

All the vertices of P are integer vectors

NO

(0.5,0)

(0.5,2.25) P

Integer polyhedra are very useful

ILP over Integer polyhedron is easy

Integer Polyhedron

Drop the integrality constraints, solve LP

But how can we identify integer polyhedra?

• Integer Linear Programming

• Duality

• Integer Polyhedron

• Totally Unimodular Matrices

Outline

Totally Unimodular Matrix

A is a TUM

For all square submatrix A’ of A

det(A’) is 0, +1 or -1

Question

0

0

2

-1

Is this a TUM?

All elements must be 0, +1 or -1

NO

Question

1

1

1

-1

Is this a TUM?

Determinant of the matrix = -2

NO

Question

1

1

-1

-1

Is this a TUM?

YES.

Question

1

-1

-1

1

Is this a TUM?

YES.

Property

A is a TUM

b is an integer vector, b Z∈ n

Polyhedron P = {x | Ax ≤ b}

P is an integer polyhedron Proof?

Proof Sketch

Let v be a vertex of P.

A’ is a full-rank square submatrix of A

b’ is the corresponding subvector of b

A’v = b’ Why?

Proof Sketch

Let v be a vertex of P.

A’ is a full-rank square submatrix of A

b’ is the corresponding subvector of b

A’v = b’ v = adj(A’)b’/det(A’)

v is integer

Integer +1 or -1

Totally Unimodular Matrix

How can we identify if A is TUM?

O((m+n)3) algorithm. K. Truemper, 1990

We don’t want to identify easy problem instances

We want to identify easy problems

Totally Unimodular Matrix

Some types of matrices are TUM

Easy to prove that they are TUM

Corresponding problems are provably “easy”

We will see that min-cut is one such problem

Questions?