Combinatorial Optimization - Rutgers Universityguyk/flow.pdf · 2018-12-11 · Combinatorial...

Combinatorial Optimization 1

CombinatorialOptimization

Guy Kortsarz

Size of an instance

Running time. f(n) with n: Size of the input

If the input has no numbers then size of input

equals # of items in the input.

If input has large numbers size is # of items

· log(max).

Example:

Input: a number n

Question: is n a prime?

The size of the input is logn

The naive algorithm

The naive algorithm tries all dividers of n

between 3 and n/2.

Actually, it is enough to stop at√n.

The algorithm makes√n divisions.

It runs in time EXPONENTIAL in the input

Polynomially solvable problems

• Given G(V,E) and s ∈ V find the shortest

(least number of edges) path from s to

every v 6= s

• Given G(V,E), w(e) > 0 for every e ∈ E.

find a min-cost spanning tree.

• Given G(V,E) find if G admits a simple

path that covers all edges (Euler path).

• Minimum cost cut: Given a graph G(V,E)

split V into V1 ∪ V2 = V , V1 ∩ V2 = ∅ sothat e(V1, V2) is minimum

Some problems that are probably

• ∆−TSPA salesperson comes to the USA.

Wants to drive from state to state.

Wants to cover all states and get

back to its original location. How to

do it in minimum cost?

Remark: The weights usually obey

the triangle inequality. Even if this

holds, hard problem

• Vertex cover: find min-size (cost) collection

of vertices touching all edges

• k−center: given an undirected complete

graph with distance between every two

vertices and integer k, choose a subset of

size k of centers. Minimize the maximum

distance of any non-center to its closest

center.

More problems that seem hard

• Maximum size (weight) independent sets.

An independent set is collection of vertices

which are pairwise non-neighbors

• Minimum size coloring. A coloring, a

disjoint partition of the graph to IS. The

size of the coloring is the number of sets in

the partition

• Maximum cost cut

• Steiner: Given an edge weighed graph and

a subset S ⊆ V of the vertices, find a

min-cost tree (or arborescence if directed)

containing S.

The decision versions of TSP, VC,

k-Center are in NP

• Easy to verify a solution

• Too many candidates for exhaustive search.

For TSP: (n− 1)! candidate solutions

For IS (independent set): 2n

For VC: 2n

For MC: Roughly n!

For k = n/2 in k−center(

which is roughly

An interesting example

The problem:

Given n find if it is a prime or not can be

solved in time O(log10.5 n), thus time

polynomial in the input.

The actual time may be O(log6 n).

Due to Agrawal Kayal Saxena. Solved (only) in

BUT: Given n = p · q (its known that p, q are

primes) it seems hard to find the factors

Why are we interested in NPC

problems?

• Industrial applications: Substantial

financial implications for a practical

polynomial algorithm for some NPC

problems

• Security and weapons: a practical

polynomial algorithm implies we can factor

large numbers (by binary search). Can

break most codes used....

• Who wants to be a millionaire? One

of the seven millennium problems. A

1000000 dollars reward.

I can even trace a relation to “Alice in

wonderland”: (Dodgson elections).

A central problem: maximum flow

in networks

The input for the maximum flow problem is a

directed graph G(V,E) with capacity c(e) > 0

for every e ∈ E and two distinguished nodes: a

source s and a target t

We will assume no edge enters s and no edge

leaves t

A legal flow is a flow function f : E 7→ R+ that

obeys the following restrictions:

1. Capacity constraint: f(e) ≤ c(e) for every

e ∈ E

2. Flow conservation constraint: for every

v 6= s, v 6= t,∑

e∈in(v)

f(e) =∑

e∈out(v)

The goal in a maximum flow

problem

The scenario described is that some

commodities (could be fluids or electricity or

automobiles) from s to t

The middle vertices do not create nor waste

Hence the goal is to maximize:

e∈in(t)

Example

Figure 1: A legal flow

A basic flow property

The flow can be measured at every cut.

A cut (S, S̄) is a partition of V into V = S ∪ S̄,

S ∩ S̄ = ∅ so that s ∈ S and t ∈ S̄.

The flow along the cut is:

e∈e(S,S̄)

f(e)−∑

e∈e(S̄,S)

This is called the net flow along the cut

Example

1+2+1−1−1=3

Figure 2: A cut and the net flow along the cut

All net flows are equal

Z−Y=X−W

Z+W=X+Y

Figure 3: Moving one vertex from side to side

makes no difference

The net flow does not depend on

the cut

Since moving a single vertex from left to right

does not change the flow (and similarly, moving

a vertex from right to left does not change the

flow) by moving vertices (except for s, t)

around we can get any proper partition.

Thus, for any S, S̄,

f(S) =∑

e∈S 7→S̄ −∑

e∈S̄ 7→S = |f |For example, the value of the flow also equals

the amount of flow that goes out of s

The capacity of a cut

Given a cut S, S̄, s ∈ S, and t ∈ S̄ the capacity

of the cut is defined by:

c(S)∑

e∈S 7→S̄

Remarks:

• The capacity of a cut depends on the

graph and the partition. In does not

depend on any specific flow

• In the capacity of a cut (unlike net flow)

we only take edges that go from S to S̄

Every cut capacity bounds any

flow value

Claim 1 For every cut S, S̄, and legal flow f ,

|f | ≤ c(S).

Proof.

|f | =∑

e∈e(S,S̄)

f(e)−∑

e∈e(S̄,S)

≤∑

e∈e(S,S̄)

f(e) ≤∑

e∈e(S,S̄)

= c(S)

The world of the dwarfs and giants

Every cut capacity bounds from above any flow

value.

c(S’)

f’’f * S *

Figure 4: The capacity of every cut is at least

as large as the value of every flow

If there are f∗ and S∗ so that |f∗| = c(S∗) then

the flow is maximum and the capacity is

minimum

The min-cut max-flow theorem

We are going to prove:

Theorem 1 For every network, the min-cut

equals the max-flow

This was proved by Ford and Fulkerson in

1955. The proof used the first ever algorithm

for finding the max-flow.

The min-cut is also discovered

How do we compute a max-flow?

Improvements: the simple possibility:

Figure 5: Any path of non-saturated edges from

s to t gives a unit improvement of flow

Some time flow reduction on edges

Figure 6: Flow on b 7→ a is redirected into b 7→ t

The residual graph

M M−11

1,M1,M

Figure 7: The residual graph

The residual graph: definition

If an edge e = (u, v) has capacity c and flow f

1. It has an edge in the same direction of e iff

2. It has an edge in the reverse direction of e

(this means the edge (v, u)) if and only if

3. If f < c then the edge in the same

direction as e has capacity c− f .

4. If f > 0 the edge in the reverse direction to

e has capacity f .

Intuition

Idea: we are splitting the edges into original

and reverse edges.

Original edges: how much can we increase f?

No more than c− f .

Reverse edges: how much can we decrease f?

By no more than f .

Instead of going in a “no entry” street (on

edges in the reverse direction) we simply add

the anti-parallel edge

s to t paths in R(G, f)

Claim: if there is an s to t path in P in

R(G, f) we can increase f .

The idea: let

d1 = mine that is an original edge in P

c(e)− f(e),

d1 = mine that is a reverse edge in P

d = min{d1, d2}.

It must be that d > 0 because there is a path in

R(G, f) between s and t.

Add d units of flow to every original edge in P

and subtract d flow units from every reverse

The flow is still legal

+d −d

−d −d

−d +d

Figure 8: The balance is always preserved

The flow has increased

Since the path is from s to t it starts with an

original edge (s, x).

The flow along (s, x) was increased by d.

Thus the value of the flow is larger: the value

of the flow can be measured at s.

But: if no s to t path exists in R(G, f) does

this mean that f is a maximum flow? Yes!

Proof of the min-cut max-flow

theorem

Consider all vertices that are reachable in

R(G, f) from s:

Figure 9: The flow along the cut of reachable

from s vertices

The cut that equals the flow

The vertex s with all the other vertices it can

reach in R(G, f) is a subset S ⊆ V

The subset S contains s but not t

All edges that go from S to S̄ are saturated

All edges that go from S̄ to S have flow zero

Therefore c(S, S̄) = |f |Thus, the flow is maximum and the cut is

minimum

We still need to show an algorithm that finds a

flow with no augmenting paths in polynomial

in the input time

The Ford and Fulkerson algorithm

Find a directed path between s and t in

R(G, f). How? See later (BFS).

Augment the flow along the path by the

bottleneck.

Iterate.

The following example shows that this

algorithm can have bad running time:

Figure 10: Flow on b 7→ a is redirected into b 7→ t

The running time of FF and better

algorithms

The above example shows the running time can

be O(M).

Thus not polynomial in the size of the input

Two choices give polynomial time algorithms:

1. Select the least number of edges paths

when you augment (later we see how to

find such path)

2. Select the highest bottleneck path

The Edmonds-Karp algorithm

Select BFS paths. Requires O(|E|).Claim: The distance in R(G, f) from s to x can

never decrease

Try to prove it at home. Provide proof later

Claim: The number of augmenting paths is

O(|E| · |V |)Proof: the distance between s and t can be

only 1 or 2, and so on up to |V | − 1.

How much time can the distance stay at k?

Claim: at most |E| augmenting paths can the

distance stay at k.

If we prove it we get O(|E| · |V |) augmentations

for O(|E|2|V |) running time

A single augmentation reverses an edge

This edge can no longer serve in any shortest

Figure 11: An edge that no longer can be used

Therefore after O(|E|) augmenting paths there

is no length k path from s to t

The bottleneck algorithm

Let R be the difference between the maximum

flow and the flow at the moment.

Claim: there is always a single path that

augments the flow by at least R/m for m = |E|.Before we prove it, let us see that this gives:

Claim: the bottleneck algorithm runs in

O (m · (logn+ logM)) time with M the

maximum capacity. Note: this is a polynomial

in the input time.

Let f be the maximum flow value.

After the next augmentation,

Ri+1 = R

1− 1

Therefore after m · ln f augmentations:

Ri+1 ≤ f ·(

1− 1

)m·ln f

≤ 1.

We show this by noting that(

1− 1

)m ln f

1− 1

)m)ln f

≤ f ·(

Flow decomposition

The path claim: Given some flow f with value

|f | = x we show that we can select an s to t

path and a number 0 < d ≤ x so that:

: There exists a path P from s to t in G and

all the edges on the path carry at least d

units of flow

At least one edge in P carries d units of flow

exactly

If we can do that, we get the bottleneck claim

The path claim implies the

bottleneck claim

Find P as promised in the path claim

Zero the flow on P

Iterate.

The process defines at most m paths because

for every P chosen the flow of at least one edge

is zeroed.

Conclusion: the reverse augmentation shows

that we may augment the flow to maximum in

at most m paths (always!).

Let |f | be the maximum flow and R the current

one. By averaging, at least one of the paths

carries (|f | −R)/m flow units or more

Remark: there are m paths that ONLY

INCREASES FLOW ALONG EDGES.

The path claim

Start with |f |. Let e1 = (v, t) be an edge

entering t. Decrease the flow on this edge by 1.

There most be an edge entering v with

positive flow. Choose e2 = (u, v), f(e2) > 0 and

decrease the flow on e2 by 1.

There must be still flow entering u. Select e3

entering u and reduce the flow along e3 by 1,

A problem that may happen is the formation of

cycles

A cycle

Figure 12: A cycle is formed

Proof, continued

But cycles can be discarded and the process

can continue.

Until we get to s.

We found a path from s to t and define d to be

the least flow on this path.

Thus the path claim and the decomposition

claim both hold.

Question: why do we not get O(|E|2) time

algorithm?

The BFS algorithm

BFS(G)

1. Q← {s}

2. For all v ∈ V \ {s} dist(s, v)←∞.

dist(s, s)← 0.

3. While Q 6= ∅ do(a) v ← Delete(Q).

(b) u← N(v)

(c) While u 6= Null do:

i. If dist(s, u) =∞ then

A. dist(s, u)← dist(s, v) + 1

B. Add(Q, u)

ii. u← next(u)

4. Output the distance vector.

Maximum independent sets in

bipartite graphs

Given a graph G(V,E), a set W ⊆ V is an

independent set if there are no edges internal to

A graph is bipartite if V can be split into two

independent sets V = V1 ∪ V2 and V1 ∩ V2 = ∅Let G(V1, V2, E) be a bipartite graph.

We show how to find a maximum independent

set in G.

The flow network

Form the following flow network: direct the

edges of E from V1 to V2.

Add a source s and a directed edge from s to

each v1 ∈ V1 with capacity 1.

Add a directed edge from each vertex in V2 to t

with capacity 1.

Give all the edges of E capacity ∞.

The minimum cut

Consider the minimum cut in the network.

Let V s1 be the vertices of V1 in the side of s in

the cut, and V t1 = V \ V s

1 be the vertices of V1

in the side of t in the cut.

Similarly for t.

V_1^t V_2^t

Figure 13: The cut

A crucial property

The cut containing in the left side {s} and in

the right side V \ {s} has capacity of |V1|: thereare |V1| capacity 1 edge from s to all of V1.

Any edge from V S1 to V t

2 has capacity ∞.

Thus, it can not be the case that this edge will

participate in a minimum cut, as there are

finite cuts.

So there are no edges from any vertex in V S1 to

any vertex from V t2

The size of the cut

We show that the capacity of the cut is

|V t1 |+ |V s

Check which edges go from the side of s in the

cut to the side of t. There are no edges from

V s1 to V t

2 by the previous answer.

The edges from V t1 to V s

2 are in the “wrong”

direction (from the side of t to the side of s and

not vice-versa).

There certainly are no edges from V s1 to V s

V s2 to V t

2 , because the graph is bipartite.

There are edges from s to V t1 , each edge of

capacity 1, so |V t1 | in total, and similarly, from

V s2 to t, with a capacity of V s

2 in total.

So, the minimum cut (or maximum flow)

equals |V t1 |+ |V s

1∪ V t

2is a maximum independent

V s1 ∪ V t

2 is an independent set.

Under this restriction, we need to minimize

|V t1 |+ |V s

2 |, because this is the capacity of the

So, we maximize

(|V | − (|V t1 |+ |V s

2 |)) = |V s1 |+ |V t

2 |So V s

1 ∪ V t2 is a maximum independent set

Finding min-cuts

The global minimum cut on an UNDIRECTED

graph is the partition S, .S̄, S, S̄ 6= ∅ so that

e(S, S̄) = {e = (u, v) : u ∈ S, v ∈ S̄} isminimum.

If we know x ∈ S, y ∈ S̄, the following flow

network gives the cut:

1. Every edge e = (u, v) is replaced by two

edges u 7→ v and v 7→ u

2. Let x be source and y be sink

3. Compute the max flow, and min-cut

This takes O(n2) cut computation.

Improvement: fix x to be any vertex. Search

only for y (n flow computations)

A better algorithm, by Karger

Let G be an undirected graph with parallel

edges allowed.

1. If the graph has only 2 vertices a, b return

the edge(s) between a and b

2. Choose an edge at random

Remark: A pair of vertices is more likely

to be chosen if it has many parallel edges

3. Contract the pair of vertices. Retain

parallel edges but delete self loops

4. Let G′ be the new graph

5. Recurse on G′

Example

AF EDC

ABF CDE

P(MINIMUM)=6/7*5/6*4/5*2/3

Figure 14: The Karger algorithm

The probability that the min-cut

results

Pr(A1 ∩A2 . . . ∩AK) = Pr(A1) · Pr(A2 | A1)

· Pr(A3 | A1 ∩ A2) · . . .

Say that the min-size of the cut is k

So, deg(v) ≥ k for every v.

So |E| ≥ n · k/2Therefore, the probability that an edge in the

cut is not contracted is:

1− k

nk/2= 1− 2

The probability that the result is

the min-cut, continued

Claim: the degrees of vertices after

contractions are also at least k

Proof: The edges touching a “super-vertex”

give a cut.

Therefore: number of edges after the first

contraction, at least k · (n− 1)/2

Probability contracting a cut edge at least

1− 2

n− 1.

Computation

1− 2

n− 1

1− 2

n− 2

· . . .

1− 2

n− 2

n− 3

n− 1

n− 4

n− 2

5· 24· 13

=2 · 1

n · (n− 1)

A randomized alg with one sided

error probability

Run the algorithm n(n− 1)/2 times.

The probability that non of those runs give the

min-cut is at most:

1− 2

n(n− 1)

)n(n−1)/2

If we run it n2 · (n− 1)/2 times probability of

error at most exp(−n)

Combinatorial Optimization - Rutgers Universityguyk/flow.pdf · 2018-12-11 · Combinatorial...

Documents

Geometric Algorithms and Combinatorial Optimization

Online Stochastic Combinatorial Optimization

Methods and Models for Combinatorial Optimization · Methods and Models for Combinatorial Optimization Introduction Luigi De Giovanni, Marco Di Summa ... combinatorial optimization

Integer and Combinatorial Optimization: Introductioneaton.math.rpi.edu/faculty/Mitchell/courses/matp... · Integer and Combinatorial Optimization Combinatorial optimization An optimization

Optimal' Hopfield Network for Combinatorial Optimization ... · C. Combinatorial Optimization Problems with Linear Cost Function Combinatorial optimization problems we attack in

Learning Combinatorial Optimization Algorithms over Graphs · Learning Combinatorial Optimization Algorithms over Graphs ... combinatorial structure hof S. 5. A generic ... the partial

Computing in Combinatorial Optimization

P2P Combinatorial Optimization - Department of … · P2P Combinatorial Optimization, ... •Continuous or Combinatorial •Search ... • Every iteration of the algorithm corresponds

Introduction to Combinatorial Optimization

Integer and Combinatorial Optimization - Lehigh Universitycoral.ie.lehigh.edu/~ted/files/papers/ICO-EORMS11.pdf · Integer and Combinatorial Optimization ... Integer optimization

Combinatorial Optimization - MA4502 · PDF fileCombinatorial Optimization MA4502 ... Basics of Complexity Theory 2. ... Randomized Algorithms Combinatorial Optimization April 14, 2015

Research trends in Combinatorial Optimization - · PDF fileResearch trends in Combinatorial Optimization ... 2 Combinatorial Optimization Complexity aspects ... algorithms and applications

Combinatorial Optimization for University Examination ... · Index Terms—combinatorial optimization, examination proctor assignment, mixed integer programming, optimization in university

Learning Combinatorial Optimization Algorithms over Graphspapers.nips.cc/paper/7214-learning-combinatorial... · Combinatorial optimization problems over graphs arising from numerous

Integer and Combinatorial Optimization: Algorithm …eaton.math.rpi.edu/.../05A_complexitybeameripad.pdfInteger and Combinatorial Optimization: Algorithm Complexity John E. Mitchell

Message Passing and Combinatorial Optimization · combinatorial optimization. This thesis is concerned with the application of graphical models in solving combinatorial optimization

Combinatorial Optimization - GBV

Modelling and solving complex combinatorial optimization ...yde/Papers/thesis_Trung2015.pdf · Combinatorial optimization is a branch of optimization. Its domain is Combinatorial

Metaheuristics in Combinatorial Optimization

Combinatorial Optimization in Clustering