A dual ascent framework for Lagrangean decomposition of combinatorial problemsPaul Swoboda1 Jan Kuske2 Bogdan Savchynskyy3

1 IST Austria 2 Heidelberg University, Germany 3 TU Dresden, Germanye-mail: pswoboda@ist.ac.at

Goal and contributionGoal: Efficient dual block coordinate ascent solvers for a wide range of large scale combina-torial problems from computer vision and beyond.Contribution:

We propose a new class of general LP-relaxations of ILP-problems, calledInteger Relaxed Pairwise Separable Linear Programs (IRPS-LP).

(IRPS-LP) generalize the local polytope relaxation for CRFs.(IRPS-LP) allows explicit modelling of allowed configurations, instead of forbidding them with∞-costs.This leads to more compact formulations and lower computational cost.Relaxations of the multicut [4] (poster #75) and graph matching [5] (poster #69 tommorow afternoon)problems can be written as (IRPS-LP).

We analyze dual block coordinate ascent for (IRPS-LP).We prove monotonical improvement of a dual lower bound.Popular algorithms TRWS [2], SRMP [3] and MPLP [1] for inference in CRFs are special cases.

Practical impact:Efficient algorithms for multicut and graph matching have been written in our framework(for extensive details see corresponding posters).C++ implementation available at https://github.com/pawelswoboda/LP_MP.

Problem formulation (IRPS-LP)Factor graph G = (F,E):

factors i ∈ F with variables µi ∈ Xi ⊆ {0,1}di

couplings ij ∈ E ⊆(F2

)with constraints A(i ,j)µi = A(j ,i)µj.

(IRPS-LP):

minµ∈ΛG

k∑i=1

〈θi, µi〉

ΛG :=

{(µ1 . . . µk)

∣∣∣∣ µi ∈ conv(Xi) ∀i ∈ FA(i ,j)µi = A(j ,i)µj ∀ij ∈ E

}.

µ1 ∈ X1 µ2 ∈ X2

µ3 ∈ X3 µ4 ∈ X4

A(1,2)µ1=A(2,1)µ2

A(1,3) µ

1 =A(3,1) µ

3

A(2,4) µ

2 =A(4,2) µ

4

A(3,4)µ3=A(4,3)µ4

Example: MAP-inference in CRFs as (IRPS-LP)CRF:

Graph G = (V,E),label space X =

∏u∈V Xu,

∀u ∈ V: unary costs θu : Xu → R,∀uv ∈ E: pairwise costs θuv : Xu × Xv → R.

MAP-inference:

minx∈X

∑u∈V

θu(xu) +∑

uv∈Eθuv(xuv) .

(IRPS-LP): F = V ∪ E, E = {{u,uv}, {v ,uv} : uv ∈ E}. Simplex: ∆n = {x ∈ Rn+ :∑n

i=1 xi〉 =1}.

minµ∈LG

〈θ, µ〉 :=∑u∈V

〈θu, µu〉 +∑uv∈E

〈θuv , µuv〉

LG =

µu ∈ conv(Xu) : µu ∈ ∆|Xu|,u ∈ Vµuv ∈ conv(Xuv) : µuv ∈ ∆|Xuv |,uv ∈ EA(uv ,u)µuv = A(u,uv)µu :

∑xv∈Xv

µuv(xu, xv) = µu(xu) .

u v

w

CRF

µu ∈ ∆Xu µv ∈ ∆Xv

µw ∈ ∆Xw µvw ∈ ∆Xv ·Xw

µuv ∈ ∆Xu·Xv

µuw ∈ ∆Xu·Xw

µ u(x

u)=

∑ x vµ u

v(x

uv) µ

(xv )=

∑xu µ

uv (xuv )

µ(x

v )=

∑xwµ

vw (xvw )

′µ(xw)=∑

xv µvw(xvw)

µ(x

u)=

∑ x wµ u

w(x

uw)

′µ(xw)=∑

xu µuw(xuw)

CRF as (IRPS-LP)

Example: graph matchingGiven:

CRF.Label spaces Xu ⊂ L are part of an universe.

minx∈X

∑u∈V

θu(xu) +∑uv∈E

θuv(xuv)︸ ︷︷ ︸CRF

s.t. xu 6= xv ∀u 6= v︸ ︷︷ ︸uniqueness constraints

Example: V = {u, v ,w}, L = {l , l ′, l ′′}.Unary factors: Each node takes a label.Label factors: Each label can be takenonce.

∑ =1

∑ =1

∑ =1

∑= 1∑= 1∑= 1

u v w

l

l ′

l ′′

(IRPS-LP):

minµ∈LG,µ̃ 〈θ, µ〉

s.t. µ̃ ∈

∀s ∈ L :∑u∈V

µ̃u(s) ≤ 1

additional (IRPS-LP) factorsfor uniqueness constraint

µu(s) = µ̃s(u), s ∈ Xu couple CRF and uniqueness constraints .

Example: multicutOptimization problem:

Weighted graph G = (V ,E).Find partition of G.

minx∈{0,1}|E|

∑e∈E

θexe , s.t. ∀ cycles C : ∀e′ ∈ C :∑

e∈C\{e′}xe ≥ xe′︸ ︷︷ ︸

cycle inequalities

.

(IRPS-LP):Factors: F = E ∪ 3-cycle(E).Couplings: E = {{e,C} : C ∈ 3-cycle(E),e ∈ C}

Motivation: Dual block coordinate ascent for CRFs

Observation: TRWS [2] is state of the artamong LP-based methods forMAP-inference for CRFs:Contribution: Develop dual blockcoordinate ascent methods inspired byTRWS [2] for (IRPS-LP).

Dual lower boundDualize primal problem w.r.t. coupling constraints:

Primal variables P :=

{µ =

(µ1...µk

): µ ∈

(conv(x1)...conv(Xk)

)}.

Coupling constraints: Aµ = (A(i ,j)µi − A(j ,i)µj)ij∈E = 0.Maximize dual lower bound: maxφ [D(φ) := minµ∈P〈θ, µ〉 + 〈φ,Aµ〉].Reparametrization: θφ := θ + A>φ.

Elementary updates – Admissible messages

For factor i ∈ BF take subset ofneighbors N ⊆ {j : ij ∈ E}.Consider dual variables φi j related tocouplings j ∈ N.Change (φi j)j∈N by ∆ such thatD(φ−∆) ≥ D(φ).

Illustration in the figure:factor 1, neighbors N = {2,3}.

µ1 ∈ X1 µ2 ∈ X2

µ3 ∈ X3 µ4 ∈ X4

φ12

φ13

Elementary updates – Admissible messagesComputation of ∆:

x̂i ∈ arg minxi∈Xi〈θφi , xi〉 : opt. configuration before update.

x̂i ∈ arg minxi∈Xi〈θφ−∆i , xi〉 : opt. configuration after update.

〈θφ−∆i , x̂i〉 ≥ 〈θφi , x̂i〉 : costs do not decrease.

∆ = arg max∆−admissible〈δ, θφ−∆i 〉, δ(s){> 0, x̂i(s) = 1< 0, x̂i(s) = 0

AlgorithmVisit all factors i ∈ F and perform elementary updates:

Special cases: TRWS [2], SRMP [3] and MPLP [1] for MAP-inference in CRFs.

Results: graph matching

10−1 100 101

101

102

103

runtime(s)car

log(en

ergy

-lower

bou

nd)

AMPHUNGARIAN-BP

DD

10−1 100 101

100

101

102

103

runtime(s)motor

log(en

ergy

-lower

bou

nd)

10−1 100 101 102102

103

104

105

runtime(s)worms

log(en

ergy-lower

bou

nd)

For more details please see [5] and corresponding poster #69 on Tuesday afternoon.

Results: multicut

10−2 10−1 100−5,000

−4,800

−4,600

−4,400

−4,200

−4,000

runtime(s)knott-3d-150

energy

MP-CCGC

MC-ILPCC-Fusion-RWSCC-Fusion-RHC

10−1 100 101 102−2.75

−2.74

−2.73

−2.72

−2.71

·104

runtime(s)knott-3d-300

10−1 100 101 102

−8

−7.8

−7.6

·104

runtime(s)knott-3d-450

For more details please see [4] and corresponding poster #75.

References[1] A. Globerson and T. S. Jaakkola. Fixing max-product: Convergent message passing algorithms for

MAP LP-relaxations. In NIPS, pages 553–560, 2007.

[2] V. Kolmogorov. Convergent tree-reweighted message passing for energy minimization. IEEE Trans.Pattern Anal. Mach. Intell., 28(10):1568–1583, 2006.

[3] V. Kolmogorov. A new look at reweighted message passing. IEEE Trans. Pattern Anal. Mach. Intell.,37(5):919–930, 2015.

[4] P. Swoboda and B. Andres. A message passing algorithm for the minimum cost multicut problem. InCVPR, 2017.

[5] P. Swoboda, C. Rother, H. Abu Alhaija, D. Kainmueller, and B. Savchynskyy. Study of Lagrangeandecomposition and dual ascent solvers for graph matching. In CVPR, 2017.

Acknowledgements. The first author was supported by the European Research Council under the European Unions Seventh Framework Programme(FP7/2007-2013)/ERC grant agreement no 616160, the second author by DFG Grant GRK 1653 and the last author by the European Research Council under the

European Union’s Horizon 2020 research and innovation programme (grant agreement No 647769) and the DFG Grant ”ERBI” SA 2640/1-1.