An Efficient Message-Passing Algorithm for the M-Best MAP Problem

An Efficient Message-Passing Algorithm for the M-Best MAP Problem

Dhruv Batra

(Currently)Research Assistant Professor

TTI-Chicago

(Spring 2013)Assistant Professor

Virginia Tech

Local Ambiguity• Graphical Models

(C) Dhruv Batra 2

x1

x2

…

xn

MAP

Inference

Most Likely AssignmentMAP Problem

Cat

Hat

Global Ambiguity• “While hunting in Africa, I shot an elephant in my pajamas.

How an elephant got into my pajamas, I’ll never know!”

– Groucho Marx (1930)

(C) Dhruv Batra 3

M-Best MAP

• Useful for:– Generating multiple hypotheses when model is inaccurate– Passing on hypotheses to next stage in cascade– Show multiple solutions to users

• Generalization of MAP, thus NP-Hard

(C) Dhruv Batra 4

History

MAP M-Best MAP

(C) Dhruv Batra 5

History

MAP M-Best MAP

Message-Passing Algs- Dynamic Programming- Belief Propagation style

[Pearl ’82], [Lauritzen & Spiegelhalter ‘88],

[Shafer & Shenoy ‘86]

[Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11]

(C) Dhruv Batra 6

History

MAP M-Best MAP

Message-Passing Algs- Dynamic Programming- Belief Propagation style

[Pearl ’82], [Lauritzen & Spiegelhalter ‘88],

[Shafer & Shenoy ‘86]

[Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11]

Linear ProgrammingFormulation

[Schlesinger ‘76], [Wainwright et al. ‘05],

[Komodakis ’07][Fromer & Globerson ’09]

(C) Dhruv Batra 7

History

MAP M-Best MAP

Message-Passing Algs- Dynamic Programming- Belief Propagation style

[Pearl ’82], [Lauritzen & Spiegelhalter ‘88],

[Shafer & Shenoy ‘86]

[Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11]

Linear ProgrammingFormulation

[Schlesinger ‘76], [Wainwright et al. ‘05],

[Komodakis ’07][Fromer & Globerson ’09]

Message-Passing for solving LP

[Schlesinger ‘76], [Wainwright et al. ‘05],

[Kolmogorov ‘06], [Komodakis ’07], [Werner ’07]

(C) Dhruv Batra 8

This Work[Batra UAI ’12]?

Contributions• First message-passing alg for solving M-Best MAP LP

of [Fromer & Globerson NIPS09] • Guaranteed to get exact solution to LP• Orders of magnitude faster than a generic LP solver

(C) Dhruv Batra 9

10 32 100 316 1000 3162 10000 31614 1000000

20

40

60

80

100

120

140

160

LP-solver

Our Approach

#Nodes

Tim

e (s

ec)

Better

Outline

(C) Dhruv Batra 10

Tree-MRFM=2

Tree-MRFM>2

Loopy MRFM=2

Loopy MRFM>2

M

Cycles

- Partition Enumeration Scheme [Fromer & Globerson NIPS09]

- Others

Details in Paper

M=2 M>2 Schemes

Background

• Over-Complete Representation

11(C) Dhruv Batra

x1

x2

…

xnXikxk

……

…

kx1

…

……

… …

kx1

1

1

0

0

0

0

0

0

1

0

0

0

0

1

0

0

Background

• Over-Complete Representation

12(C) Dhruv Batra

x1

x2

…

xnXi

… …

k2x1

100000000000

010000000000

Background• MAP Integer Program

(C) Dhruv Batra 13

Background• MAP Linear Program

• Properties– If LP-opt is integral, MAP is found– LP always integral for trees– Efficient message-passing schemes for solving LP

(C) Dhruv Batra 14

Outline

(C) Dhruv Batra 15

Tree-MRFM=2

Tree-MRFM>2

Loopy MRFM=2

Loopy MRFM>2

M

Cycles

M-Best MAP LP: Tree

(C) Dhruv Batra 16

Spanning-Tree Inequality

[Fromer & Globerson NIPS09]

M-Best MAP LP: Tree

(C) Dhruv Batra 17

~ 106 x 106

Generic LP-solver: CPLEX

[Fromer & Globerson NIPS09]

M-Best MAP LP: Tree• Lagrangian Relaxation

(C) Dhruv Batra 18

Dualize

2-PassBelief Propagation

Similarity-Augmented Energy

M-Best MAP LP: Tree• Lagrangian Relaxation

• Dual Problem

(C) Dhruv Batra 19

2nd Best MAP energy

Concave (Non-smooth)

Lower-Bound on 2nd Best MAP energy

upergradient Ascent

M-Best MAP LP: Tree• Lagrangian Relaxation

• Dual Problem

(C) Dhruv Batra 20

upergradient Ascent

Primal Block Dual Block

primal point

dual point

M-Best MAP LP: Tree• Lagrangian Relaxation

• Dual Problem

• Guarantees– Suitable choice of stepsize solves Lagrangian [Shor ‘85]

– LP => Strong Duality

(C) Dhruv Batra 21

upergradient Ascent

Outline

(C) Dhruv Batra 22

Tree-MRFM=2

Tree-MRFM>2

Loopy MRFM=2

Loopy MRFM>2

M

Cycles

M-Best MAP LP: Loopy-MRFs

(C) Dhruv Batra 23

…,,

M-Best MAP LP: Loopy-MRFs

(C) Dhruv Batra 24

Dualize

…,,

Problems1. Exponentially many Lagrangian Terms2. Collection of factors not a tree

Exponentially Many Terms

(C) Dhruv Batra 25

Primal Block

primal point

Dual Block

dual point

Constraint Management

primal

point

dual point

Tree Subset

upergradient Ascent

Dynamic

…

Exponentially Many Terms

(C) Dhruv Batra 26

Primal Block Dual Block

Constraint Management

primal

point

dual point

Tree Subset

…, ,

upergradient Ascent

Dynamic

Max-Weight Spanning TreeSame as [Fromer & Globerson]

Loopy Graph

(C) Dhruv Batra 27

Problems1. Exponentially many Lagrangian Terms2. Collection of factors not a tree

…

Dual Decomposition

M-Best MAP LP: Loopy-MRFs• Guarantees

– Dynamic Supergradient Ascent w/ Max-Violation Oracle solves Lagrangian Relaxation [Emiel & Sagastizabal ‘08]

– LP => Strong Duality

(C) Dhruv Batra 28

Experiments• Synthetic Data

– Trees– Grid Graphs– Energies sampled from Gaussians

• Methods– STEELARS: Spanning TREE LAgrangian Relaxation Scheme

[Proposed]

– STRIPES [Fromer & Globerson NIPS09]– BMMF [Yanover & Weiss NIPS03]– NILSSON [Nilsson Stat. & Comp. 98]

(C) Dhruv Batra 29

Results: Tree-MRFs

(C) Dhruv Batra 30

Better

Results: Loopy-MRFs

(C) Dhruv Batra 31

Better

Extension: Diverse M-Best

(C) Dhruv Batra 32

Diverse M-Best Solutions in MRFsBatra, Yadollahpour, Guzman, ShakhnarovichECCV 2012

Task-Specific

Diversity

Extension: Diverse M-Best• Interactive Segmentation

(C) Dhruv Batra 33

Image + Scribbles 2nd Best Mode2nd Best MAPMAP

1-2 Nodes Flipped 100-500 Nodes Flipped

Extension: Diverse M-Best

(C) Dhruv Batra 34

Input MAP Best Mode

Conclusions• First message-passing alg for solving M-Best MAP LP • Guaranteed to get exact solution to LP• Orders of magnitude faster than a generic LP solver

• Extension: – Diverse M-Best Solutions in MRFs

Batra, Yadollahpour, Guzman, ShakhnarovichECCV 2012

(C) Dhruv Batra 35

Thank you!

(C) Dhruv Batra 36

Results: Tree-MRFs

(C) Dhruv Batra 37

Quality of Solutions: Loopy-MRFs

(C) Dhruv Batra 38

Results: Loopy-MRFs

(C) Dhruv Batra 39

Applications• What can we do with multiple solutions?

– More choices for “human/expert in the loop”

(C) Dhruv Batra 40

Applications• What can we do with multiple solutions?

– More choices for “human/expert in the loop”– Input to next system in cascade

(C) Dhruv Batra 41

Step 1 Step 2 Step 3Top M

hypotheses

Top M

hypotheses

Applications• What can we do with multiple solutions?

– More choices for “human in the loop”– Rank solutions

(C) Dhruv Batra 42

[Carreira and Sminchisescu, CVPR10]

State-of-art segmentation on PASCAL Challenge 2011

~10,000