An Efficient Message-Passing Algorithm for the M-Best MAP Problem
Dhruv Batra
(Currently)Research Assistant Professor
TTI-Chicago
(Spring 2013)Assistant Professor
Virginia Tech
Local Ambiguity• Graphical Models
(C) Dhruv Batra 2
x1
x2
…
xn
MAP
Inference
Most Likely AssignmentMAP Problem
Cat
Hat
Global Ambiguity• “While hunting in Africa, I shot an elephant in my pajamas.
How an elephant got into my pajamas, I’ll never know!”
– Groucho Marx (1930)
(C) Dhruv Batra 3
M-Best MAP
• Useful for:– Generating multiple hypotheses when model is inaccurate– Passing on hypotheses to next stage in cascade– Show multiple solutions to users
• Generalization of MAP, thus NP-Hard
(C) Dhruv Batra 4
History
MAP M-Best MAP
(C) Dhruv Batra 5
History
MAP M-Best MAP
Message-Passing Algs- Dynamic Programming- Belief Propagation style
[Pearl ’82], [Lauritzen & Spiegelhalter ‘88],
[Shafer & Shenoy ‘86]
[Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11]
(C) Dhruv Batra 6
History
MAP M-Best MAP
Message-Passing Algs- Dynamic Programming- Belief Propagation style
[Pearl ’82], [Lauritzen & Spiegelhalter ‘88],
[Shafer & Shenoy ‘86]
[Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11]
Linear ProgrammingFormulation
[Schlesinger ‘76], [Wainwright et al. ‘05],
[Komodakis ’07][Fromer & Globerson ’09]
(C) Dhruv Batra 7
History
MAP M-Best MAP
Message-Passing Algs- Dynamic Programming- Belief Propagation style
[Pearl ’82], [Lauritzen & Spiegelhalter ‘88],
[Shafer & Shenoy ‘86]
[Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11]
Linear ProgrammingFormulation
[Schlesinger ‘76], [Wainwright et al. ‘05],
[Komodakis ’07][Fromer & Globerson ’09]
Message-Passing for solving LP
[Schlesinger ‘76], [Wainwright et al. ‘05],
[Kolmogorov ‘06], [Komodakis ’07], [Werner ’07]
(C) Dhruv Batra 8
This Work[Batra UAI ’12]?
Contributions• First message-passing alg for solving M-Best MAP LP
of [Fromer & Globerson NIPS09] • Guaranteed to get exact solution to LP• Orders of magnitude faster than a generic LP solver
(C) Dhruv Batra 9
10 32 100 316 1000 3162 10000 31614 1000000
20
40
60
80
100
120
140
160
LP-solver
Our Approach
#Nodes
Tim
e (s
ec)
Better
Outline
(C) Dhruv Batra 10
Tree-MRFM=2
Tree-MRFM>2
Loopy MRFM=2
Loopy MRFM>2
M
Cycles
- Partition Enumeration Scheme [Fromer & Globerson NIPS09]
- Others
Details in Paper
M=2 M>2 Schemes
Background
• Over-Complete Representation
11(C) Dhruv Batra
x1
x2
…
xnXikxk
……
…
kx1
…
……
… …
kx1
1
1
0
0
0
0
0
0
1
0
0
0
0
1
0
0
Background
• Over-Complete Representation
12(C) Dhruv Batra
x1
x2
…
xnXi
… …
k2x1
100000000000
010000000000
Background• MAP Integer Program
(C) Dhruv Batra 13
Background• MAP Linear Program
• Properties– If LP-opt is integral, MAP is found– LP always integral for trees– Efficient message-passing schemes for solving LP
(C) Dhruv Batra 14
Outline
(C) Dhruv Batra 15
Tree-MRFM=2
Tree-MRFM>2
Loopy MRFM=2
Loopy MRFM>2
M
Cycles
M-Best MAP LP: Tree
(C) Dhruv Batra 16
Spanning-Tree Inequality
[Fromer & Globerson NIPS09]
M-Best MAP LP: Tree
(C) Dhruv Batra 17
~ 106 x 106
Generic LP-solver: CPLEX
[Fromer & Globerson NIPS09]
M-Best MAP LP: Tree• Lagrangian Relaxation
(C) Dhruv Batra 18
Dualize
2-PassBelief Propagation
Similarity-Augmented Energy
M-Best MAP LP: Tree• Lagrangian Relaxation
• Dual Problem
(C) Dhruv Batra 19
2nd Best MAP energy
Concave (Non-smooth)
Lower-Bound on 2nd Best MAP energy
upergradient Ascent
M-Best MAP LP: Tree• Lagrangian Relaxation
• Dual Problem
(C) Dhruv Batra 20
upergradient Ascent
Primal Block Dual Block
primal point
dual point
M-Best MAP LP: Tree• Lagrangian Relaxation
• Dual Problem
• Guarantees– Suitable choice of stepsize solves Lagrangian [Shor ‘85]
– LP => Strong Duality
(C) Dhruv Batra 21
upergradient Ascent
Outline
(C) Dhruv Batra 22
Tree-MRFM=2
Tree-MRFM>2
Loopy MRFM=2
Loopy MRFM>2
M
Cycles
M-Best MAP LP: Loopy-MRFs
(C) Dhruv Batra 23
…,,
M-Best MAP LP: Loopy-MRFs
(C) Dhruv Batra 24
Dualize
…,,
Problems1. Exponentially many Lagrangian Terms2. Collection of factors not a tree
Exponentially Many Terms
(C) Dhruv Batra 25
Primal Block
primal point
Dual Block
dual point
Constraint Management
primal
point
dual point
Tree Subset
upergradient Ascent
Dynamic
…
Exponentially Many Terms
(C) Dhruv Batra 26
Primal Block Dual Block
Constraint Management
primal
point
dual point
Tree Subset
…, ,
upergradient Ascent
Dynamic
Max-Weight Spanning TreeSame as [Fromer & Globerson]
Loopy Graph
(C) Dhruv Batra 27
Problems1. Exponentially many Lagrangian Terms2. Collection of factors not a tree
…
Dual Decomposition
M-Best MAP LP: Loopy-MRFs• Guarantees
– Dynamic Supergradient Ascent w/ Max-Violation Oracle solves Lagrangian Relaxation [Emiel & Sagastizabal ‘08]
– LP => Strong Duality
(C) Dhruv Batra 28
Experiments• Synthetic Data
– Trees– Grid Graphs– Energies sampled from Gaussians
• Methods– STEELARS: Spanning TREE LAgrangian Relaxation Scheme
[Proposed]
– STRIPES [Fromer & Globerson NIPS09]– BMMF [Yanover & Weiss NIPS03]– NILSSON [Nilsson Stat. & Comp. 98]
(C) Dhruv Batra 29
Results: Tree-MRFs
(C) Dhruv Batra 30
Better
Results: Loopy-MRFs
(C) Dhruv Batra 31
Better
Extension: Diverse M-Best
(C) Dhruv Batra 32
Diverse M-Best Solutions in MRFsBatra, Yadollahpour, Guzman, ShakhnarovichECCV 2012
Task-Specific
Diversity
Extension: Diverse M-Best• Interactive Segmentation
(C) Dhruv Batra 33
Image + Scribbles 2nd Best Mode2nd Best MAPMAP
1-2 Nodes Flipped 100-500 Nodes Flipped
Extension: Diverse M-Best
(C) Dhruv Batra 34
Input MAP Best Mode
Conclusions• First message-passing alg for solving M-Best MAP LP • Guaranteed to get exact solution to LP• Orders of magnitude faster than a generic LP solver
• Extension: – Diverse M-Best Solutions in MRFs
Batra, Yadollahpour, Guzman, ShakhnarovichECCV 2012
(C) Dhruv Batra 35
Thank you!
(C) Dhruv Batra 36
Results: Tree-MRFs
(C) Dhruv Batra 37
Quality of Solutions: Loopy-MRFs
(C) Dhruv Batra 38
Results: Loopy-MRFs
(C) Dhruv Batra 39
Applications• What can we do with multiple solutions?
– More choices for “human/expert in the loop”
(C) Dhruv Batra 40
Applications• What can we do with multiple solutions?
– More choices for “human/expert in the loop”– Input to next system in cascade
(C) Dhruv Batra 41
Step 1 Step 2 Step 3Top M
hypotheses
Top M
hypotheses
Applications• What can we do with multiple solutions?
– More choices for “human in the loop”– Rank solutions
(C) Dhruv Batra 42
[Carreira and Sminchisescu, CVPR10]
State-of-art segmentation on PASCAL Challenge 2011
~10,000