Belief Propagation in Large, Highly Connected

Graphs for 3D Part-Based Object Recognition

Frank DiMaio and Jude ShavlikComputer Sciences Department

University of Wisconsin – MadisonUSA

Part-Based Object Recognition

A part-based model describes an object using a pairwise Markov Field(Felzenszwalb et al 2000, Sudderth et al 2004, Isard 2003)

Object described using Undirected part graph G=(V,E) Vertex potential functions Edge potential functions

Part-Based Object Recognition

Probability of a configuration U={ui} – given an image I – is the product of potential functions

edges

( )st s ts t

ψ u ,u

vertices

( | )s ss

u I

( | )P U I

For part-based object recognition Skeletal graph for tightly coupled parts Occupancy graph ensures no other parts

collide in 3D space

Inferring Part Locations with Belief Propagation (BP) Want to find part configuration maximizing

product of potential functions

Use belief propagation (BP) to approximate marginal distributions

Iterative, message-passing method (Pearl 1988)

A message, mi→j, from part i to part j indicates where i currently expects to find j

Belief Propagation Example

b( torso | image)

b( head | image)

b( left arm | image)

b( left leg | image)

b( right arm | image)

b( right leg | image)

Belief Propagation Example

m head→torso(torso)

b( torso | image)

b( head | image)

mR.leg→torso

mL.leg→torso

mR.arm→torso

mL.arm→torso

Belief Propagation Example

b( torso | image)

b( head | image)

b( left arm | image)

b( left leg | image)

b( right arm | image)

b( right leg | image)

What if the Graph has Thousands of Parts? In a graph with N parts and E edges

BP running time and memory requirements O(E) Skeleton graph typically sparse – O(N) edges Occupancy graph fully connected – O(N 2) edges

In very large graphs, O(N 2) runtime intractable

AggBP (our system) approximates O(N 2) occupancy messages using O(N) messages

Message Approximation Illustrated

2

3

5 71

6

8

3

6

5

4

441

1 4

bm

442

2 4

bm

448

8 4

bm

4

477 4

bm

4 4b

4 4b

4 4b 4 4b

Message Approximation Illustrated

2

3

5 71

6

8

4

Accumulator

Experiment I: Density Map InterpretationGLU TYR PHE THR LEU GLN ILE

ARG GLY ARG GLU ARG PHE…

GLY31ALA30

GLN29ALA28

LYS26

LEU25GLU24

LEU23

GLU21

ASN20

LEU19

GLU18ARG17

PHE16

MET15

GLU14PHE13

ARG12

GLU11ARG10

GLY9ARG8

ILE7

GLN6

LEU5

THR4

PHE3

TYR2

GLU1

ALA22

ASP27

LoopyBP vs. AggBP:Runtime

Number of Parts

Nor

mal

ized

Run

time

0

5

10

15

20

25

30

15 25 35 45 55 65 75 85 95

AggBP

LoopyBP

LoopyBP vs. AggBP: Accuracy

BP iteration

0

2

4

6

8

10

0 5 10 15 20

AggBP

LoopyBP RM

S E

rror

Experiment II: Synthetic Graph Generator

increase branching factor

allow spatialoverlap

vary radii

[skeleton graph]

LoopyBP vs. AggBP:Accuracy

0 1 2 5

stdev(part size)

RM

S E

rror

0

2

4

6

0 2 4

LoopyBP

AggBP

Conclusions AggBP makes belief propagation tractable in

large, highly connected graphs

For part-based modeling, runtime and storage is reduced from O(N 2) to O(N)

AggBP’s solutions on two datasets are as good as standard BP’s in less time

Acknowledgements Dr. George Phillips NLM Grant 1R01 LM008796 NLM Grant 1T15 LM007359