Daphne Koller

Message Passing

Belief PropagationAlgorithm

ProbabilisticGraphicalModels

Inference

Daphne Koller

BD

C

A

2: B,C

1: A,B

3: C,D

4: A,D

C

A

B D

Cluster Graph

Daphne Koller

Passing Messages

2: B,C

1: A,B

3: C,D

4: A,D

C

A

B D2,1 = 1

y1(A,B) y4(A,D)

y2(B,C) y3(C,D)

1,4 = B 2,1(B) y1(A,B)

Daphne Koller

Passing Messages

2: B,C

1: A,B

3: C,D

4: A,D

C

A

B D1,2 =2,1 =

2,3 =3,2 =

3,4 =4,3 =

1,4 =4,1 =

B 2,1(B) y1(A,B)D 3,4(D) y4(A,D)

C 2,3(C) y3(C,D)A 1,4(A) y4(A,D)

B 1,2(B) y2(B,C)D 4,3(D) y3(C,D)

A 4,1(A) y1(A,B)C 3,2(C) y2(B,C)

Daphne Koller

Cluster Graphs• Undirected graph such that:– nodes are clusters Ci {X1,…,Xn}

– edge between Ci and Cj associated with sepset Si,j Ci Cj

• Given set of factors , we assign each k to a cluster C(k) s.t. Scope[k] C(k)

• Define

Daphne Koller

Example Cluster Graph

1: A, B, C

3: B,D,F

2: B, C, D 5: D, E

4: B, EBB

BC

DD

E

1 = 1

2 = 2 3 6

4 = 5

5 = 4

3 = 7

Daphne Koller

Different Cluster Graph

1: A, B, C

3: B,D,F

2: B, C, D 5: D, E

4: B, EBB

B,C

DD

E

Daphne Koller

Message Passing

1: A, B, C

3: B,D,F

2: B, C, D 5: D, E

4: B, EBB

BC

DD

E

Daphne Koller

Belief Propagation Algorithm• Assign each factor k to a cluster C(k)

• Construct initial potentials• Initialize all messages to be 1• Repeat

– Select edge (i,j) and pass message

• Compute

Daphne Koller

Belief Propagation Run

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0 5 10 15 20Iteration #

P(a

1)

True posterior

Daphne Koller

Different BP Run

Daphne Koller

Summary• Graph of clusters connected by sepsets • Adjacent clusters pass information to each

other about variables in sepset– Message from i to j summarizes everything i

knows, except information obtained from j

• Algorithm may not converge• The resulting beliefs are pseudo-marginals• Nevertheless, very useful in practice

Daphne Koller

END END END