On Distributing a Bayesian Network

1

On Distributing a Bayesian On Distributing a Bayesian NetworkNetwork

Thor Whalen

Metron, Inc.

2

Major discussion points

• Functionality afforded by old code

• Issues and problems with old approach

• Functionality trying to add

• Functionality successfully added

• What is not yet working correctly

• What sill have to try to add

3

Outline

• Need a Review of JT?

• Distributing a BN using the JT: Issues.

• Directions for Solutions

• Matlab: New functionality and experiments

• And now what?

4

a

b c

d e

f

g

h

Secondary Structure/Junction Tree• multi-dim. random variables• joint probabilities (potentials)

Bayesian Network• one-dim. random variables• conditional probabilities

abd

ade

ace

ceg

eghdef

ad ae ce

de eg

5

Building a Junction Tree

DAG

Moral Graph

Triangulated Graph

Junction Tree

Identifying Cliques

6

Step 1: Moralization

a

b c

d e

f

g

h

a

b c

d e

f

g

h

a

b c

d e

f

g

h

1. For all w V:• For all u,vpa(w) add an edge e=u-v.

2. Undirect all edges.

GMG = ( V , E )

7

Step 2: Triangulation

Add edges to GM such that there is no cyclewith length 4 that does not contain a chord.

NO YES

a

b c

d e

f

g

h

a

b c

d e

f

g

h

GM GT

8

a

b

d

a

c

e

d e

f

a

d e

e

g

h

c

e

g

a

b c

d e

f

g

h

Bayesian NetworkG = ( V , E )

a

b c

d e

f

g

h

a

b c

d e

f

g

h

Moral graph GM Triangulated graph GT

abd

ade

ace

ceg

eghdef

ad ae ce

de eg

seperators

Junction graph GJ (not complete)e.g. ceg egh = eg Cliques

e

e

e

a

e

9

Junction graph GJ (not complete)

abd

ade

ace

ceg

eghdef

ad ae ce

de eg

e

e

e

a

e

abd

ade

ace

ceg

eghdef

ad ae ce

de eg

Junction tree GJT

There are several methods to find MST.

Kruskal’s algorithm: choose successively a link of

maximal weight unless it creates a cycle.

10

a

b

d

a

c

e

d e

f

a

d e

e

g

h

c

e

g

abd

ade

ace

ceg

eghdef

ad ae ce

de eg

sepsets

In JT cliquesbecomesvertices

GJT

Ex: ceg egh = eg

11

Message Passing from clique A to clique B

1. Project the potential of A into SAB

2. Absorb the potential of SAB into B

Projection Absorption

Propagating potentialsPropagating potentials

12

1. COLLECT-EVIDENCE messages 1-52. DISTRIBUTE-EVIDENCE messages 6-10

Global PropagationGlobal Propagation

32

5

1 48 10

9

7

6

Root

abd

ade

ace

ceg

eghdef

ad ae ce

de eg

13

Using the JT for the MSBN

• How?

• Issue: Clique Granularity

• Issue: Clique Association

• Issue: MSBN must have tree structure

14

Take a JT

15

Partition JT into sub-trees

16

Issue: Clique Granularity

17

Issue: Clique Granularity

18

Issue: Clique association

19

Issue: Clique association

20

Issue: MSBN has a tree structure

21

Issue: MSBN has a tree structure

Can’t communicate here!

22

Issue: MSBN has a tree structure

Can’t communicate here!

Though these two subnets may share many variables.

23

Directions for Solutions

• Controlling Granularity

• Increasing Granularity

• Choosing the JT

• Alternative communication graphs

24

Controlling Granularity

• Moralization and Triangulation → Cliques

• Moralization: No choice.

• Triangulation: Some choice.

• Triangulate keeping the subnets in mind.

25

Increasing Granularity

• Can we break down cliques?

Conjecture:Given two sets of random variables X and Y, let

dist(P(X-Y|Y),P(X-Y|X Y))(X,Y) =

vol(X Y)

Given a clique Z whose set of random variables Z is covered by sets X and Y. If ∆(X,Y) is small enough then we may replace Z by X and Y.

Z X Y

26

Choosing The JT

• Once the cliques have been formed (hence the JG), any maximal weight spanning tree of the JG will do for the JT.

• Again, we should choose this tree so as to reduce the overhead when connecting subnets internally.

27

Alternative communication Graphs

We raise the question as to wether it is possible to perform intra- and extra-subnet calibration using some non-tree sub-graph of the JG…

28

MatLab: The new Stuff:

• Netica can talk to the Matlab tool.

• View/Interact tool is nicer

• JG for communication graph… (or not)

• Query cliques

29

Propagating with cycles

ABC BCE

BDE DEF

BC

EBE

B

DE

30

Mysterious convergence

ABC CE

BDE DEF

C

EE

B

DE