Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Session 8Bayesian Networks II

Theory Summary

Conditional IndependenceA node is conditionally independent of its

non-descendants given its parentsA node is conditionally independent of all other nodes in the network given the Markov blanket, i.e., its parents, children and children's parents.

(Un)conditional independence in belief networks

D-connected Definitiond-connected: if there is a non-blocked path from X to Y in the Bayesian graph - otherwise d-separated.Thus: d-separation -> X ⟂ Y | C, but not d-connection -> conditional dependence.

Conditional Independence Proof

Conditional Independence Proofa) Show that p(C|A,B) = p(C|B) is equivalent to P(A,C|B) = P(A|B)P(C|B).

b) Draw three Bayesian networks with nodes A, B and C where this derived independence holds, but not A ⟂ C.

Conditional Independence Proof: Solutiona) Show that p(C|A,B) = p(C|B) is equivalent to P(A,C|B) = P(A|B)P(C|B).

Solution:

Conditional Independence Proof: Solutionb) Draw three Bayesian networks with nodes A, B and C where this derived independence (A ⟂ C | B) holds, but not A ⟂ C.

A

B

C

C

B

A

A

B

C

Independence of Random Variables

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)S

C

A


P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | ∅ ?S

C

A


P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | ∅ ?

Yes, because of the local Markov property(A variable is independent of all its non-descendants when conditioned on its parents)

S

C

A


P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | C ?S

C

A


P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | C ?

No! The only path between A and S has C as a collider.It doesn’t block the path when conditioned on C!

S

C

A


P(A, S, C) = P(C | A, S) P(A) P(S)

Adjust the model to capture that people inconstruction have a higher likelihood of beingsmokers as well as being exposed to asbestos,and visualize

S

C

A


P(A, S, C) = P(C | A, S) P(A) P(S)

Adjust the model to capture that people inconstruction have a higher likelihood of beingsmokers as well as being exposed to asbestos,and visualize

Consider: working in construction (X)

P(A, S, C, X) = P(C | A, S) P(A | X) P(S | X) P(X)

S

C

A

X

Independence in Bayesian Networks


Source: Freiburg exercises 6

http://ais.informatik.uni-freiburg.de/teaching/ss12/ki/exercises/sheet06-english.pdf

Independence in Bayesian Networksa) Determine which of the following conditional independence statements follow:

i) Cold ⟂ Winterii) Winter ⟂ NegligentDriveriii) Winter ⟂ RadioSilent | BatteryProblemiv) Winter ⟂ EngineNotStarting | BatteryProblemv) Cold ⟂ NegligentDriver | RadioSilentvi) NegligentDriver ⟂ Winter | TankEmptyvii) NegligentDriver ⟂ Winter | TankEmpty, EngineNotStarting

b) Compute P(E=true | N=true, C=false) given the conditional probabilitiesc) List all nodes in the Markov blanket for node LightsOnOverNight.

Independence in Bayesian NetworksWe know that a node is

● conditionally independent of its non-descendants given its parents● independent from all other nodes given his Markov blanket

(=parents, children & other parents of his children)

Thus:

1. Cold ¬⟂ Winter (Intuitive: they are connected)2. Winter ⟂ NegligentDriver (Parents(Winter) = ∅ and ND is not a descendant of Winter)3. Winter ⟂ RadioSilent | BatteryProblem (MarkovBlanket(RadioSilent) = BatteryProblem)4. Winter ¬⟂ EngineNotStarting | BatteryProblem (E.g. if B=true, then information about W affects E, because if W=false,

it increases odds of C=false, which means more likely L=true, which means it’s more likely that N=true, thus more chance of T=true, thus adding information about chance of E=true)

5. Cold ¬⟂ NegligentDriver | RadioSilent (Similar reason as above. E.g. if R=true, then probably B=true. So if then C=false, then L is likely true, meaning that N is likely true.)

6. NegligentDriver ⟂ Winter | LightsOnOverNight (Parents(W) = ∅, N is not a descendant of W, L is not collider for both)7. NegligentDriver ¬⟂ Winter | LightsOnOverNight, EngineNotStarting (E is collider for N and W: If you know the effect

(E), and you know one of the causes is true (e.g. W), then this gives information about the other cause (N) )

Independence in Bayesian Networksb) Compute P(E=true | N=true, C=false) given the conditional probabilities

Solution: The easiest way is to calculate the probabilities from top to bottom & condition on parents of nodes, conditioned on original condition set

P(L=t | N=t, C=f) = 0.3P(T=t | N=t, C=f) = 0.1P(B=t | N=t, C=f) = P(L=t | N=t, C=f) * P(B=t | L=t, C=f)

+ P(L=f | N=t, C=f) * P (B=t | L=f, C=f) = 0.3 * 0.8 + 0.7 * 0.01 = 0.247

Source: Freiburg exercises solution 6

http://ais.informatik.uni-freiburg.de/teaching/ss18/ki/exercises/sheet06-solution.pdf


Source: Freiburg exercises solution 6

P(E=t | N=t, C=f) = P(B=t | N=t, C=f) * P(T=t | N=t, C=f) * P(E=t | B=t, T=t) + P(B=t | N=t, C=f) * P(T=f | N=t, C=f) * P(E=t | B=t, T=f) + P(B=f | N=t, C=f) * P(T=t | N=t, C=f) * P(E=t | B=f, T=t) + P(B=f | N=t, C=f) * P(T=f | N=t, C=f) * P(E=t | B=f, T=f)

= 0.247 * 0.1 * 0.9 + 0.247 * 0.9 * 0.7 + 0.753 * 0.1 * 0.8 + 0.753 * 0.9 * 0.05= 0.02223 + 0.15561 + 0.06024 + 0.033885= 0.271965 ≈ 27%

http://ais.informatik.uni-freiburg.de/teaching/ss18/ki/exercises/sheet06-solution.pdf

Independence in Bayesian Networksc) List all nodes in the Markov blanket for node LightsOnOverNight.

Parents

Children

Other parents of childrenNot node itself!

D-Separation

D-SeparationWhat?

A concept that helps decide whether a set of variables X is independent of Y given C.

How?

D-Separation(a) List all the variables that are d-separated

from F given E(b) List all the variables that are d-separated

from F given E and K(c) Specify the Markov Blanket of H, D, and E


from F given E


from F given E

When conditioned on E, F is d-connected with

{}

When conditioned on E, F is d-separated from

{}

conditioned on


from F given E


{}


{A,C,D,G,I}

conditioned on

Unconditioned collider K

Conditioned non-collider E


from F given E


{B}


{A,C,D,G,I}

conditioned on


from F given E


{B,H}


{A,C,D,G,I}

conditioned on


from F given E


{B,H}


{A,C,D,G,I,J}

conditioned on

Unconditioned collider H


from F given E


{B,H,K}


{A,C,D,G,I,J}

conditioned on


from F given E


{B,H,K,L}


{A,C,D,G,I,J}

conditioned on

D-Separation(b) List all the variables that are d-separated

from F given E and K




{}


{}

conditioned on




{A}


{}

conditioned on

Conditioned on collider shared child K




{A,B}


{}

conditioned on




{A,B,C}


{}

conditioned on





{A,B,C,D}


{}

conditioned on





{A,B,C,D,G}


{}

conditioned on





{A,B,C,D,G,H}


{}

conditioned on




{A,B,C,D,G,H,I}


{}

conditioned on





{A,B,C,D,G,H,I,J}


{}

conditioned on

Conditioned on collider descendant K




{A,B,C,D,G,H,I,J}


{L}

conditioned on

Conditioned non-collider K

D-Separation(c) Specify the Markov Blanket of H, D, and E

D-Separation(c) Specify the Markov Blanket of H, D, and E

H = {F, J, K, I}

D = {C, E, G, I}

E = {B, D, C}

Parents

ChildrenChildren’s parents

Hidden Markov Model

Excercise 5: HMMOn some days, the TA responds, while on others he doesn't (Y).

The TA can be at work or on holiday (X), but the student cannot perceive this directly.

On some days at work, the TA is too busy to reply, so the chance of the TA replying on a work day is 0.70.

TAs do not have many days off, so the chance that the TA is at work given that he was at work yesterday is 0.99.

However, when they are on holiday, they usually take several consecutive days, so the chance of a TA being on holiday when he was on holiday the day before is 0.90.

Excercise 5: HMMa) What is the probability the TA will reply on day 102 given that he is on holiday

on day 100b) What is the probability the TA will reply on day 102 given that he is on holiday

on day 100 and replies on day 101

Bayesian network

5.a) Solution

5.a) Solution: numerical

5.b) Solution

Documents

Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent