56
Session 8 Bayesian Networks II

Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Session 8Bayesian Networks II

Page 2: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Theory Summary

Page 3: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent
Page 4: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Conditional IndependenceA node is conditionally independent of its

non-descendants given its parentsA node is conditionally independent of all other nodes in the network given the Markov blanket, i.e., its parents, children and children's parents.

Page 5: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

(Un)conditional independence in belief networks

Page 6: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-connected Definitiond-connected: if there is a non-blocked path from X to Y in the Bayesian graph - otherwise d-separated.Thus: d-separation -> X ⟂ Y | C, but not d-connection -> conditional dependence.

Page 7: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Conditional Independence Proof

Page 8: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Conditional Independence Proofa) Show that p(C|A,B) = p(C|B) is equivalent to P(A,C|B) = P(A|B)P(C|B).

b) Draw three Bayesian networks with nodes A, B and C where this derived independence holds, but not A ⟂ C.

Page 9: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Conditional Independence Proof: Solutiona) Show that p(C|A,B) = p(C|B) is equivalent to P(A,C|B) = P(A|B)P(C|B).

Solution:

Page 10: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Conditional Independence Proof: Solutionb) Draw three Bayesian networks with nodes A, B and C where this derived independence (A ⟂ C | B) holds, but not A ⟂ C.

A

B

C

C

B

A

A

B

C

Page 11: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random Variables

Page 12: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)S

C

A

Page 13: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | ∅ ?S

C

A

Page 14: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | ∅ ?

Yes, because of the local Markov property(A variable is independent of all its non-descendants when conditioned on its parents)

S

C

A

Page 15: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | C ?S

C

A

Page 16: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)

Is A ⫫ S | C ?

No! The only path between A and S has C as a collider.It doesn’t block the path when conditioned on C!

S

C

A

Page 17: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)

Adjust the model to capture that people inconstruction have a higher likelihood of beingsmokers as well as being exposed to asbestos,and visualize

S

C

A

Page 18: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:

P(A, S, C) = P(C | A, S) P(A) P(S)

Adjust the model to capture that people inconstruction have a higher likelihood of beingsmokers as well as being exposed to asbestos,and visualize

Consider: working in construction (X)

P(A, S, C, X) = P(C | A, S) P(A | X) P(S | X) P(X)

S

C

A

X

Page 19: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence in Bayesian Networks

Page 20: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence in Bayesian Networks

Source: Freiburg exercises 6

Page 21: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence in Bayesian Networksa) Determine which of the following conditional independence statements follow:

i) Cold ⟂ Winterii) Winter ⟂ NegligentDriveriii) Winter ⟂ RadioSilent | BatteryProblemiv) Winter ⟂ EngineNotStarting | BatteryProblemv) Cold ⟂ NegligentDriver | RadioSilentvi) NegligentDriver ⟂ Winter | TankEmptyvii) NegligentDriver ⟂ Winter | TankEmpty, EngineNotStarting

b) Compute P(E=true | N=true, C=false) given the conditional probabilitiesc) List all nodes in the Markov blanket for node LightsOnOverNight.

Page 22: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence in Bayesian NetworksWe know that a node is

● conditionally independent of its non-descendants given its parents● independent from all other nodes given his Markov blanket

(=parents, children & other parents of his children)

Thus:

1. Cold ¬⟂ Winter (Intuitive: they are connected)2. Winter ⟂ NegligentDriver (Parents(Winter) = ∅ and ND is not a descendant of Winter)3. Winter ⟂ RadioSilent | BatteryProblem (MarkovBlanket(RadioSilent) = BatteryProblem)4. Winter ¬⟂ EngineNotStarting | BatteryProblem (E.g. if B=true, then information about W affects E, because if W=false,

it increases odds of C=false, which means more likely L=true, which means it’s more likely that N=true, thus more chance of T=true, thus adding information about chance of E=true)

5. Cold ¬⟂ NegligentDriver | RadioSilent (Similar reason as above. E.g. if R=true, then probably B=true. So if then C=false, then L is likely true, meaning that N is likely true.)

6. NegligentDriver ⟂ Winter | LightsOnOverNight (Parents(W) = ∅, N is not a descendant of W, L is not collider for both)7. NegligentDriver ¬⟂ Winter | LightsOnOverNight, EngineNotStarting (E is collider for N and W: If you know the effect

(E), and you know one of the causes is true (e.g. W), then this gives information about the other cause (N) )

Page 23: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence in Bayesian Networksb) Compute P(E=true | N=true, C=false) given the conditional probabilities

Solution: The easiest way is to calculate the probabilities from top to bottom & condition on parents of nodes, conditioned on original condition set

P(L=t | N=t, C=f) = 0.3P(T=t | N=t, C=f) = 0.1P(B=t | N=t, C=f) = P(L=t | N=t, C=f) * P(B=t | L=t, C=f)

+ P(L=f | N=t, C=f) * P (B=t | L=f, C=f) = 0.3 * 0.8 + 0.7 * 0.01 = 0.247

Source: Freiburg exercises solution 6

Page 24: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence in Bayesian Networks

Source: Freiburg exercises solution 6

P(E=t | N=t, C=f) = P(B=t | N=t, C=f) * P(T=t | N=t, C=f) * P(E=t | B=t, T=t) + P(B=t | N=t, C=f) * P(T=f | N=t, C=f) * P(E=t | B=t, T=f) + P(B=f | N=t, C=f) * P(T=t | N=t, C=f) * P(E=t | B=f, T=t) + P(B=f | N=t, C=f) * P(T=f | N=t, C=f) * P(E=t | B=f, T=f)

= 0.247 * 0.1 * 0.9 + 0.247 * 0.9 * 0.7 + 0.753 * 0.1 * 0.8 + 0.753 * 0.9 * 0.05= 0.02223 + 0.15561 + 0.06024 + 0.033885= 0.271965 ≈ 27%

Page 25: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Independence in Bayesian Networksc) List all nodes in the Markov blanket for node LightsOnOverNight.

Parents

Children

Other parents of childrenNot node itself!

Page 26: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation

Page 27: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-SeparationWhat?

A concept that helps decide whether a set of variables X is independent of Y given C.

How?

Page 28: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E(b) List all the variables that are d-separated

from F given E and K(c) Specify the Markov Blanket of H, D, and E

Page 29: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

Page 30: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

When conditioned on E, F is d-connected with

{}

When conditioned on E, F is d-separated from

{}

conditioned on

Page 31: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

When conditioned on E, F is d-connected with

{}

When conditioned on E, F is d-separated from

{A,C,D,G,I}

conditioned on

Unconditioned collider K

Conditioned non-collider E

Page 32: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

When conditioned on E, F is d-connected with

{B}

When conditioned on E, F is d-separated from

{A,C,D,G,I}

conditioned on

Page 33: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

When conditioned on E, F is d-connected with

{B,H}

When conditioned on E, F is d-separated from

{A,C,D,G,I}

conditioned on

Page 34: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

When conditioned on E, F is d-connected with

{B,H}

When conditioned on E, F is d-separated from

{A,C,D,G,I,J}

conditioned on

Unconditioned collider H

Page 35: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

When conditioned on E, F is d-connected with

{B,H,K}

When conditioned on E, F is d-separated from

{A,C,D,G,I,J}

conditioned on

Page 36: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(a) List all the variables that are d-separated

from F given E

When conditioned on E, F is d-connected with

{B,H,K,L}

When conditioned on E, F is d-separated from

{A,C,D,G,I,J}

conditioned on

Page 37: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

Page 38: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{}

When conditioned on E, F is d-separated from

{}

conditioned on

Page 39: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A}

When conditioned on E, F is d-separated from

{}

conditioned on

Conditioned on collider shared child K

Page 40: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B}

When conditioned on E, F is d-separated from

{}

conditioned on

Page 41: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B,C}

When conditioned on E, F is d-separated from

{}

conditioned on

Conditioned on collider shared child K

Page 42: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B,C,D}

When conditioned on E, F is d-separated from

{}

conditioned on

Conditioned on collider shared child K

Page 43: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B,C,D,G}

When conditioned on E, F is d-separated from

{}

conditioned on

Conditioned on collider shared child K

Page 44: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B,C,D,G,H}

When conditioned on E, F is d-separated from

{}

conditioned on

Page 45: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B,C,D,G,H,I}

When conditioned on E, F is d-separated from

{}

conditioned on

Conditioned on collider shared child K

Page 46: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B,C,D,G,H,I,J}

When conditioned on E, F is d-separated from

{}

conditioned on

Conditioned on collider descendant K

Page 47: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(b) List all the variables that are d-separated

from F given E and K

When conditioned on E, F is d-connected with

{A,B,C,D,G,H,I,J}

When conditioned on E, F is d-separated from

{L}

conditioned on

Conditioned non-collider K

Page 48: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(c) Specify the Markov Blanket of H, D, and E

Page 49: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

D-Separation(c) Specify the Markov Blanket of H, D, and E

H = {F, J, K, I}

D = {C, E, G, I}

E = {B, D, C}

Parents

ChildrenChildren’s parents

Page 50: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Hidden Markov Model

Page 51: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Excercise 5: HMMOn some days, the TA responds, while on others he doesn't (Y).

The TA can be at work or on holiday (X), but the student cannot perceive this directly.

On some days at work, the TA is too busy to reply, so the chance of the TA replying on a work day is 0.70.

TAs do not have many days off, so the chance that the TA is at work given that he was at work yesterday is 0.99.

However, when they are on holiday, they usually take several consecutive days, so the chance of a TA being on holiday when he was on holiday the day before is 0.90.

Page 52: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Excercise 5: HMMa) What is the probability the TA will reply on day 102 given that he is on holiday

on day 100b) What is the probability the TA will reply on day 102 given that he is on holiday

on day 100 and replies on day 101

Page 53: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

Bayesian network

Page 54: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

5.a) Solution

Page 55: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

5.a) Solution: numerical

Page 56: Session 8 Bayesian Networks II - DTAI Section · Independence in Bayesian Networks We know that a node is conditionally independent of its non-descendants given its parents independent

5.b) Solution