Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Session 8Bayesian Networks II
Theory Summary
Conditional IndependenceA node is conditionally independent of its
non-descendants given its parentsA node is conditionally independent of all other nodes in the network given the Markov blanket, i.e., its parents, children and children's parents.
(Un)conditional independence in belief networks
D-connected Definitiond-connected: if there is a non-blocked path from X to Y in the Bayesian graph - otherwise d-separated.Thus: d-separation -> X ⟂ Y | C, but not d-connection -> conditional dependence.
Conditional Independence Proof
Conditional Independence Proofa) Show that p(C|A,B) = p(C|B) is equivalent to P(A,C|B) = P(A|B)P(C|B).
b) Draw three Bayesian networks with nodes A, B and C where this derived independence holds, but not A ⟂ C.
Conditional Independence Proof: Solutiona) Show that p(C|A,B) = p(C|B) is equivalent to P(A,C|B) = P(A|B)P(C|B).
Solution:
Conditional Independence Proof: Solutionb) Draw three Bayesian networks with nodes A, B and C where this derived independence (A ⟂ C | B) holds, but not A ⟂ C.
A
B
C
C
B
A
A
B
C
Independence of Random Variables
Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:
P(A, S, C) = P(C | A, S) P(A) P(S)S
C
A
Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:
P(A, S, C) = P(C | A, S) P(A) P(S)
Is A ⫫ S | ∅ ?S
C
A
Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:
P(A, S, C) = P(C | A, S) P(A) P(S)
Is A ⫫ S | ∅ ?
Yes, because of the local Markov property(A variable is independent of all its non-descendants when conditioned on its parents)
S
C
A
Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:
P(A, S, C) = P(C | A, S) P(A) P(S)
Is A ⫫ S | C ?S
C
A
Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:
P(A, S, C) = P(C | A, S) P(A) P(S)
Is A ⫫ S | C ?
No! The only path between A and S has C as a collider.It doesn’t block the path when conditioned on C!
S
C
A
Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:
P(A, S, C) = P(C | A, S) P(A) P(S)
Adjust the model to capture that people inconstruction have a higher likelihood of beingsmokers as well as being exposed to asbestos,and visualize
S
C
A
Independence of Random VariablesThere is a synergistic relationship between asbestos exposure (A), smoking (S), and cancer (C). A model describing this relationship is given by:
P(A, S, C) = P(C | A, S) P(A) P(S)
Adjust the model to capture that people inconstruction have a higher likelihood of beingsmokers as well as being exposed to asbestos,and visualize
Consider: working in construction (X)
P(A, S, C, X) = P(C | A, S) P(A | X) P(S | X) P(X)
S
C
A
X
Independence in Bayesian Networks
Independence in Bayesian Networks
Source: Freiburg exercises 6
Independence in Bayesian Networksa) Determine which of the following conditional independence statements follow:
i) Cold ⟂ Winterii) Winter ⟂ NegligentDriveriii) Winter ⟂ RadioSilent | BatteryProblemiv) Winter ⟂ EngineNotStarting | BatteryProblemv) Cold ⟂ NegligentDriver | RadioSilentvi) NegligentDriver ⟂ Winter | TankEmptyvii) NegligentDriver ⟂ Winter | TankEmpty, EngineNotStarting
b) Compute P(E=true | N=true, C=false) given the conditional probabilitiesc) List all nodes in the Markov blanket for node LightsOnOverNight.
Independence in Bayesian NetworksWe know that a node is
● conditionally independent of its non-descendants given its parents● independent from all other nodes given his Markov blanket
(=parents, children & other parents of his children)
Thus:
1. Cold ¬⟂ Winter (Intuitive: they are connected)2. Winter ⟂ NegligentDriver (Parents(Winter) = ∅ and ND is not a descendant of Winter)3. Winter ⟂ RadioSilent | BatteryProblem (MarkovBlanket(RadioSilent) = BatteryProblem)4. Winter ¬⟂ EngineNotStarting | BatteryProblem (E.g. if B=true, then information about W affects E, because if W=false,
it increases odds of C=false, which means more likely L=true, which means it’s more likely that N=true, thus more chance of T=true, thus adding information about chance of E=true)
5. Cold ¬⟂ NegligentDriver | RadioSilent (Similar reason as above. E.g. if R=true, then probably B=true. So if then C=false, then L is likely true, meaning that N is likely true.)
6. NegligentDriver ⟂ Winter | LightsOnOverNight (Parents(W) = ∅, N is not a descendant of W, L is not collider for both)7. NegligentDriver ¬⟂ Winter | LightsOnOverNight, EngineNotStarting (E is collider for N and W: If you know the effect
(E), and you know one of the causes is true (e.g. W), then this gives information about the other cause (N) )
Independence in Bayesian Networksb) Compute P(E=true | N=true, C=false) given the conditional probabilities
Solution: The easiest way is to calculate the probabilities from top to bottom & condition on parents of nodes, conditioned on original condition set
P(L=t | N=t, C=f) = 0.3P(T=t | N=t, C=f) = 0.1P(B=t | N=t, C=f) = P(L=t | N=t, C=f) * P(B=t | L=t, C=f)
+ P(L=f | N=t, C=f) * P (B=t | L=f, C=f) = 0.3 * 0.8 + 0.7 * 0.01 = 0.247
Source: Freiburg exercises solution 6
Independence in Bayesian Networks
Source: Freiburg exercises solution 6
P(E=t | N=t, C=f) = P(B=t | N=t, C=f) * P(T=t | N=t, C=f) * P(E=t | B=t, T=t) + P(B=t | N=t, C=f) * P(T=f | N=t, C=f) * P(E=t | B=t, T=f) + P(B=f | N=t, C=f) * P(T=t | N=t, C=f) * P(E=t | B=f, T=t) + P(B=f | N=t, C=f) * P(T=f | N=t, C=f) * P(E=t | B=f, T=f)
= 0.247 * 0.1 * 0.9 + 0.247 * 0.9 * 0.7 + 0.753 * 0.1 * 0.8 + 0.753 * 0.9 * 0.05= 0.02223 + 0.15561 + 0.06024 + 0.033885= 0.271965 ≈ 27%
Independence in Bayesian Networksc) List all nodes in the Markov blanket for node LightsOnOverNight.
Parents
Children
Other parents of childrenNot node itself!
D-Separation
D-SeparationWhat?
A concept that helps decide whether a set of variables X is independent of Y given C.
How?
D-Separation(a) List all the variables that are d-separated
from F given E(b) List all the variables that are d-separated
from F given E and K(c) Specify the Markov Blanket of H, D, and E
D-Separation(a) List all the variables that are d-separated
from F given E
D-Separation(a) List all the variables that are d-separated
from F given E
When conditioned on E, F is d-connected with
{}
When conditioned on E, F is d-separated from
{}
conditioned on
D-Separation(a) List all the variables that are d-separated
from F given E
When conditioned on E, F is d-connected with
{}
When conditioned on E, F is d-separated from
{A,C,D,G,I}
conditioned on
Unconditioned collider K
Conditioned non-collider E
D-Separation(a) List all the variables that are d-separated
from F given E
When conditioned on E, F is d-connected with
{B}
When conditioned on E, F is d-separated from
{A,C,D,G,I}
conditioned on
D-Separation(a) List all the variables that are d-separated
from F given E
When conditioned on E, F is d-connected with
{B,H}
When conditioned on E, F is d-separated from
{A,C,D,G,I}
conditioned on
D-Separation(a) List all the variables that are d-separated
from F given E
When conditioned on E, F is d-connected with
{B,H}
When conditioned on E, F is d-separated from
{A,C,D,G,I,J}
conditioned on
Unconditioned collider H
D-Separation(a) List all the variables that are d-separated
from F given E
When conditioned on E, F is d-connected with
{B,H,K}
When conditioned on E, F is d-separated from
{A,C,D,G,I,J}
conditioned on
D-Separation(a) List all the variables that are d-separated
from F given E
When conditioned on E, F is d-connected with
{B,H,K,L}
When conditioned on E, F is d-separated from
{A,C,D,G,I,J}
conditioned on
D-Separation(b) List all the variables that are d-separated
from F given E and K
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{}
When conditioned on E, F is d-separated from
{}
conditioned on
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A}
When conditioned on E, F is d-separated from
{}
conditioned on
Conditioned on collider shared child K
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B}
When conditioned on E, F is d-separated from
{}
conditioned on
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B,C}
When conditioned on E, F is d-separated from
{}
conditioned on
Conditioned on collider shared child K
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B,C,D}
When conditioned on E, F is d-separated from
{}
conditioned on
Conditioned on collider shared child K
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B,C,D,G}
When conditioned on E, F is d-separated from
{}
conditioned on
Conditioned on collider shared child K
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B,C,D,G,H}
When conditioned on E, F is d-separated from
{}
conditioned on
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B,C,D,G,H,I}
When conditioned on E, F is d-separated from
{}
conditioned on
Conditioned on collider shared child K
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B,C,D,G,H,I,J}
When conditioned on E, F is d-separated from
{}
conditioned on
Conditioned on collider descendant K
D-Separation(b) List all the variables that are d-separated
from F given E and K
When conditioned on E, F is d-connected with
{A,B,C,D,G,H,I,J}
When conditioned on E, F is d-separated from
{L}
conditioned on
Conditioned non-collider K
D-Separation(c) Specify the Markov Blanket of H, D, and E
D-Separation(c) Specify the Markov Blanket of H, D, and E
H = {F, J, K, I}
D = {C, E, G, I}
E = {B, D, C}
Parents
ChildrenChildren’s parents
Hidden Markov Model
Excercise 5: HMMOn some days, the TA responds, while on others he doesn't (Y).
The TA can be at work or on holiday (X), but the student cannot perceive this directly.
On some days at work, the TA is too busy to reply, so the chance of the TA replying on a work day is 0.70.
TAs do not have many days off, so the chance that the TA is at work given that he was at work yesterday is 0.99.
However, when they are on holiday, they usually take several consecutive days, so the chance of a TA being on holiday when he was on holiday the day before is 0.90.
Excercise 5: HMMa) What is the probability the TA will reply on day 102 given that he is on holiday
on day 100b) What is the probability the TA will reply on day 102 given that he is on holiday
on day 100 and replies on day 101
Bayesian network
5.a) Solution
5.a) Solution: numerical
5.b) Solution