Upload
theresa-janis-morgan
View
214
Download
2
Embed Size (px)
Citation preview
Selected Topics in Graphical Models
Petr Šimeček
Independence Unconditional Independence:
Discrete r.v. Continuous r.v.
Conditional Independence: Discrete r.v.
Continuous r.v.
YX yxyxyx ,)p()p(),p( ..P)f()f(),f( sayxyx
ZYX |
zyx
zyzxzzyx
,,
),p(),p()p(),,p(
..P
),f(),f()f(),,f(
sa
zyzxzzyx
List of Independence RelationshipsN random variables X1, X2, …, XN and their
distribution P
List of all conditional and unconditional independence relations between them
)},(|),(),(
,},...,1{,,);|,{(
CiXBiXAiX
disjunctNCBACBA
iii
Representation by Graph
X6
X5 X4
X3
X2
X1
X1
X3
X2
X4
Example – Sprinkler Network
Rain
WetGrass
Sprinkler
Cloudy
Example – Sprinkler Network
Rain
WetGrass
Sprinkler
CloudyCLOUDY
T F
0.5 0.5
Example – Sprinkler Network
Rain
WetGrass
Sprinkler
Cloudy
SPRINK T F
C=T 0.1 0.9
C=F 0.5 0.5
CLOUDY
T F
0.5 0.5
RAIN T F
C=T 0.8 0.2
C=F 0.2 0.8
Example – Sprinkler Network
Rain
WetGrass
Sprinkler
Cloudy
SPRINK T F
C=T 0.1 0.9
C=F 0.5 0.5
CLOUDY
T F
0.5 0.5
WET GRASS T F
R=T S=T 0.99 0.01
R=T S=F 0.9 0.1
R=F S=T 0.9 0.1
R=F S=F 0 1
RAIN T F
C=T 0.8 0.2
C=F 0.2 0.8
Example – Sprinkler Network
R
W
S
C
),|P().|P().|P().P(),,,P( SRWCSCRCWSRC
The number of parameters needn’t grow
exponentially with the number of
variables!
It depends on the number of parents
of nodes.
Purpose 1– Propagation of Evidence
Rain
WetGrass
Sprinkler
Cloudy
What is the probability that it is raining if we know that grass is wet?
Propagation of EvidenceIn general: I have observed some
variable(s). What is the probability of other variable(s)? What is the most probable value(s)?
Why don’t transfer BN to contingency table? Marginalization does not work for N large: needs 2N memory, much time, has low precision…
Propagation of EvidenceIn general: I have observed some
variable(s). What is the probability of other variable(s)? What is the most probable value(s)?
Why don’t transfer BN to contingency table? Marginalization does not work for N large: needs 2N memory, much time, has low precision…
Purpose 2 – Parameter Learning
Rain
WetGrass
Sprinkler
Cloudy
SPRINK T F
C=T ? ?
C=F ? ?
CLOUDY
T F
? ?
WET GRASS T F
R=T S=T ? ?
R=T S=F ? ?
R=F S=T ? ?
R=F S=F ? ?
RAIN T F
C=T ? ?
C=F ? ?
Parameter LearningWe know: graph (CI structure) sample (observations) of BN
We don’t know: conditional probabilistic distributions
(could be estimated by MLE, Bayesian stat.)
Purpose 3 – Structure Learning
CLOUDY SPRINKLER RAIN WET GRASS
TRUE FALSE FALSE FALSE
FALSE TRUE FALSE FALSE
TRUE FALSE TRUE FALSE
FALSE FALSE FALSE FALSE
FALSE TRUE FALSE FALSE
FALSE FALSE TRUE TRUE
TRUE FALSE TRUE TRUE
TRUE FALSE TRUE FALSE
… … … …
Structure LearningWe know: independent observations (data) of BN sometimes, the casual ordering of vars
We don’t know: graph (CI structure) conditional probabilistic distributions
Solution: CI tests maximization of some criterion – huge s. space
(AIC, BIC, Bayesian approach)
Example – Entry Examination
Markov EquivalenceSome of arcs can be changed without
changing CI relationships.
The best one can hope to do is to identify the model up to Markov equivalence.
RainWet
GrassRain
WetGrass
Structure Learning Theory
algorithms proved to be asymptotically right Janžura, Nielsen (2003)
1 000 000 observations for 10 binary variables
Practice in medicine – usually 50-1500 obs. BNs are often used in spite of that
Structure Learning - Simulation 3 variables, take m from 100 to 1000 for each m do 100 times
generate of Bayesian network generate m samples use K2 structure learning algorithm
count the probability of successful selection for each m
This should give an answer to the question:“Is it a chance to find the true model?”
200 400 600 800 1000
45
50
55
60
65
70
75
Number of Observations
Pro
ba
bili
ty o
f Tru
e M
od
el S
ele
ctio
n (%
)
To Do List:
software: free, open source, easy to use, fast, separated API
more simulation: theory x practice
popularization of structural learning
Czech literature: maybe my PhD. thesis
References: Castillo E. et al. (1997): Expert Systems
and Probabilistic Network Models, Springer Verlag.
Neapolitan R. E. (2003): Learning Bayesian Networks, Prentice Hall.
Janžura N., Nielsen J. (2003): A numerical method for learning Bayesian Networks from Statistical Data, WUPES.