Upload
bethany-robbins
View
213
Download
0
Embed Size (px)
Citation preview
Bayesian Nets and Applications
2
Naïve Bayes What happens if we have more than one piece of
evidence? If we can assume conditional independence
Overslept and trafficjam are independent, given late A and B are conditionally independent given C just in case B
doesn't tell us anything about A if we already know C: P(late|overslept Λ trafficjam) =
αP(overslept Λ trafficjam)|late)P(late) = αP(overslept)|late)P(trafficjam|late)P(late)
Naïve Bayes where a single cause directly influences a number of effects, all conditionally independent
Independence often assumed even when not so
3
Bayesian Networks A directed acyclic graph in which each node is
annotated with quantitative probability information A set of random variables makes up the network nodes A set of directed links connects pairs of nodes. If there
is an arrow from node X to node Y, X is a parent of Y Each node Xi has a conditional probability
distributionP(Xi|Parents(Xi) that quantifies the effect of the parents on the node
4
Example Topology of network encodes conditional
independence assumptions
5
Smart
Good test taker
Understands material
Hard working
Exam Grade Homework Grade
6
Smart
Good test taker
Understands material
Hard working
Exam Grade Homework Grade
Smart
True False
.5 .5
Hard Working
True False
.7 .3
S Good Test Taker
True False
True .75 .25
False .25 .75
S HW UM
True False
True True .95 .05
True False .6 .4
False True .6 .4
False False .2 .8
7
Conditional Probability Tables
Smart
True False
.5 .5
Hard Working
True False
.7 .3
S Good Test Taker
True False
True .75 .25
False .25 .75
S HW UM
True False
True True .95 .05
True False .6 .4
False True .6 .4
False False .2 .8
GTT UM Exam Grade
A B C D F
True True .7 .25 .03 .01 .01
True False .3 .4 .2 .05 .05
False True .4 .3 .2 .08 .02
False False .05 .2 .3 .3 .15
Homework Grade
UM A B C D F
True .7 .25 .03 .01 .01
False .2 .3 .4 .05 .05
8
Compactness A CPT for Boolean Xi with k Boolean parents has
2k rows for the combinations of parent values Each row requires one number p for Xi=true (the
number for Xi=false is just 1-p) If each variable has no more than k parents, the
complete network requires O(nx2k) numbers Grows linearly with n vs O(2n) for the full joint
distribution Student net: 1+1+2+2+5+5=11 numbers (vs. 26-
1)=31
9
Conditional Probability
10
Global Semantics/Evaluation
Global semantics defines the full joint distribution as the product of the local conditional distributions:
P(x1,…,xn)=∏in
=1P(xi| Parents(Xi))e.g.,
P(EG=AΛGTΛ⌐UMΛSΛHW)
11
Global Semantics
Global semantics defines the full joint distribution as the product of the local conditional distributions:
P(X1,…,Xn)=∏in=1P(Xi|Parents(Xi))
e.g., Observations:S, HW, not UM, will I get an A? P(EG=AΛGTΛ⌐UMΛSΛHW)
= P(EG=A|GT Λ⌐UM)*P(GT|S)*P(⌐UM |HW ΛS)*P(S)*P(HW)
12
Conditional Independence and Network Structure The graphical structure of a Bayesian network
forces certain conditional independences to hold regardless of the CPTs.
This can be determined by the d-separation criteria
13
a
b
c
a
b
c
b
a c
Linear
Converging
Diverging
14
D-separation (opposite of d-connecting)
A path from q to r is d-connecting with respect to the evidence nodes E if every interior node n in the path has the property that either
It is linear or diverging and is not a member of E It is converging and either n or one of its decendents is
in E
If a path is not d-connecting (is d-separated), the nodes are conditionally independent given E
15
Smart
Good test taker
Understands material
Hard working
Exam Grade Homework Grade
16
S and EG are not independent given GTT S and HG are independent given UM
Medical Application of Bayesian Networks:Pathfinder
18
Pathfinder Domain: hematopathology diagnosis
Microscopic interpretation of lymph-node biopsies Given: 100s of histologic features appearing in
lymph node sections Goal: identify disease type
malignant or benign Difficult for physicians
19
Pathfinder System Bayesian Net implementation Reasons about 60 malignant and benign
diseases of the lymph node Considers evidence about status of up to 100
morphological features presenting in lymph node tissue
Contains 105,000 subjectively-derived probabilities
20
21
Commercialization Intellipath Integrates with videodisc libraries of
histopathology slides Pathologists working with the system make
significantly more correct diagnoses than those working without
Several hundred commercial systems in place worldwide
22
Sequential Diagnosis
23
Features Structured into a set of 2-10 mutually
exclusive values Pseudofollicularity
Absent, slight, moderate, prominent
Represent evidence provided by a feature as F1,F2, … Fn
24
Value of information User enters findings from microscopic analysis of tissue
Probabilistic reasoner assigns level of belief to different diagnoses
Value of information determines which tests to perform next
Full disease utility model making use of life and death decision making
Cost of tests Cost of misdiagnoses
25
26
27
Group Discrimination Strategy Select questions based on their ability to
discriminate between disease classes For given differential diagnosis, select most
specific level of hierarchy and selects questions to discriminate among groups
Less efficient Larger number of questions asked
28
29
30
Other Bayesian Net Applications Lumiere – Who knows what it is?
31
Other Bayesian Net Applications Lumiere
Single most widely distributed application of BN Microsoft Office Assistant Infer a user’s goals and needs using evidence about user
background, actions and queries VISTA
Help NASA engineers in round-the-clock monitoring of each of the Space Shuttle’s orbiters subsystem
Time critical, high impact Interpret telemetry and provide advice about likely failures Direct engineers to the best information In use for several years
Microsoft Pregnancy and Child Care What questions to ask next to diagnose illness of a child
32
Other Bayesian Net Applications Speech Recognition
Text Summarization
Language processing tasks in general