View
250
Download
3
Tags:
Embed Size (px)
Citation preview
Probabilistic thinking – part 1
Nur Aini Masruroh
Events
Event is a distinction about some states of the world
Example: Whether the next person entering the room is a beer
drinker The date of the next general election Whether it will be raining tonight Our next head of department Etc
Clarity test
When we identify an event, we have in mind what we meant. But will other people know precisely what you mean? Even you may not have precise definition of what you have in mind
To avoid ambiguity, every event should pass the clarity test Clarity test: to ensure that we are absolutely clear and precise about the
definition of every event we are dealing with in a decision problem The clarity test is conducted by submitting our definition of each
event to a clairvoyant A clairvoyant is a hypothetical being who is:
Competent and trustworthy Knows the outcome of any past and future event Knows the value of any physically defined quantity both in the past and future Has infinite computational (mental) power and is able to perform any reasoning
and computation instantly and without any effort
Clarity test (cont’d)
Passing the clarity test: If and only if the clairvoyant can tell its outcome
without any further judgment
Example: The next person entering this room is a beer drinker
• What is a beer drinker? What is a beer?
The next person entering this room is a graduate• What is a graduate?
Possibility tree
Single event tree Example: event “the next person entering this room is
a businessman” Suppose B represents a businessman and B’
otherwise,
Possibility tree
Two-event trees Simultaneously consider several events Example: event “the next person entering this room is
a businessman” and event “the next person entering this room is a graduate” can be jointly considered
Reversing the order of events in a tree
In the previous example, we have considered the distinctions in the order of “businessman” then “graduate”, i.e., B to G.
The same information can be expressed with the events in the reverse order, i.e., G to B.
Multiple event trees
We can jointly consider three events businessman, graduate, and gender.
Using probability to represent uncertainty
Probability: Frequentist view
Probabilities are fundamentally dispositional properties of non-deterministic physical systems
Probabilities are viewed as long-run frequencies of events This is the standard interpretation used in classical statistics
Subjective (Bayesian) view Probabilities are representations of our subjective degree of
belief Probabilities in general are not necessarily ties to any physical
or process which can be repeated indefinitely
Assigning probabilities to events
To assign probabilities, it depends on our state of information about the event
Example: information relevant to assessment of the likelihood that the next person entering the room is a businessman might include the followings: There is an alumni meeting outside the room and most of them are
businessman You have made arrangement to meet a friend here and she to your
knowledge is not a businessman. She is going to show up any moment. Etc
After considering all relevant background information, we assign the likelihood that the next person entering the room is a businessman by assigning a probability value to each of the possibilities or outcomes
Marginal and conditional probabilities
In general, given information about the outcome of some events, we may revise our probabilities of other events
We do this through the use of conditional probabilities The probability of an event X given specific outcomes of another
event Y is called the conditional probability X given Y The conditional probability of event X given event Y and other
background information ξ, is denoted by p(X|Y, ξ) and is given by
0)|(for)|(
)|(),|(
Yp
Yp
YXpYXp
Factorization rule for joint probability
Changing the order of conditioning
Suppose in the previous tree we have
There is no reason why we should always conditioned G on B. suppose we want to draw the tree in the order G to B
Need to flip the tree!
Flipping the tree
Graphical approach Change the ordering of the underlying possibility tree Transfer the elemental (joint) probabilities from the original tree
to the new tree Compute the marginal probability for the first variable in the new
tree, i.e., G. We add the elemental probabilities that are related to G1 and G2 respectively.
Compute conditional probabilities for B given G
Bayes’ theorem Doing the above tree flipping is already applying Bayes’theorem
Bayes’ Theorem
Given two uncertain events X and Y. Suppose the probabilities p(X|ξ) and p(Y|X, ξ) are known, then
X
XYpXpYp
where
Yp
XYpXpYXp
)||()|()|(
)|(
),|()|(),|(
Probabilistic dependency or relevance
Let A be an event with n possible outcomes ai, i=1,…,n
B be an event with m possible outcomes bj ,j=1,…,m
Event A is said to be probabilistically dependent on event B if
p(A|bj, ξ) ≠ p(A|bk, ξ) for some j ≠ k The conditional probability of A given B is different for different outcomes
or realizations of event B. we also say that B is relevant to A Event A is said to be probabilistically independent on event B if
p(A|bj, ξ) = p(A|bk, ξ) for all j = k The conditional probability of A given B is the same for all outcomes or
realizations of event B. we also say that B is irrelevant to A In fact, if A is independent of B, then p(A|B, ξ) = p(A|ξ) Intuitively, independence means knowing the outcome of one event
does not provide any information on the probability of outcomes of the other event
Joint probability distribution of independent events
In general, the joint probability distribution for any two uncertain events A and B is
p(A, B|ξ)=p(A|B, ξ)p(B| ξ) If A and B are independent, then since p(A|B,ξ)=p(A| ξ),
we have
p(A, B|ξ)=p(A|ξ) p(B|ξ) The joint probability of A and B is simply the product of
their marginal probabilities In general, the joint probability for n mutually
independent events is
p(X1, X2, …, Xn|ξ)=p(X1|ξ) p(X2|ξ)… p(Xn-1|ξ) p(Xn|ξ)
Conditional independence or relevance
Suppose given 2 events, A and B, and they are found to be not independent
Introduce event C with 2 outcomes, c1 and c2 If C=c1 is true, and we have
p(A|B, c1, ξ)=p(A|c1, ξ)
If C=c2 is true, we have
p(A|B, c2, ξ)=p(A|c2, ξ)
Then we say that event A is conditionally independent of event B given event C
Definition (Conditional Independence):
given 3 distinct events A, B, and C, if p(A|B, ck, ξ)=p(A|ck, ξ) for all k, that is the conditional probability table (CPT) for A given B and C repeats for all possible realizations of C, then we say that A and B are conditional independent given C, and denote by C|BA
Conditional independence (cont’d)
If then p(A|B, C, ξ)=p(A|C, ξ) Example:
Given the following conditional probabilities:
p(a1|b1, c1)= 0.9 p(a2|b1, c1)= 0.1
p(a1|b2, c1)= 0.9 p(a2|b2, c1)= 0.1
p(a1|b1, c2)= 0.8 p(a2|b1, c2)= 0.2
p(a1|b2, c2)= 0.8 p(a2|b2, c2)= 0.2
we conclude that
Note that A is not (marginally) independent of B unless we can show that p(a1|b1) = p(a1|b2) with more information
CBA |
CBA |
Join probability distribution of conditional probability distribution
Recall, by factorization rule, the joint probability for A, B, and C is
p(A, B, C|ξ)= p(A| B, C, ξ)p(B|C, ξ)p(C| ξ)
If A is independent of B given C, then since
p(A| B, C, ξ) = p(A|C, ξ) we have
p(A, B, C|ξ)= p(A| C, ξ)p(B|C, ξ)p(C| ξ)
To be continued…
See you next week!