Thinking Backwards

  • View

  • Download

Embed Size (px)

Text of Thinking Backwards

  • 8/22/2019 Thinking Backwards


    Thinking Backward forKnowledge AcquisitionRoss D. Shachter and David E. Heckerman

    This article examines the direction inwhich knowledge bases are constructed

    for diagnosis and decision making Whenbuilding an expert system, it is traditionalto elicit knowledge from an expert in thedirection in which the knowledge is to be

    applied, namely, from observable evi-dence toward unobservable hypothesesHowever, experts usually find it simpler to

    reason in the opposite direction-fromhypotheses to unobservable

    evidence-because this direction reflectscausal relationships Therefore, we arguethat a knowledge base be constructed fol-

    lowing the experts natural reasoningdirection, and then reverse the directionfor use This choice of representationdirection facilitates knowledge acquisi-

    tion in deterministic domains and isessential when a problem involves uncer-tainty We illustrate this concept with

    influence diagrams, a methodology forgraphically representing a joint probabili-ty distribution Influence diagrams pro-

    vide a practical means by which an expertcan characterize the qualitative and quan-titative relationships among evidence andhypotheses in the appropriate direction

    Once constructed, the relationships caneasily be reversed into the less intuitivedirection in order to perform inference and

    diagnosis, In this way, knowledgeacquisition is made cognitively simple;

    the machine carries the burden of trans-lating the representat ion

    A few years ago, we were dis-cussing probabilistic reasoningwith a colleague who works in com-puter vision He wanted to calculatethe likelihood of a tiger being presentin a field of view given the digitizedimage. OK, we replied, If the tigerwere present, what is the probabilitythat you would see that image? Onthe other hand, if the tiger were notpresent, what is the probability youwould see it? Before we could saywhat is the probability there is atiger in the first place? our colleaguethrew up his arms in despair Whymust you probabilists insist on think-ing about everything backwards?

    Since then, we have pondered thisquestion. Why is it that we want tolook at problems of evidential reason-ing backward? After all, the task ofevidential reasoning is, by definition,the determination of the validity ofunobservable propositions fromobservable evidence; it seems best torepresent knowledge in the directionit will be used. Why then should werepresent knowledge in the oppositedirection, from hypothesis to evi-dence?In this article, we attempt toanswer this question by showing howsome backward thinking can simplifyreasoning with expert knowledge 1 Webelieve that the best representa tionfor knowledge acquisition is the sim-plest representation which capturesthe essence of an experts beliefs Weargue that in many cases, this repre-sentation will correspond to a direc-tion of reasoning that is opposite thedirection in which expe rt knowledgeis used in uncertain reasoning.This question has relevance to arti-ficial intelligence applicationsbecause several popular expert systemarchitectures represent knowledge in

    the direction from observables tounobservab les. For example, in theMYCIN certainty factor (CF) model(Shortliffe and Buchanan 1975),knowledge is represented as rules ofthe form


  • 8/22/2019 Thinking Backwards


    simpler to construct assessments inthis direction (Kuipers and Kassier1984). Furthermore, representations ofexpert knowledge are often less com-plex in the causal direction. Thus,these three observations suggest thatthere might be advantages to repre-senting knowledge in the directionopposite the d irection of usage. Thisargument is summarized in figure I

    CAUSE a.ssesmentc EFFECT(usually) (usually)Unobservable - usage Observable

    Figure 1.Two Directions of Representation.

    Most real-world problems involve causalinteractions in which one event (thecause) affects the outcome of another (theeffect) We often use an effect that we canobserve to help us learn about a causewhich we cannot d irectly observe. How-ever, we are more comfortable quantify-ing the relationship between the twoevents in the causal direction, assessin gthe likelihood of the effect if only weknew whether the cause were true

    Influence DiagramsIn this article, we examine theseissues in the context of probabilitytheory. Nonetheless, we believe thatthe distinction between direction ofusage and direction of natural assess-ment is a fundamental issue, indepen-dent of the language in which belief isrepresented.Within the theory of probability,several graphical representations existfor uncertain ty that feature bothdirected graphs (Wright 1921; Howardand Matheson 1981; Pearl 1986;Cooper 1984) and undirected graphs(Speed 19781 Spiegelhalter 1985). Thedifferent approaches share the basicconcept of the factorization of anunderlying joint distribution and theexplicit representation of conditionalindependence. We use a directedgraph representation method becausedirection is central to our discussion.In particular, we use the influencediagram representation scheme(Howard and Matheson 1981).Although the influence diagram has

    proven intuitive for knowledge acqui-sition, the representation is preciseand well-grounded in theory. Each ofthe oval nodes in an influence diagramrepresents a random variable or uncer-tain quantity, which can take on twoor more possible values The arcs indi-cate conditional dependence: a vari-ables probability distribution dependson the outcomes of its direct predeces-sors in the graph. For example, threepossible influence diagrams exist forthe two random variables X and Yshown in figure 2. In the first case, Xhas no predecessors, so we assess amarginal (unconditional] distributionfor X and a conditional distribution forY given X. In the next case, with thearc reversed, we assess a marginal dis-tribution for Y and a conditional dis-

    Figure 2.Influence Diagrams with Two NodesThe joint probability distribution for twovariables can be represented by three dif-ferent influence diagrams In diagram a,there is a probability distribution for vari-able X and a conditional distribution forvariable Y given that we observe the valueof X (indicated by the arc from X to Y) Indiagram b, there is an unconditional dis-tribution for Y and a distribution for Xgiven Y Diagram c asserts that the twovariables are independent because we canobtain their joint distribution from anunconditional distribution for each. Anytwo var iables can be modeled by a or b,but the missing arc in diagram c imposesa strong assump tion on the relationshipbetween X and Y, namely, that knowing Xwill not change our distribution for Y andvice versa

    tribution for X given Y. Both corre-spond to the same fundamental modelat the underlying joint distribution,but the two diagrams represent twodifferent ways of factoring the model.The transformation between them,which involves reversing the directionof conditioning and, hence, the rever-sal of the arc, is simply Bayes theo-rem. Finally, in the third case, neithernode has any predecessors, so X and Yare independent. Therefore, we can

    obtain the joint by assessing marginaldistributions for both X and Y. Whenthe two variables are independent, weare free to use any of the three forms,but we prefer the last one, whichexplicitly reveals this independence inthe graph.As a detailed example, consider theinfluence diagram in figure 3. This fig-ure shows the relationship betweendiabetes and a possible symptom ofthe disease, blue toe. Associated withthe influence diagram a re the proba-bility distributions, a marginal(unconditional) distribution for dia-betes, and a conditional distributionfor blue toe given diabetes.In figure 4, we see four possibleinfluence diagrams for the uncertainvariables X, Y, and Z. In the firstcase-the general situation-the threevariables appear to be completelydependent. We assess a marginal dis-tribution for X and conditional distri-butions for Y, given X, and Z given Xand Y. In general, there are n! factor-izations of a joint distribution amongn variables. Each possible permutationleads to a different influence diagram.In the second case in the figure, thethree variables are completely inde-pendent; in the third case, X and Y aredependent, but Z is independent ofboth of them. In the fourth case, wesee conditional independence. Theabsence of an arc from X to Z indi-cates that although X and Z are depen-dent, they are independent given Y.This type of conditional independenceis an important simplifying assump-tion for the construction and assess-ment of uncertainty models.Returning to the earlier diabetesexample, blue toe is caused byatherosclerosis, which might becaused by diabetes. This relationshipis shown in figure 5. The absence of adirect arc from diabetes to blue toeindicates that given the state ofatherosclerosis, knowing whether thepatient has diabetes tells us nothingabout whether the patient has bluetoe. The probability distributionsneeded to completely describe thismodel are shown below the influencediagram. They are a marginal distribu-tion for diabetes [the same as previ-ously), and conditional distributionsfor atherosclerosis, given diabetes, andblue toe given atherosclerosis.


  • 8/22/2019 Thinking Backwards


    formation is accomplished by revers-ing the arcs in the influence diagram,which corresponds to the general ver-sion of Bayes theorem (Howard andMatheson 1981; Olmsted 1983;Shachter 1986), as shown in figure 6.As long as no other path existsbetween chance nodes (either proba-bilistic or deterministic node] X and Y,we can reverse the arc from X to Y. Ifthere were another path, the newlyreversed arc would create a cycle. Inthe process of the reversal, bothchance nodes inherit each others pre-decessors. This inher

Search related