VALIDATION AND VERIFICATION OF EXPERT SYSTEMS USING EVIDENCE FLOW GRAPHS - / - ' /

Lee A. Becker, Peter. E. Green, R. James Duckworth, Jayant Bhatnagar

Intelligent Machines Laboratory Worcester Polytechnic Institute

Worcester, MA 01609

Abstract

This paper describes an ongoing investigation into the use of evidence flow graph techniques for performing V&V of expert systems. This method involves translating a rule-base into an evidence flow graph, a representation originally developed for real-time intelligent systems in distributed environments, and then running simulations of the evidence flow graph. Certain errors can be found during the translation process. The simulations can detect output sensitivity to rule firing order, to order of presentation of inputs, and to small changes in input values.

Figure 1 depicts our framework for the development of a verified expert system. One important feature of our approach is that the verification is divided into a set of distinct processes. Performance analysis and verification take place first at the knowledge level, then again after information about the execution environment has been incorporated. This is followed by a hardware failure effects analysis before testing in a simulated real-world environment. One cannot just verify a knowledge base. If the rule base is not invariant over all control strategies, then this must be known, and the rule base and control regime must be verified as a pair. In addition, any modification to either the rule base or the control regime requires that the pair be reverified. It is obvious that if the rule base were invariant over all control strategies, the control regime could be changed and reverification would not be necessary. This would support portability.

The work described in this paper was supported by NASA. grant numbers NAG-1-809 and NAG-1-964.

SFFECTS ANALYSIS

TESTING IN SIMULATED REAL TORLD ENVISONMENT

Figure 1 : The Proposed Model for Verification of Expert Systems

Evidence Flow Graphs (EFG) were developed at Worcester Polytechnic lnstitute as a representation for rule-based, HearsayIBlackboard-based, and communicating expert object-based expert systems 3p5.

An evidence flow graph representation for a rule-based expert system replaces each rule with a process that is triggered by the arrival of data into the data base, either from an external source or from the execution of another decision process, as shown in figure 2. The result is a graph with data flow properties ' in which input data triggers rule executions until outputs are generated.

k a o n p m u s (DP) execuoon u m a d by

rhe 3mval or m a s q a mvellmg along arcs

2% A Simple Ue-based E x p n Sysnm

Rulel: I f A a n d B h C RuieZ: I fDandEthenF Rule!: I f C m d F r h e n G

2. Xules m a n p i onm endence Bow gnph

DP1: Whm A and B a m v e send C D DP3

DPZ: WhenDmdEamvesendFtoDPJ

DP3: When C md F arnve ourpur G

Figure 2: Transformation of a Rule-Based Expert System into an Evidence Flow Graph

The word 'process' here is used in the operating systems sense, as meaning computational object with its own register state, memory map, local data and stack, and access to code and fixed data. In an Evidence Flow Graph context, a decision process is a node in the graph which has an input queue for receiving messages from its input graph arcs. State variables may be contained within decision processes and partial solutions are transmitted along the arcs of the graph.

An EFG is a directed graph which represents decision making in terms of the collective behavior of several independent processes. The processes are characterized by the ability to make decisions in a limited problem domain and by the ability to communicate the results of these decisions by passing messages to other decision processes. The processes may range in complexity from simple logical operations to implementations of complicated decision

making paradigms. An EFG can execute in a manner similar to a data flow program. As messages arrive at a decision process they are stored until all the messages necessary for the decision process to execute are available. The process then "fires", consuming each input message, and generating new messages which are passed to those decision processes which require them.

EFGs provide a unified representation that can be mapped onto different computer hardware architectures. These graphs and their application are being investigated as part of an overall research project into how to build intelligent systems that are able to function in real-time in uncertain environments.

In real-time systems uncertainty arises from a number of sources. During the processing of the input the environment may change. Input data may be erroneous due to noise, imperfect sensors, or faulty human data entry. Signal processing algorithms can produce false outputs, due to factors such as aliasing, which are used as inputs to the expert systems. Finally, the necessarily limited knowledge contained in the rule base itself may cause uncertainty.

EFGs are also of value in the verification of expert systems. Figure 3 depicts the use of flow graphs for performance analysis on different knowledge representations. An important feature of our approach is transforming the knowledge representation used into a graph theoretic form from which it can be analyzed and simulated using techniques developed for non-linear control systems.

FLOW GRAPHS Lr' SIMULATION dl

t VERIFY

BY SIMULATIONS

output J

Figure 3: Evidence Flow Graphs as a Unifying Representation

Knowledae-Level Verification For A Rule-Rased Re~resentatlon

The Knowledae-Re~resentation-to-EFG Translator

A translator takes the knowledge representation and yields an EFG. The knowledge is in the form of Horn-clause rules, where the antecedent is a conjunction of predicates and the consequent is a conclusion. There are specially designated input predicate nodes and output final conclusion nodes, as well as nodes for any subconclusions. For each rule there is a directed link from each of the predicates of the antecedent (input nodes or subconclusions) to the node of the conclusion or subconclusion in the consequent. Weights on the links are based on the number of conjuncts. When a parameter is referred to in several relational predicates, there is a directed link from the parameter to each of the nodes for the relational predicates. Appendix 1 contains the sample rule base. Appendix 2 contains the EFG representation that was generated from this rule base by the graph generator.

The translator combines graph generation with static rule base checking that precedes the dynamic testing implemented by the simulator. Static checking enables the identification of those rules that contain undefined conjuncts in their conditions.

The Simulator

In the research described in this paper we have investigated knowledge-level verification and have demonstrated our approach using a rule-based knowledge representation. Our system contains two modules: a rule-base-to-EFG translator and a simulation program. A third logical module, post-processing, currently is done by hand.

A simulator executes the EFG. All nodes, except for the input and relational predicate nodes, are updated with a weighted sum of the values of their input arcs. When several rules have the same conclusion, the update values are treated as a queue which takes the maximum of their input values.

The update values are sent as messages to nodes to which there is a directed arc. The values are real numbers between 0.0 and 1.0. For the initial stage of V&V (the knowledge level) it might be assumed that the work cells (nodes) fire as soon as their inputs are available and that there is no contention for computing resources. One could then pick processing cell times at random from a distribution and test for many possible execution sequences. Simulation continues until all activity ceases in the network.

Once a simple simulator was built and tested, it became evident that a post process will need to be developed to aid the user in analyzing the simulation results. The pattern of node firings (and message passings) is recorded in a logfile by the simulator. A post processor could then do various analyses on this file, for example to determine nodes that have never fired or nodes that have fired very often. The rules corresponding to these nodes warrant additional scrutiny. The post processor also could compile results from multiple runs with the same input, perhaps available at different times. Thus results from different input orderings and with different firing orders can be compared to see if the results are always the same. In other words, the output of the post processor will allow the identification of invariance of results with different input orderings, with different firing orders, as well as with parameter variation. The development of such a post processor is crucial to analysis of large-scale systems.

A variety of different types of testing are supported by this approach. The most common type of checking done on expert systems is for consistency 718112*13. Static analysis on the EFG can yield this kind of information. In fact, several systems which do consistency checking translate a rule base into a inference net or graph for their analysis 2. Such a graph structure could also be used to derive or generate sets of in uts for structural testing, if desired P2. We concentrate here on the kinds of dynamic testing which can be done using simulation.

If there are available test cases which specify the conclusion to be reached for a set of inputs, these can be executed using the simulator and wrong conclusions can be detected. This kind of testing can also be readily done by executing the expert system itself. However, there may be significant difficulty in assembling a large, well-distributed set of test cases O. With the proposed method there are a number of types of testing which do NOT require the

availability of test cases. All these involve running multiple simulations with inputs values randomly generated from within the operational profile.

One type of testing which is very significant, which does not require knowing the desired conclusion for a set of input values, is testing whether the same conclusion will be reached regardless of the order that the input values become available. This is relevant when the system acts on the basis of the first conclusion reached. For a given set of input values, multiple runs are made with different orderings of various subsets of the inputs.

For sensitivity testing the values of parameters are randomly varied within their operational profiles to determine whether any parameter is critical in its effect on the input, i.e. small changes in its value cause significant changes in the output. The effects of different degrees of belief of input predicates can also be examined. The EFG can be partitioned to allow this testing to be carried out only on the relevant subset of nodes.

For some applications it may be possible or necessary to specify critical conclusions that are to be reached only under certain conditions or are not to be reached under certain conditions. These specifications can be tested using multiple simulations with randomly generated inputs. For these applications, it is critical to partition the EFG to allow more exhaustive testing.

At present a prototype rule-base-to-EFG translator has been completed; this is written in LISP. A simulation program to run an EFG with varying input values and orderings has been completed; this is written in SIMSCRIPT. A rule base for a small expert system has been translated into an EFG, and the simulation programs have been used to run the network. Appendix 1 contains the sample rule base. Appendix 2 contains the EFG representation that was generated from this rule base by the graph generator. In addition, errors have been inserted into the sample

rule base to demonstrate the kinds of errors that the proposed approach can detect.

It was found that the order of presentation of input data can affect the output from a set of rules. By corollary, the order in which the inference engine executes the rules may affect the output. This can cause problems when undesired outputs are produced before all the data is available or before all possible rules have been executed. It was also found possible to affect the resultant output by making small changes to critical parameters.

The set of techniques developed is general in nature and has a number of advantages over other techniques for detecting problems as part of the verification process:

a) It provides a uniform representation for various knowledge representations and control strategies.

b) The EFG allows for analysis to recognize unused inputs and subconclusions, unreachable conclusions, disjoint and hence partitionable subgraphs, and relationships between inputs and outputs. It also provides a visually comprehensible representation in which many of these can be readily recognized.

c) It allows for simulation using techniques developed for non-linear stochastic systems.

d) It allows the consideration of different orders of input availability, and potentially allows for multiple data values for a single parameter.

e) It allows for sensitivity testing to determine where small changes in the values of input parameters will result in different conclusions.

Although the EFG techniques succeeded in discovering a number of kinds of errors, it is clear that additional work needs to be done tn

- - - - -- make these techniques practically useful. This work includes

1) the development of a program that will automatically develop simulation test sequences based on meta-knowledge about such items as possible ranges of input data and order of data availability, and

2) the development of a post-processor program to automatically analyze the output data from the simulation runs and to detect problems. The simulation program generates a large volume of data when performing monte-carlo analyses which is not practical to examine by hand. This post-processing program will need to embody knowledge about faults that could occur and how to detect them.

A major limitation of the original demonstration system was that it was limited to a single knowledge representation and control strategy, and that that control strategy was embedded into the EFG. In particular, the kinds of meta-knowledge previously embedded include the use of weighted sum as the update function for conjunction, the use of maximum as the update function for disjunctions, the comining of certainty factors with the number of conjuncts to form weights on the links. The control meta-knowledge must be stated independently and represented explicitly in order to allow the same network to be used with different control strategies and confidence calculations. This is a major thrust of our current work. We are developing a general node structure for evidence flow graphs which is capable of representing knowledge for a variety of knowledge representations, control strategies, and confidence calculations (e.g. MYCINIBayesian, Dempster-Shafer, OPS5, CLIPS). We are currently developing translators for two of these, Pearl's Causal Nets and CLIPS.

A successful start has been made on developing techniques which can be used for the verification of expert systems. The work described in this report has hopefully laid some of the foundation which can be used to assure that the expert systems used in our space program are reliable and safe.

Appendix 1 : Sample Rule Base

LHS* CF

(((has-skin) (moves-around) (breathes)) animal .9)

(((moves-around) (breathes) (eats)) animal .9)

RHS

((aniyal (has-fins) (can-swim)) fish .9)

(((bites) (length > 5) fish) shark .9)

(((edible) fish) .9)

(((has-wings) animal) .9)

((bird (can-f ly)) .9)

((bird (long-legs)) .9)

salmon

bird

canary

ostrich

(((warm-blooded) animal(suckles-young)) mammal .9)

((mammal (talks) (age < 100)) human

.9)

(((lives-on-trees) (age < 1 00) mammal) ape .9)

*The conjuncts are enclosed in parentheses, if they are input predicates, but not if they are inferred predicates, i.e. those on the RHS of some rule.

Appendix 2: Evidence Flow Graph Representation of the Rule Base

References

1. Ackerman, W.B. 1982. Data Flow Languages. )FEE C- 15.2: 15-25.

2. Chisvin, L. 1988. M a Discrete Fveu S~mulat~on to Predict the Network Communication Performance of Messaae-Rased Data Flow Multiprocesso~ Svstems. Master's Thesis, Worcester Polytechnic Institute, Worcester, MA.

3. Green, P.G. and W.R. Michalson. 1987. Real-Time Evidential Reasoning and Network Based Processing. Proceedhas of the IFFF First Annual International Conference on Neural Networks, Vol. 2, pp. 359-365.

4. Johnson, S. 1988. Validation of Highly Reliable, Real-Time Knowledge-Based Systems. Proceedinas of the 2nd Annual Worksho~ on Space Operations Automation

Robotics (SOAR 8Q.

5. Michalson, W.R., Green, P.E., Duckworth, R.J. 1987. Evidence Flow Graphs: A Unified Representation for Distributed Artificial Intelligence Systems. Worcester Polytechnic Institute Report EE871MP10.

6. Michalson, W.R. 1988. A Computing . .

e for Real-Ttme De- Ph.D. Dissertation, Worcester Polytechnic Institute, Worcester, MA.

7. Nguyen, T.A., Perkins, W.A., Laffey, T.J. and Pecora, D. 1985. Checking an expert system knowledge base for consistency and completeness. IJCAI9, pp. 376-378.

8. Nguyen, T.A., Perkins, W.A., Laffey, T.J. and Pecora, D. 1987. Knowledge base verification. mMaaazlne, V01.8, NO.^., pp. 65-79.

10. O'Keefe, R.M., Balci, O., and Smith, E.P. I 1987. v e r t Svstem Performance,

IEEE Expert, Vo1.2, NO.^., pp. 81 -89.

11. Russell, E.C. 1983. W n a Models with Sirnscri~t 11.5. CACI, Inc., Los Angeles, CA.

12. Stachowitz, R.A., Chang, C.L., Stock, T.S., and Combs, J.B. 1987. Building Validation Tools for Knowledge-Based Systems. Proceedincls of the First Annual Worksho~ on S ~ a c e O~erations Automation and Robotics (SOAR '87), pp. 209-21 5, NASA Conference Publication 2491, Houston, TX., August 1987.

13. Suwa, M., Scott, A.C. and Shortliffe, E.H. 1982. An approach to verifying completeness and consistency in a rule-based expert system. Al M-, Vo1.3, No.4, pp. 16-21.