Explainable Systems: The Inference Web Approach

Explainable Systems:

The Inference Web Approach

Paulo Pinheiro da SilvaStanford University

In collaboration with Deborah L. McGuinness, Richard E. Fikes, Cynthia Chang, Priyendra Deshwal, Dhyanesh Narayanan, Alyssa Glass, Selene Makarios, Jessica Jenkins, Bill Millar, Eric Hsu and many people from IBM, SRI, ISI, IHMC, U. Toronto, U. Trento, U.

Fortaleza, U. Texas Austin, Rutgers U., Maryland U., Batelle, SAIC, UCSF, MIT W3C

Paulo Pinheiro da Silva

Overview

1. What are explainable systems and why should we care about them?

2. Inference Web: Enabling Explainable Systems

3. Explainable Systems in Action

4. Explainable Systems 10 years from now


Explanation NeedGoogle-2.0, Google-2.0,

where is where is Paulo’s office?Paulo’s office?

1) Stanford, CA, USA

2) Manchester, UK

I need to send Paulo a letter

but I don’t know his address.

I believe Paulo lives in the U.S.

So, Stanford, CA, USA.

appears to be a possible answer.

Google-2.0, why is Google-2.0, why is Paulo’s address Paulo’s address

“Manchester, UK”?“Manchester, UK”?

[Betty]


Explanation in Action

Paulo At Manchester, UK

Paulo At University of Manchester University of Manchester At Manchester, UK

transitivity of At

Source: http://www.cs.man.ac.uk/~pinheirpSource usage: May/2002

Source: http://www.cs.man.ac.ukSource usage: May/2002

OK, “Manchester, UK” was OK, “Manchester, UK” was Paulo’s address in May, 2002 Paulo’s address in May, 2002

and we are in 2005 !!and we are in 2005 !!

Why should I Why should I believe these?believe these?

Why should I Why should I believe this?believe this?

I’ll send his letter to Stanford.I’ll send his letter to Stanford.

[Betty]


What are Explainable Systems?

question answer

expl. 1explanationrequest 1

explanationrequest n expl. n

…

question

answer explanationrequest 1

answer

explanation 1expl. 1

answerunderstanding

expl. n

…

question

explanationrequest 1

explanationrequest 1

explanationrequest n

…

explanation n[Bob]


Why should we care about explainable systems?

As system users, we often need: To understand system’s response To trust system’s responses

Many explanation concerns are the same as in early systems such as Shortliffe’s MYCIN [1976] Swartout’s XPLAIN [1983]


Why should we care about explainable systems even more now? Systems are far more complex than 30

years ago Hybrid and distributed processing, e.g., web services, the

Grid Large number of heterogeneous, distributed information

sources, e.g., the Web More variation in reliability of information sources, e.g.,

information extraction Sophisticated information integration methods, e.g., SIMS,

TSIMMIS

Now we have less understanding (and sometimes less trust) of system’s answers and behavior

Now we have even more reasons for systems to explain their responses


How to Enable Explainable Systems?

Which information do

I have to generate an

explanation?

1 -> ((allof (the played-by of (the instances of Project-Leader)) where (It isa Person)) = (:set *Helen *Jody)) 2 -> (allof (the played-by of (the instances of Project-Leader)) where (It isa Person)) 3 -> (forall (the played-by of (the instances of Project-Leader)) where (It isa Person) It) 4 -> (the played-by of (the instances of Project-Leader)) 5 -> (the instances of Project-Leader) 5 (1) Local value(s): (:set *COGS-Proj-Leader-1 *HI-LITE-ProjectLeader-1 *SKIPR-ProjectLeader-1) 6 -> (:set *COGS-Proj-Leader-1 *HI-LITE-ProjectLeader-1 *SKIPR-ProjectLeader-1) [for (the instances of Project-Leader)] 6 <- (*COGS-Proj-Leader-1 *HI-LITE-ProjectLeader-1 *SKIPR-ProjectLeader-1) [(:set... 5 (2) From inheritance: (:set *COGS-Proj-Leader-1 *HI-LITE-ProjectLeader-1 *SKIPR-ProjectLeader-1)

I may have (or may be able to

record) data describing how I

manipulate information to

produce answers!

question answer

expl. 1explanationrequest 1

explanationrequest n expl. n

…


Explainable System Challenge

Explanation

TrustUnderstanding

Information Manipulation Data

The GAPThe GAP


Overview

1. What are explainable systems and why

should we care about them?

2. Inference Web: Enabling

Explainable Systems




Requirements for Explainable Systems Information Manipulation Traces

hybrid, distributed, portable, shareable, combinable encoding of proof fragments supporting multiple justifications

Presentation multiple display formats supporting browsing, visualization, etc.

Abstraction understandable summaries

Interaction multi-modal mixed initiative options including natural-language and GUI dialogues, adaptive, context-sensitive interaction

Trust source and reasoning provenance, automated trust inference

[McGuinness & Pinheiro da Silva, ISWC 2003, J. Web Semantics 2004]



Explanation

Proof Markup Language

InformationManipulation

Data


Proof Markup Language:Node Sets and Inference Steps

AND Intro (Î)

ModusPonens

(MP)

A

DirectAssertion

From Doc1

B

DirectAssertionfrom Doc2

A->(A^B)

Direct AssertionFrom KB1

A->(A^B) A

A^B

MP A B

A^B

ÎDA

A^B

A^B

DirectAssertion (DA)

from KB1

A DAG ofPML Node Sets(a collectionof justifications)

ExtractedProofs for the conclusion A^B


Encoding Hybrid and Distributed Proof Fragments

conclusion: A ^ B

http://foo.com/NS.owl#NS123

http://foo.com/NS.owl#NS124 http://bar.com/NS.owl#NS125

Proof Markup Language has a web-based solution for distribution Specification written in W3C’s OWL Each node set has one URI

Node sets can be used to combine proofs generated by multiple agents

OMEGA [Siekmann et al.,CADE2002] has a nice solution for hybrid proofs

hasLanguage: KIF(and A B)

rule: Modus Ponens (MP)hasEngine: JTP


Information Manipulation Traces

Proof Markup LanguageDifferences Formal Proofs Information

manipulation traces

Use of rules MandatoryOptional use or use of

‘unregistered rule’

SentencesWritten in some formal language (e.g., KIF, CL,

DIMACS, etc.)

Written in a formal or informal language including

natural language

Use of multiple representation languages

Uncommon Common

Proof Markup Language covers the full spectrum of information manipulation traces!

[Pinheiro da Silva, McGuinness & Fikes, IS 2005]



Explanation


ProvenanceMeta-data


Data


Infrastructure: IWBase Meta-data useful for disclosing knowledge

provenance and reasoning information such as descriptions of inference engines along with their supported inference

rules Information sources such as organizations, publications and

ontologies Languages along with their axioms

Core IWBase as well as domain IWBases OWL files for interoperability and database for

scaling [McGuinness & Pinheiro da Silva, IIWeb 2003]


Statistics for relevant domain independent meta-data:

12Languages

6Derived Rules

10Method Rules

38Declarative Rules

56Axioms

29Inference Engines

Infrastructure: Core IWBase

select

select



Explanation

Presentation

ProvenanceMeta-data


Data



Browsing Proofs (1/2) Enable the visualization of proofs (and abstracted proofs) Proofs can be “extracted” and browsed from both local and

remote PML node sets and can be combined Links provide access to proof-related meta-information

selectselect


Browsing Proofs (2/2)



Explanation

Presentation

Abstraction

ProvenanceMeta-data


Data



Knowledge Provenance Elicitation

A^BDAÎMP

A

DA

B

DA

A->(A^B)

DA

A->(A^B) A

A^BMP

A B

A^BÎ

A^B

Dir.Ass.

(CNN,BBC) (BBC,NYT) (CNN)

CNNBBC NYT

Why should I believe this?

Google-2.0 says ‘A^B’ is the answer

for my question.

“has opinion”“has opinion”

“has opinion”

Provenance information may be essential for users to trust answers.Data provenance (aka data lineage) is defined and studied in the database literature. [Buneman et al., ICDT 2001] [Cui and Widom, VLDB 2001]Knowledge provenance extends data provenance by adding data derivation provenance information[Pinheiro da Silva, McGuinness & McCool, Data Eng. Bulletin, 2003]


Knowledge Provenance Example

Answer

Source

Source


Abstracting Proofs

Explanation tactics (a.k.a. rewriting rules) may be used to abstract proofs into more understandable and manageable explanations

Enable the use of axioms as inference rules preventing the presentation of primitive (and potentially less interesting and useful) rules

Eliminate intermediate results from proofs


Abstracting Proofs: An Example (1/2)

Direct assertion

(Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03)

(Holds* (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03)

Generalized Modus Ponens

(Holds ((hasOffice JoesephGradgrind GradgrindFoods) Apr1_03)

(implies (and (Holds (owner ?person ?object) ?when) (organization ?object)) (Holds* (hasOffice ?person ?object) ?when))

Direct assertion(organization

GradgrindFoods)

Assumption

(not (Ab (hasOffice JosephGradgrind ?where) ?when))

Direct assertion(Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03)

Direct assertion

(organization GradgrindFoods)

(Holds (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03)

(implies (and (Holds* ?f

?t)) (not (Ab ?f ?t)) (Holds ?f ?t))

Organization Owner Typically Has Office at Organization

Generalized Modus PonensDirect assertion

ABSTRACTED PROOF

(Holds ((owner ?person ?object) ?when)

(implies (and (Holds (owner ?person ?object) ?when)) (organization ?object)) (Holds* (hasOffice ?person ?object) ?when))

(implies (and (Holds* ?f ?t)) (not (Ab ?f ?t)) (Holds ?f ?t))

(not (Ab (hasOffice ?person ?object) ?when))

Direct assertion

(Holds ((hasOffice ?person ?object) ?when)


(Holds* ((hasOffice ?person ?object) ?when)


(organization ?object)

Direct assertion

Abstractor algorithm1) Match conclusion (key for

selecting tactics)2) Match leaf nodes3) Unify

5) Apply the assertion-level rule6) Propagate justified nodes

Direct assertion

Tactic Library

Explanation tactic: “Organization Owner Typically Has Office at Organization”

4) Propagate conclusion


Abstracting Proofs: An Example (2/2)

Direct assertion(Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03)

Direct assertion(organization

GradgrindFoods)

(Holds (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03)

Organization Owner Typically Has Office at Organization

ABSTRACTED PROOF

ABSTRACTED PROOF IN DISCURSIVE STYLE

A rule says that the owner of an organization typically has an office in an organization

Because • JosephGrardgrind owned GradgrindFoods on April 1st 2003 • GradgrindFood is an organization

therefore • JosephGradgrind had an office at GradgrindFoods on April 1st, 2003.

Assertion-level rules are introduced in [Huang, PRICAI 1996].

Maybury describes strategies for rewriting abstracted proofs into English [AAAI 1991, AAAI 1993].

Explanation tactics supports multi-level abstraction of proofs



Explanation

Interaction

Presentation

Understanding

Abstraction

ProvenanceMeta-data


Data



Explaining Answers: GUI Explainer

Users can exit the explainer providing feedback about their satisfiability with explanation(s)

Users can ask for alternative explanations

Select action



Explanation

Presentation

Abstraction

Proof Markup LanguageInference

Meta-LanguageInference

RuleSpecs

ProvenanceMeta-data


Data

Interaction

Understanding


Inference Meta Language (InferenceML)

ndUI: '(forall (' N ')' q ')' |- ' (forall (' N - N.i ')' q[t/N.i] ')';; (Name N) (Sent q) (Term t)

Example:

An inference rule involves pattern of transformations on expressions to produce a conclusion

InferenceML uses schemas to state such transformations

InferenceML defines a schema to be a pattern, which is any expression of CL in which: some lexical items have been replaced by a schematic

variable (or meta-variable)


(and A B)

MP

(A)

DA

(implies (A) (and A B))

DA

Checking Proofs

MP: x; '(implies ' x y ')' |- y ;; (Sent x y)

(implies (A) (and A B)); |-

FromIWBase

(A) (and A B)

binding of expressions to schematic variables:

• x binds to (A)• y binds to (and A B)

the rule schema instantiates directly to:

(A) ; (implies (A) (and A B)) |- (and A B)

=


Trust


Explanation

Presentation

Abstraction

InferenceMeta-Language

InferenceRule

Specs

ProvenanceMeta-data


Data

Interaction

Understanding



IWTrust: Trust in Action

(CNN,XYZ) (XYZ,NYT) (CNN)

A^BDAÎMP

A

DA

B

DA

A->(A^B)

DA

A->(A^B) A

A^BMP

A B

A^BÎ

B

DA

CNNXYZ NYT

Why should I trust the answer?

0

0

++

++

++

+

+ +0

Google-2.0 says ‘A^B’ is the answer

for my question.

? ?

?

Trust can be inferred from a Web of Trust.

IWTrust provides infrastructure for building webs of trust.

The infrastructure includes a trust component responsible for computing trust values for answers.IWTrust is described in[Zaihrayeu, Pinheiro da Silva & McGuinness, iTrust 2005]

A^B


Inference Web and Paulo Paulo is a co-technical leader of the Inference Web

project Paulo was the main IW developer during 1 ½ years Paulo has been the manager of the IW development

team including members with the following profile: 1 research programmer 3 masters students 1 Ph.D. student

Paulo has organized the IW weekly meetings Paulo has been responsible for presenting and

demonstrating IW solutions at several DARPA and ARDA PI meetings

Paulo has participated of the writing of grant proposals


Overview

1. What are explainable systems and why

should we care about them?

2. Inference Web: Enabling Explainable

Systems




Application Areas Information extraction – IBM (UIMA), Stanford (TAP) Information integration – USC ISI (Prometheus/Mediator); Rutgers

University (Prolog/Datalog) Task processing – SRI International (SPARK) Theorem proving

First-Order Theorem Provers –SRI International (SNARK); Stanford (JTP); University of Texas, Austin (KM)

SATisfiability Solvers – University of Trento (J-SAT) Expert Systems – University of Fortaleza (JEOPS)

Service composition – Stanford, University of Toronto, UCSF (SDS) Semantic matching – University of Trento (S-Match) Debugging ontologies – University of Maryland, College Park

(SWOOP/Pellet) Problem solving – University of Fortaleza (ExpertCop) Trust Networks – U. of Trento (IWTrust)

No single explanation approach has been used in so many diversified areas as Inference Web!


Extraction as Inference Goal: To provide browsable justifications of

information extraction Strategy: Reuse, adapt, and integrate

existing technology: justification technology - Inference Web extraction technology - IBM’s UIMA

Requires that systems to describe their processing as logical inferences Requires a new perspective: IE as Inference [Murdock, Pinheiro da Silva et al., AAAI’s SSS 2005]


Extraction As Inference:An Example (1/2)

Direct assertion from KB1

(Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03)

(Holds* (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03)


(Holds ((hasOffice JoesephGradgrind GradgrindFoods) Apr1_03)


(implies (and (Holds (owner ?person ?object) ?when) (organization ?object)) (Holds* (hasOffice ?person ?object) ?when)) Assumption

(not (Ab (hasOffice JosephGradgrind ?where) ?when))

(implies (and (Holds* ?f

?t)) (not (Ab ?f ?t)) (Holds ?f ?t))

Generalized Modus PonensDirect assertion from KB1

Document Coreference


Extracted Entity Classification

IBM Cross-Annotator Coreference

Joseph Gradgrind is the ownerof Gradgrind Foods[organization]

[refers to GradgrindFoods]

Entity Identification

IBM EAnnotator

Joseph Gradgrind is the ownerof Gradgrind Foods[organization]

Entity Recognition

Direct assertion from gradgrind.txt

Joseph Gradgrind is the ownerof Gradgrind Foods



Solution:

A taxonomy of extraction tasks expressed as inference rules

Components that record IE justifications using rules in the taxonomy

We have identified 9 types of extraction inferences: 6 for analysis, and 3 for integration


Extraction As Inference:An Example (2/2)

Paulo is a PhD student at University of Manchester.

Paulo At Manchester, UK

Paulo At University of Manchester University of Manchester At Manchester, UK

transitivity of At

http://www.cs.man.ac.uk/~pinheirp http://www.cs.man.ac.uk

University of Manchester is located in Manchester, UK.

Why should I believe that Why should I believe that these documents say that?these documents say that?

Why should I Why should I believe these?believe these?

Why should I Why should I believe this?believe this?

TheoremProving

InformationExtraction

[Betty]


Explaining Tool Responses

Explain (v. tr.)1: “To offer reasons for the actions, beliefs,

or remarks of (oneself).”

Questions and Answers

Requests and Responses

GeneralizationInferences for explaining answers (aka beliefs)

Inferences for explaining answers (aka beliefs), and tasks (including actions)

New perspective: Task processing as inference

1Dictionary.com


NL Explainer: An Example

<user>: What are you doing now?<system>: I am trying to get an approval to buy a

laptop.<user>: Why? [note: “Why?” is rephrased to “Why are you trying

to get an approval to buy a laptop?]<system>: I have completed the previous requirement

to get quotes so I am now working on get approval.<user>: OK, I am happy with your explanation.

Levering explanation dialogues as in [Fiedler, IJCAI 2001]

Using natural language support as in [Allen et al., AAMAS 2002]


Overview

1. What are explainable systems and why should

we care about them?

2. Inference Web: Enabling Explainable Systems


4. Explainable Systems 10 years from

now


Inference Web Contributions

Trust

Explanation

Presentation

Abstraction

InferenceMeta-Language

InferenceRule

Specs

ProvenanceMeta-data


Data

Interaction

Understanding


1. Language for encoding hybrid, distributed proof fragments based on web technologies. Support for both formal and informal proofs (information manipulation traces).

2. Support (registry, language, services) for knowledge provenance.

4. Multiple strategies for proof abstraction, presentation and interaction.5. End-to-end trust value computation for answers.

3. Declarative inference rule representation for checking hybrid, distributed proofs.

6. Comprehensive solution for explainable systems.

1 2 3

4

2

5

6

4

4


Open Issues Automated generation of explanation tactics Performance for abstracting and checking proofs Use of machine learning and user modeling to

support interaction Adaptive explanations Explanation contexts Modeling user knowledge

Metrics and evaluations for explainable systems


http://www.w3.org/2004/Talks/0412-RDF-functions/slide4-0.html

Three Years From Now An initial research community working on explainable

systems Adaptive explanations based on user modeling IWBase registration of a large set of software systems

Registration of a comprehensive set of primitive rules Established library of explanation tactics First generation of metrics and

evaluation methods for explainable systems

Inference Web is a solution for the Semantic Web proof and trust layers


Ten Years From Now An established research community working on

explainable systems A theory for explainable systems Established metrics for explainable systems First (or second) generation of industrial

explainable systems A standard language for encoding information

manipulation traces (probably derived from PML among other proposals). The language will include support for the following: probabilistic reasoning inductive reasoning


and Inference Web Immediate connections

Explaining Task Processing TaskTracer CALO

with Intelligent Information Systems team Explaining Tool Responses

Explaining WYSIWYT – with End Users Shaping Effective Software team

Potential connections Explanation generation

FilteringLearning

Explanation-based learning with Learning and Adaptive Systems team

Explaining pattern and object recognition from videos and graphs

with Computer Graphics and Vision

Documents

Explainable Systems: The Inference Web Approach