mindswap maryland information and network dynamics lab

mindswapmaryland information and network dynamics lab semantic web agents project

Debugging and Repair of OWL Ontologies

Aditya Kalyanpur

MINDSWAP Research Group

University of Maryland, College Park

04/10/23 2


Thesis Statement

• Hypothesis:“Techniques developed in this Thesis, to support the Debugging and Repair of Semantic Defects in OWL Ontologies, are useful for Ontology Modelers and Users.”

04/10/23 3


Contributions• Developed a theoretical framework for

understanding, pinpointing and resolving defects in OWL Ontologies

• Implemented framework in a widely used OWL Ontology Editor/Reasoner

• Demonstrated through Performance and Usability Evaluations that framework is useful for OWL Ontology modelers/users

04/10/23 4


Outline• Introduction

– OWL and DL– Motivation for Debugging OWL Ontologies

• Thesis– End-to-End Framework for Debugging/Repair

• Service I: Axiom Pinpointing• Service II: Root Error Pinpointing• Service III: Ontology Repair

• Conclusion and Future Work

04/10/23 5


Ontologies and OWL• What is an Ontology?

– A formal vocabulary used to define terms (concepts/relations) in a certain domain

• What is OWL?– Web Ontology Language– Key Component of the Semantic Web (Berners Lee et al, 2001)– Published as W3C Recommendation in Feb 2004 – Design based on Web architecture

• URIs for identifying resources• Links to combine Ontologies• XML for syntax

– Has 3 sub-languages: OWL-Lite, OWL-DL, OWL-Full– Semantics based on Description Logics

04/10/23 6


Description Logics (DLs)• Formalism for representing knowledge• Decidable subset of First Order Logic (FOL)

• Basic building blocks– Concepts: 1-place predicates in FOL (sets), e.g., Person (x)– Roles: 2-place predicates in FOL (relations), e.g., hasBrother (x,y)– Individuals: Constants, e.g., Bob, John

04/10/23 7


What can OWL-DL do for you?

04/10/23 8


OWL(DL) Ontology Engineering Tools• Swoop: “Web Ontology” Browser/Editor*

• Browsing/Editing of Multiple OWL Ontologies

• Plenty of Advanced Functionalities- Search/Compare/Map across Ontologies (terms)- Change Management and Version Control- Collaborative Annotation- Automatic Partitioning of Ontologies• Recent release of Swoop has 8000+ downloads• 300+ users on mailing lists• Uses Pellet OWL-DL Reasoner

*Aditya Kalyanpur, Bijan Parsia, Evren Sirin, Bernardo Cuenca-Grau, James Hendler, "Swoop: A 'Web' Ontology Editing Browser", Journal of Web Semantics Vol 4(2), 2005

04/10/23 9


Working with OWL Ontologies• However, initially Swoop had little support for

debugging/repairing defects in OWL– Common defects: unsatisfiable concepts and

inconsistent ontologies

• Lots of other Ontology Editors with little or no debugging support, e.g.– Knublauch et al, “The Protege OWL Plugin..”, ISWC 2004– Sure et al, “OntoEdit..”, ISWC 2002– Haarslev et al, “OntoXpl..”, WI 2004

04/10/23 10


Defects in OWL Ontologies• Broadly, four kinds:

– Syntactic Defects (e.g. non well-formed XML)• Uninteresting, most cases trivial to solve

– Style Defects (e.g. unintended redundancy in the ontology)

• Analogous to messy code

– Modeling Defects (unintended inferences)• Requires domain knowledge to detect/resolve

– Semantic Defects (unsat. concepts & inconsistent ont.)

• Automatically detected by a DL Reasoner, difficult to explain or resolve correctly

04/10/23 11


Related Work: DL Explanation/Debugging• Explanation

– Explaining subsumption in CLASSIC, ALC (Mcguinness et al 95)– …

• Semantic Defects– Non-Standard Reasoning Services for Debugging DL Terminologies

(Schlobach et al 2003) – Knowledge Integration in DL (Meyer et al 2005)– A General Diagnosis Method for Ontologies (Friedrich et al 2005)– Heuristic Approach to Debugging OWL-DL Ontologies (Wang et al 2005)– …

• Modeling Defects– Chimaera: Merging and Diagnosis Ontology Environment (Mcguinness et

al, 2000)– Symptom Ontology (Baclawski et al 2004)

• Miscellaneous– Inference Web (Mcguinness et al 2003)

04/10/23 12


Motivation: Debugging Semantic Defects

144 out of 395 classes in this ontology are unsatisfiable!

?!

Tambis OWL Ontology (Swoop Screenshot)

04/10/23 13


Causes of Semantic Defects• Main Problem: Difficulty in understanding

inferences in complex and expressive DLs• Additional Issues:

– Interlinking of OWL Ontologies• e.g., Merge of CYC and SUMO creates numerous

unsatisfiable concepts (Schlobach 2005)

– Migration to OWL• Translating ontologies in languages pre-dating

OWL such as KIF, GRAIL, DAML, OIL etc., e.g., Tambis

04/10/23 14


Need for Debugging Services• Without adequate debugging support,

– Newcomers to OWL have difficulty understanding errors

• e.g., leads to weakening ontology (under-specifying)

– Even DL experts find it hard to debug errors in large, complex ontologies

• My goal is to build debugging services that:– Explain the cause of the contradiction– Pinpoint the key problematic components– Suggest remedies

04/10/23 15


Automated Support• Debugging / Repair Services

1. Axiom Pinpointing• Extract the error justifications, i.e., minimal sets

of (parts of) axioms responsible for the error

2. Root Error Pinpointing• Separate the critical or root errors from the

dependent or derived ones

3. Ontology Repair– Automatically generate repair plans w/

cost/benefit analysis

04/10/23 16


Axiom Pinpointing Example• SWEET-JPL (1537 classes, 2250 axioms) : 1 unsat. class

Minimal set of axioms from ontology responsible for error or clash

(Justification)

Description of clash

04/10/23 17


Axiom Pinpointing• Justifications:

Given O ² E,

Justification(E) = O’, where O’ µ O, O’ ² E and for any

O’’ ½ O’, O’’ ² E• Minimal set of axioms in the ontology in which the entailment

holds• There could be more than one justification for an entailment• (Note: Unsatisfiability is an entailment, O ² C v ?)

• Axiom Pinpointing service finds all justifications for a given ontology entailment

04/10/23 18


Axiom Pinpointing• Computing a Single Justification

– Two Distinct Approaches:• Glass Box (Reasoner Dependent)

– Modify internals of DL reasoner to track axioms responsible for entailment

• Black Box (Reasoner Independent)– Treat reasoner as an “Oracle” and perform brute-force

analysis (add/remove axioms from the ontology) to arrive at justification

04/10/23 19


Glass Box• DL reasoners typically use Tableau Algorithms:

– Proof By Refutation• To prove O ² E, we check consistency of O [ {:E}

– Decide consistency of an ontology by trying to construct a model for it: a completion graph• Ontology is inconsistent if every branch of graph has a clash (contradiction)

• Basic idea to find justification: Tableau Tracing (Baader et al, 1995)– Track the set of axioms responsible for each update to completion graph

– Identify clash-causing axioms– Earlier work done for a weaker DL: ALCF

More

04/10/23 20


Tableau Tracing Challenges for OWL-DL

• Why is this hard?– Need to deal with internal modifications done by the

reasoner, e.g., normalization, absorption (Horrocks, 1997)

– Ensuring Minimality• Need to determine if clash depends on a non-deterministic

choice or not• As expressivity increases, tableau operations become

more complex – difficult to identify precisely which axioms are responsible for an event (e.g., node merges)

Current Solution: Solves above problems, except includes few extraneous axioms due to one particular expansion rule (· nR rule)

*Kalyanpur et al, “Debugging Unsatisfiable Concepts in OWL Ontologies " Journal of Web Semantics, Vol 3(4), 2005

04/10/23 21


Glass Box EvaluationTime for a Single (In)Consistency Check vs. it’s Justification

Entailment TestsGlass Box Algorithm (Tracing) does not introduce much timing overhead

More

04/10/23 22


Glass Box Problems• Need to modify internals of reasoner

• Sometimes access to internals may not be possible (e.g. RACER)

• Finding All Justifications requires a saturation of the completion graph– i.e., continue expansion to find all possible

clashes!

• Hence we explore Black Box techniques

04/10/23 23


Black Box: Single Justification• Given Ontology O and unsatisfiable concept C,

– EXPAND:• create a new empty ontology O’• while (C is satisfiable in O’)

– insert new axioms from O into O’– SHRINK:

• for each axiom in O’– if C is unsatisfiable in (O’ -

» O’ Ã O’ -

• Final Output O’ contains a single justification• Heuristics to reduce no. of iterations of each loop

– EXPAND stage: Start with concept definition axioms and add axioms that are structurally connected (also look for negation)

– SHRINK stage: Use a sliding window technique to prune a bunch of axioms in one go

04/10/23 24


Black Box Evaluation

Entailment Tests

Time for a Single (In)Consistency Check vs. it’s Justification

04/10/23 25


Axiom Pinpointing: Finding All Justifications• Tambis Ontology: Most Unsatisfiable Concepts have

between 2-4 Justifications.

04/10/23 26


Black Box: All Justifications• We use Reiter’s Hitting Set Tree (HST)

Algorithm– Short Introduction to Reiter’s Algorithm: A general

theory of diagnosis (1987)• A system (set of components) to be diagnosed is

characterized by:– Conflict Sets (CS): subset of components which cause the

inconsistency in the system

• Repair Solutions characterized by– Hitting Sets (HS): a HS for a collection of sets C is a set that

hits or intersects each set s 2 C

• Repair = minimal HS for collection of CS• Reiter HST algorithm generates minimal HS

04/10/23 27


HST Algorithm Example• Conflict Sets: {1,3,7}, {1,2,5}, {3,4,6}, {4,5,8}• Minimal Hitting Sets: {3,5}, {1,4}, {1,3,8}, …

1

3,4,63

4,5,84

58

46

4,5,8

4x

…continue expansion of tree

1,2,5

{1,3,4} is a Hitting Set (non minimal)

minimal

Early Path Termination

04/10/23 28


Modifying HST to Find All Justifications• Given an inconsistent ontology, Justifications ! Minimal

Conflict Sets– We have a technique to find an arbitrary justification for an

inconsistency– Our Goal is to find All Justifications, i.e., all minimal conflict sets

• Hence, we use HST as follows:– Initialize HST with any one justification– Remove axioms from the ontology and find new justifications on the fly– In building the HST, we discover all justifications

Reiter’s algorithm is used to find all minimal hitting sets…we use it to find all minimal conflict sets instead

• Also, we can make use of Reiter’s HST optimizations– However, algorithm is still exponential: depends on no. of justifications

and size of justifications

More

04/10/23 29


Evaluation

Entailment Tests

Time for a Single (In)Consistency Check vs. All Justifications

04/10/23 30


Axiom Pinpointing Issue• So far, we have seen how to find all justifications

for an entailment– Justifications are in terms of asserted axioms

– To resolve error, we can remove any one axiom• However, a lot of additional information may be discarded• We need to find parts of axioms responsible for error

04/10/23 31


(Precise) Axiom Pinpointing• Basic Idea: Split axioms into smaller sized

equivalent axioms*– Normalize and split across conjunctions

• A v C u D ! {A v C, A v D}• A t B v 8R.(C u D) ! {A v 8R.C, A v 8R.D,

B v 8R.C, B v 8R.D}

– Cannot split across existentials easily, i.e.,• A v 9R.(C u D) not equivalent to A v 9R.C u 9R.D

• Hence, introduce new concept E and rewrite as:• A v 9R.E, E v C, E v D, C u D v E*For details see: Kalyanpur et al “Beyond Axioms: Axioms: Fine-Grained Justifications for Arbitrary Entailments in OWL-DL" In DL 2006

04/10/23 32


• Precise JustificationsJUSTIFYprecise(E) in O = JUSTIFY(E) in O’where O’ is maximally split version of O

• Optimizations:– Lazy Splitting: Only split justification sets on the fly (instead of

splitting entire O beforehand)– Eliminate obvious irrelevant axiom parts

• i.e., entities appearing only once in the justificatione.g. Justification(C = ?): 1. C v B u D

2. D v E u :E u F

(B, F are irrelevant)

Axiom Pinpointing

04/10/23 33


Axiom Pinpointing Summary

• Can be used to find (Precise) Justifications• Great from a debugging point of view - pinpoint

axioms responsible for error• Additional Applications (beyond debugging):

– Explanation of arbitrary inferences (not just contradictions)• …since all inferences in OWL can be reduced to consistency

checking (Horrocks et al, 2004)– Optimization of standard reasoning tasks for continuously

changing ontologies (Halaschek et al, 2006)• …since Justifications act as a form of truth maintenance

04/10/23 34


Root Error Pinpointing• For a large no. of unsatisfiable classes

– Going down the list and fixing errors one by one is time-consuming, pointless and sometimes harmful

– Need to identify critical errors (roots) that propagate and cause errors elsewhere (derived)

– Examples of propagation of errors:• A v B, where B is unsatisfiable

• A v 9R.B, where B is unsatisfiable • A v B t C, where both B, C are unsatisfiable

04/10/23 35


Root Error Pinpointing

Only 3 Root Unsatisfiable Concepts!

Fixing 3 Roots resolves 78 Errors

New Set of Root Concepts Uncovered

Ã Fixing these leaves 0 errors

04/10/23 36


Dependency between Unsat. Classes

• Root and Derived– Root: named class whose unsatisfiability does not

depend on the unsatisfiability of another named class• E.g. A v B u :B

– Derived: named class whose unsatisfiability depends on the unsatisfiability of another named class

• E.g. A v B u C, where C is unsatisfiable

– From a justifications point of view, 9JD, JD 2 Justification(Derived),

9JR, JR 2 Justification(Root),

such that JD ¶ JR

04/10/23 37


• Problem: Finding all justifications of unsatisfiable concepts is exponential

• Optimizations to detect dependencies*:– Structural Tracing

• Detects Asserted Dependencies – A v ¸1.R u 8R.C …where C is unsatisfiable– A v 9R.C …where domain(R) is unsatisfiable

– Ontology Approximation• Detects Inferred Dependencies

– { A = 9P.D; B = 9P.E u C u :C; D v E}

Detecting Root/Derived

*For details see: Kalyanpur et al “Debugging Unsatisfiable Concepts in OWL Ontologies", Journal of Web Semantics, Vol 3(4), 2006

Here A v BBut B is root

More

04/10/23 38


EvaluationOWL

OntologyUnsat.

ConceptsRoot/Derived Step-1 Time /

Step-2 Time

Tambis 144 3/141 0.033s/1.899s

DICE 76 5/71 0.001s/ 7s

Transport 62 5/57 0.002s / 2.8s

Economy 51 34/17 0.01s / 2.5s

Chemical 37 2/35 0.01s / 0.14s

Terrorism 14 5/9 ~0s / 0.951s

Time to Compute Root/Derived

04/10/23 39


User Evaluation• How effective is Axiom Pinpointing/Root-Derived

– University – 8 unsat. classes (5 roots/ 3 derived)– Sweet-JPL – 1 unsat classes– Mini-Tambis – 30 unsat classes (5 roots/ 25 derived)

n=12 subjects

Main Points:- Axiom Pinpointing is key to understanding and resolving bug- Root-Error Pinpointing matches Axiom Pinpointing for large no. of unsatisfiable classes- Both services together reduce time by a factor of 2-3

- 2-tailed T-test shows G4 significantly better than G1 for p < 0.01

04/10/23 40


Ontology Repair• So far, we have focused on:

– Identifying root errors in the ontology

– Extracting minimal set of axioms responsible for the error

• But which axioms do we remove or modify?– To aid this task, we designed an Ontology Repair service:

• Acts as guideline for repair by suggesting solutions with a cost/benefit analysis

04/10/23 41


Ontology Repair• Axiom Ranking Module

– Ranks erroneous axioms based on various metrics• Automatically:

– Arity – no. of errors (contradictions) caused» Identify common error producing axioms

– Impact – no. of entailments lost upon removal» Greater the impact, higher the axiom rank

– Usage – structural connected-ness of axioms» Greater the usage of entities in the axiom, higher the axiom

rank

• Manually:– Entailment Tests – specified by the user

» User can specify desired or undesired entailments

04/10/23 42


Ontology Repair• Solution Generation Module

– Use Uniform Cost Search (cost=rank)– Solution can be at different granularity levels:

• Single Unsatisfiable Concept at a time• All Root Unsatisfiable Concepts

– Use Root Error Pinpointing to extract roots

• All Unsatisfiable Concepts– For improved performance: iteratively repair roots only

– Smaller the granularity level, slower the process, though easier to monitor changes

04/10/23 43


Ontology Repair• Axiom Rewrite Module

– Suggest rewrites by looking at precise justifications• Weaken axiom by removing relevant parts of axioms only

– Maintain a library of commonly occurring error patterns (e.g., motivated by OWL-Pizzas paper, Rector et al, 2004)

• e.g., accidental use of equivalence instead of subclass

– If rewrite is not a strict weakening, need to verify that no new undesired entailments (e.g., unsatisfiable concepts) are introduced

For details see: Kalyanpur et al “Repairing Unsatisfiable Concepts in OWL Ontologies“, European Semantic Web Conference (ESWC) 2006, (Won Best Paper Award)

04/10/23 44


Ontology Repair

Repair Solutions:- Rewrites integrated- Ability to change plans on the fly- Preview option

Analyze Erroneous Axioms

04/10/23 45


User Evaluation

Key Points:- Using repair service, time drops by a factor of 3-8 * 2-tailed T-test shows G2 significantly better than G1 for p < 0.05-Qualitative feedback: * Subjects satisfied with solutions reached (and alternatives available) * Subjects appreciated impact analysis and axiom rewrites

n=12 subjects

04/10/23 46


Conclusion• Revisit Goal: Build debugging services to

– Explain the cause of the contradiction! Axiom Pinpointing

– Pinpoint the key problematic components ! Root Error Pinpointing

– Suggest remedies ! Ontology Repair

+ Good Debugging Support useful to understand OWL-DL Ontologies

04/10/23 47


Future Work• Exploring Extension to other Logics:

Debugging Inconsistent FOL KBs– Glass Box: Tracing through Semantic Tableaux /

Resolution proof procedures in FOL– Black Box: Directly applicable for debugging FOL

KBs’: need to evaluate and optimize

• Beyond Debugging: Explore similar techniques for other Ontology Eng. tasks– Reasoning over Dynamic (Changing) Ontologies

04/10/23 48


Contributions1. Defined and Formalized three novel OWL Debugging

and Repair services for semantic defects in OWL• More generally, services applicable for SHOIN(D) KBs

2. Designed and Analyzed decision procedures for implementing each of the services

3. Built a widely used OWL Ontology Editor, Swoop, and implemented services in Swoop, and in a reasoner, Pellet

4. Conducted Performance and Usability Evaluations on realistic OWL Ontologies Demonstrated that services can be used in an interactive ontology engineering environment and significantly help OWL Ontology diagnosis and repair

04/10/23 49


Questions• Thanks!


Extra Slides

04/10/23 51


Logical Inconsistencies• Example: Consider a Pizza Ontology O

containing classes Meat, Cheese, and property topping, s.t.:

• Meat v :Cheese (disjoint)• MeatCheesyPizza v 9topping (Meat u Cheese)

– The defn. of class MeatCheesyPizza contains a contradiction– We say that MeatCheesyPizza is unsatisfiable

• i.e., it represents the empty set

• Now if we add the assertion: “X is an instance of MeatCheesyPizza” in O,– Then O becomes inconsistent

Back

04/10/23 52


Tackling the Debugging Problem• Preliminary Swoop functionality:

– Hypertextual Navigation– Visual Cues (Highlight unsatisfiable class expressions,

e.g., 9R.C where C is unsatisfiable)– Query Support (“Show References”)– NL Explanation of Concept Definitions– Flexible Change Management (Checkpoints)– Collaborative Annotation and Discussion

• Main Drawback: Manual Process• There is a need for automated debugging support

04/10/23 53


Glass Box• DL reasoners typically use Tableau Algorithms:

– Proof By Refutation• To prove O ² E, we check consistency of O [ {:E}

– Decide consistency of an ontology by trying to construct a model for it: a completion graph

• Each node represents an individual and is labeled with a set of concepts

• Each edge is labeled with a set of role names• Graph constructed by repeatedly applying a set of expansion rules

– e.g., if (C u D) 2 L(x), we apply u rule and add C, D separately to label

• Ontology is inconsistent if every branch has a clash (contradiction)– e.g., if (C, :C) are both present in label of same node

• Basic idea to find justification: Tableau Tracing– Track the set of axioms responsible for each update to

completion graph– Identify clash-causing axioms

More

04/10/23 54


Tableau Tracing• Example to capture Justification1. MeatPizza v Pizza u 9hasTopping.Meat

2. range(hasTopping, Food)

3. Pizza v 8hasTopping.Cheese

4. Meat v Food u :Vegetable

5. Cheese v :Meat

MeatPizza, (Pizza u 9hasTopping.Meat)1MeatPizzaMeatPizza, Pizza1, 9hasTopping.Meat1MeatPizza, Pizza1, 9hasTopping.Meat1, 8hasTopping.Cheese1,3

hasTopping1

Meat1

Clash! Justification = {1} + {1,3,5} =

{1,3,5}

Meat1, Food1,2, :Vegetable1,4Meat1, Food1,2, :Vegetable1,4, Cheese1,3, :Meat1,3,5Meat1, Food1,2, :Vegetable1,4, Cheese1,3Meat1, Food1,2

Back

04/10/23 55


• Key Challenges– Internal modifications done by the reasoner

• Normalization – involves rewriting terms– :(9P.C) ! 8P.:C

• Absorption - involves combining axioms– C u D v E, C v F ! C v F u (E t :D)

– More than one reason for an event• e.g. 1: C v 9R.(D u E), 2: C v 8R.E, 3: D v :E

Tableau Tracing

C x (D u E)1yR1 Solution: Modify trace to be a set of axiom sets

(D u E){{1}}, E{{1,2},{1}}, D{{1}}, :E{{1,3}}

Solution: Trace through preprocessing

, E1,2 ,D1, :E1,3

04/10/23 56


• Additional Challenges– Determining if a clash depends on a choice (/merge)

e.g. 1: C v 9R.E, 2: C v 9R.D, 3: C v ·1.R, 4: C v 8R.:D

– Determining which axioms are responsible for the choice (esp. due to merges)

e.g. 1: C v 9R.D, 2: C v 9R.E, 3: C v 9R.:D, 4: C v ·1.R

Tableau Tracing

RRR

C

D1 E2

:D3x y z

wSolution: Record choice context (cardinality, edges)

C yR1x y E1x y

Solution: Order rules (delay choices); insert choice points

For details see: Kalyanpur et al “Axiom Pinpointing" Submitted to ODBASE, 2006

z D2

, D1,2,3 , :D1,4

Back

04/10/23 57


Black Box Justifications

04/10/23 58


HST Optimizations• Incrementally update ontology in every

step of HST algorithm– Track set of axioms responsible for each

component of completion graph– When removing an axiom from O, remove

only parts of graph “introduced” by axiom• Instead of generating graph from scratch again

– Re-apply expansion rules on updated graph

Back

04/10/23 59


Precise Justification Issue• From user perspective: Need to maintain

correspondence with original asserted terms– Replace new / split concepts by original counterparts

when presenting output• In some cases, replace new concept introduced by >

• Example:– Original KB K: { A v 9R. (C u D), A v 8R. (D u :D) }

– Split KB K’: { A v 9R.E, E v C, E v D,…A v 8R. :D } (new concept E)

– Justification for A v ? in K’: {A v 9R.E, A v 8R.D, A v 8R.:D}.

– Final Precise Justification Presented: { A v 9R.>, A v 8R. (D u :D) }

Here, the new concept E is replaced by > since it appears

only once in the justification

Two conjuncts have been recombined in the output

04/10/23 60


Structural Tracing• Root/Derived Detection Examples:

– A v C u D (D is unsat.)– A v (C t D) (Both C, D unsat)– A v 9R. C (C is unsat)– A v 9R.C (domain(R) unsat)– A v ¸1.R u 8R.C (C is unsat)– A v 9R-.C (range(R) is unsat.)– A = B , B v C u :C (Inferred)

Back

04/10/23 61


Inferred Dependency• Subsumption-safe transformations

– Approximate KB by getting rid of unsatisfiability while preserving subsumption

e.g. A = 9P. D

B = 9P. E u C u :C D v E

Replace :C by new C’, then discover A v B– Sound but not complete w.r.t subsumption

relation revealed Back

04/10/23 62


Ontology Repair UI

Detailed Preview of the Repair Solution

04/10/23 63


Sample OWL Data

Back

04/10/23 64


User Evaluation (Setup)• Selected 12 subjects with Computer Science background and

– one year experience in OWL – understanding of DL’s that varied greatly (novices to experts)

• Each subject recd. 20-30 min orientation of SWOOP debugging features (training w/ sample ontology)

• Selected 2 OWL Ontologies: University.owl and miniTambis.owl– University.owl – known domain– miniTambis.owl – foreign domain (chemical)

• Asked each subject to fix all errors in one ontology using only Debugging Services (Axiom Pinpointing + Root Error Pinpointing) and in other using the Ontology Repair service

• Subjects were randomly assigned to each ontology– Equally distributed to two groups (six per group)

• Goal to compare performance and quality improvement over previous debugging services– Noted time / steps taken for repair and received qualitative feedback

04/10/23 65


Future Work• Explore axiom rewrite strategies based on

recently proposed DL update semantics– Example: given unsatisfiable concept Koala

• Koala v 9isHardWorking.false• domain (isHardWorking, Person)• Koala v Marsupials• Marsupials v :Person

– One possible rewrite that resolves error is:• domain(isHardWorking, (Person t Marsupials))

04/10/23 66


Related Work: Diagnosing Reasoning Systems

• Debugging Logic Programs (Prolog, Datalog)– Operational debugging

• Instrument program and examine trace (break points)

– Declarative debugging• Navigate solution in a top-down manner by posing queries to an

Oracle

• Rule-Based Debuggers in Expert Systems (MYCIN, ESS)– Generate rule model and test for conflicts, redundancy, missing

conditions– Use machine learning to detect faulty clauses in rules

• Repairing ICs in Deductive Databases• Proof Explanation in ATPs

04/10/23 67


Links• Swoop w/ Pellet

– Swoop: OWL Ontology Engineering Toolkithttp://www.mindswap.org/2004/SWOOP

– Pellet: OWL-DL Reasonerhttp://www.mindswap.org/2003/pellet

More on Debugging:http://www.mindswap.org/2005/debugging

Documents

mindswap maryland information and network dynamics lab