Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
1
Robust Knowledge Representation
Frank van HarmelenAI Department
Vrije Universiteit Amsterdam
Better half an answer in timethan a full answer too late
2
What is science about?
Science is a method
for exploring uncertainty;
It delivers better models,
not revealed truth
3
a m F ⋅=
Fm
v ca=
−
⋅1 2 2/
Science =making models
4
KR makes models of what?n Representation: Structure of knowledge§ Symbolic representation of knowledge
n Inference: Patterns of reasoning§ Deriving new information from existing§ algorithms, implementations
n Examples:§ Traditional First Order Logic: Truth
§ Modalities: Knowledge, Belief
§ Non-monotonic reasoning: reasoning with exceptions§ etc.
5
KR models are based on logic
n Reasoner makes no mistakes(sound & complete)
n Reasoner has unlimited resourcesn All knowledge is availablen All knowledge is correct
An ideal reasoner under ideal circumstances
6
KR models are based on logic
Reliance on logic is a weaknessn Crisp (no approximate answers)n Abrupt (no intermediate answers)n Inefficient (no time/quality trade-off)
Reliance on logic is a strengthn Strong theoretical basisn Well known propertiesn Well known implementation techniques
7
Desiderata for Robust Knowledge Representation
Q↑↑
T→→
Instead, we would want:n Approximate answersn Incremental computationn Anytime cost/quality trade-off
Q↑↑
T→→
Reliance on logic is a weaknessn Crispn Abrupt n Inefficient
8
Can this be done in logic ?n YES!!:
I. Approximate deduction in diagnosisII. Qualitative performance profilesIII. Empirical performance profiles
n Don’t abandon logic:§ Neural Networks§ Genetic Algorithms§ Statistical models
9
Approximate Deduction:Intuition
n Turn the knob on the reasoning engine• exchange precision for cost• anytime reasoning = turn the knob gradually• characterise the effect of the approximationcan we be precise about the imprecision ?
10
Part I: Approximate Deduction…
n not yes/no answers, butn optimise a quality measuren NB: not necessarily numeric
WHAT
n AI problems are intractablen often approximate solutions sufficen anytime behaviour
WHY
n define reasoning method using `n replace ` by approximate deduction
HOW
11
… in diagnosis
nDealing with § “no diagnosis”, § “too many diagnoses”
nSometimes not interested in exact diagnosis (e.g. safe over-diagnosis)nPrefer cheap approximation over expensive
exact solution (time-pressure)nAnytime algorithms
12
1,3-S (Cadoli & Schaerf)n S = set of propositional lettersn classical inference on letters in Sn 1-S: unsound on letters outside Sn 3-S: incomplete on letters outside S
13
Intuitions for clausal form
14
Main result of Cadoli/Schaerf
efficient incremental anytime algorithms:cost of iterated computation is never higher than computing `̀2 once!
Notice: approximate, incremental, anytime
15
Definition of diagnosisnGiven: § Behaviour model BM§ Observations O
nFind§ Explanation E
nSuch that:
nReplace `̀ by `̀S1,3
16
Main results
n ABD1S diagnoses are contained in classical diagnoses
n ABD3S diagnoses contain classical diagnoses
nWhen S grows nABD1
S no new subdiagnosesn ABD3
S no new superdiagnoses
17
Intuition ABD1SABD3S
18
Strategies for choosing Sn ABD1
S = all urgent subsets of classical diagnosesn ABD3
S = all classical diagnoses that are entirely urgent
n Increase S with less urgent causes,interrupt when§ No time left:
only non-urgent diagnoses lost§ First diagnosis found:
most urgent diagnosis
19
Part II: Qualitative Performance profiles• Output-quality is function of some
varying resource– reasoning time, – inference accuracy, – representational precision
• This function is (ideally)– monotonic– diminishing returns– characterised by a performance profile
20
Classification by linear candidate confirmation1. Iterate over all classes2. Check every class with the observations;
(leading to confirmation or not)
|Cs|
recall
time
21
Classification by confirmation with filtering1. Filter the classes,
based on a subset of the observations2. Iterate over all classes3. Check every class with the observations;
(leading to confirmation or not)
filter
recall
time
22
Hierarchical classification1. First consider all classes as solutions2. Descend a classification hierarchy (depth d),
eliminating all classes on entire branches
precision
timed
23
Design byConstraint clusteringGroup constraints in non-interacting clusters1. Iterate over all k clusters2. Find an assignment per cluster
k
assignments
time
24
Design by Propose & RevisenAssign successive parametersnTest partial designsnRe-assign earlier parameters if needed
|Pars|
parameters
time
25
SummarisingnMany inference methods have
surprisingly natural anytime behaviour
nProblem:§ Only upper/lower bound,
but no quantitative measures§ Do search methods really behave like this
in practice?
26
Part III: Quantitative Performance Profiles
nMeasure quantitative profiles
nHow does quality of output change asfunction of § quality of input ?
§ quality of knowledge base ?
27
Experimental settingn Vegetation classification system§ 93 plant names§ 40 observables (max. 30 per case)§ 7586 rules§ 150 test cases
n Use recall and precision as quality measuresn Incomplete inputn Incomplete knowledge basen Incorrect knowledge base
28
Experimental results (1)
Recall: precision:Incomplete input:
29
Experimental results (2)
Recall with different input orderings:Incomplete input:
30
Experimental results (3)
(with realistic removal model)Incomplete knowledge base:
31
n Reasoner makes no mistakes(sound & complete)
n Reasoner has unlimited resources
n All knowledge is available & correct
Can this be done in logic ?
n Reasoner makes no mistakes (sound & complete)è Cadoli & Schaerf (part I)
n Reasoner has unlimited resourcesè Qualitative performance profiles (part II)
n All knowledge is available & correctè Quantitative performance profiles (part III)
An ideal reasoner under ideal circumstances ?
32
Research agenda
n Other approximate deduction relations?n Exploit other methods:§ Knowledge compilation (Kautz & Selman)§ Language weakening
n Relations between these?n New application areas:§ Semantic Web
(approximate Description Logics)§ Agent communication
(approximate terminology mappings)§ Software retrieval
(approximate pre/post-conditions, Web services)