Arguments From Experience: The PADUA Protocol

Maya Wardeh, Trevor Bench-Capon and Frans Coenen

Department of Computer ScienceUniversity of Liverpool

A Strange Beast• It is warm blooded, has hair, lays eggs, does not suckle

its young. Is it a mammal?

– Albert: It can’ be a mammal because it lays eggs– Bruce: I’ve seen mammals that lay eggs. And it does have

hair– Albert: But mammals with hair also suckle their young– Bruce: All hairy, warm blooded animals are mammals– Albert: So I suppose we must say it is

• Albert and Bruce argue on the basis of mammals they have seen

• Their experiences have been very different• There may be no right answer – yet!

PADUA

• PADUA – Protocol for Argumentation Dialogue Using Association Rules

• A dialogue game to argue about classification

• Arguments are taken directly from a data base of past examples using data mining techniques

Arguing from Experience

• Most dialogue systems are based on belief bases– Participants use facts and rules to construct

their arguments

• PADUA uses examples directly– Participants have a data base containing

collections of instances representing their past experience

• Resembles case based rather than rule based reasoning

Experiences May Differ

• Participants will have seen different samples.– Geographical: all swans are white in

the Northern hemisphere– Exceptions may be only rarely

encountered: insufficient support in some DBs

– Sample may be abnormal: in law, only hard cases seen at highest level of appeal

Advantages

• No knowledge representation bottleneck

• No advance commitment to a theory

• Can deal with gaps and conflicts• Pools experience

Dialogue Moves

• Participants point to features of the current case which are reasons why it should (or should not) be classified in a certain way

• Participants respond by citing other features which provide reasons to challenge the classification

Moves Based On Belief Bases• Claim P: P is the head of some rule• Why P: Seeks the body of rule for

which P is head• Concede P: agrees that P is true• Retract P: denies that P is true• P since S: A rule with P head and S

body

Persuasion in Belief Base Systems

• A Has a Rule with P as head, but one literal Q in the body is unknown: B shows that Q is true.

• B gives A a rule with P as head and body S. A already believes S

• A is shown to have an inconsistency: retraction enables P to be shown

Moves in Case Based Reasoning

• Citing a case: – A past case which shares features with the current case

and had the desired outcome

• Distinguishing a case: – features in the past case missing from the current case– Features in the current case missing from the past case

• Counter Example– A past case which shares features but had a different

outcome

• Arguments from Experience have many similarities

Moves in Argument from Experience• Citing reasons:

– Features in the current case which are typically associated with the desired classification, C

• Distinguishing– An additional feature which typically identifies an

exception– A feature which Cs typically have but which is not

present– A feature which increases confidence in the classification

• Counter Example– Features in the current case which are typically

associated with a different classification, not C

PADUA Scenario

A1 A2

A1: P suggests Q

A2: P’ suggests Q’

Instance Case

Class C1 Class C2 (C1)

PADUA Protocol - Basics

PADUA Protocol - BasicsPADUA Scenario

PADUA Moves1

6 2

3

4

5

1: Propose Rule

2: Distinguish

3: unwanted consequences

4: Counter Rule

5: Increase Confidence

6: Withdraw unwanted consequences

PADUA Protocol

P is a reason for C

it would be more a C if it were RCs are not Q

It need not be S

P’’ is a reason for not C

P and q is a reason for not C

Strategies

• The protocol offers a lot of scope for choice:

• Which move to make? – Introduce a new association or refine

an existing one?

• Which association to propose? – Best or just a good one? In terms of

confidence or support?

Strategies

• We have different strategies according to:– Aim: establish a rule or critique opponents

(build versus critique)– Persistence: concede when reasonable or only

when no argument left (agreeable versus disagreeable)

• Different strategies give rise to different flavours of dialogue:– Build + Disagreeable more like persuasion– Critique + Agreeable more like deliberation

Experiments

• We have experimented with a number of Data Sets:

• Poisonous Fungi• US Senators voting records

(ESQUARU 2007)• Welfare Benefits (COMMA 2008)

Welfare benefits

• Large numbers of cases • Lay adjudication: many (often

inexperienced) adjudicators• A high (often 20-30%) error rate is typical• Particular clerks and offices may make

systematic misinterpretations• PADUA can act as a moderation meeting,

allowing debate over classifications drawn from different adjudication sources

Conditions for Benefit

• Age condition: “Age appropriate to retirement” is interpreted as pensionable age: 60+ for women and 65+ for men.

• Income condition:“Available income” is interpreted as net disposable income, rather than gross income, and means that housing costs should exceed one fifth of candidates’ available income to qualify for the benefit.

• Capital condition: “Capital is inadequate” is interpreted as below the threshold for another benefit.

• Residence condition: “Resident in this country” is interpreted as having a UK address.

• Residence exception: “Service to the Nation” is interpreted as a member of the armed forces.

• Contribution condition: “Established connection with the UK labour force" is interpreted as having paid contributions in 3 of the last 5 years.

Results

• Given two databases, each containing a significant proportion of wrongly decided cases based on different systematic errors, correct classifications can be reached

• While this is true for errors concerning most attributes, success is markedly less when mistakes relate to the contribution condition

Intermediate Predicates

• These are legal concepts, which must be satisfied for the law to apply

• Need to be defined in terms of observable facts• Some can be unfolded into observable facts• Others need to applied on the balance of

consideration of a number of factors• These last present particular problems

– E.g. the contribution condition in the example– Also true for other machine learning and data mining

systems (e.g. Mozina et al)

Nesting Dialogues

• If we are aware of intermediate predicates which do not unfold appropriately into sufficient conditions, we can nest a dialogue to decide this issue within the main dialogue

• This handles the contribution problem• Confirms other work in AI and Law in

which issue based classification is more accurate than holistic approaches

Summary

• PADUA offers a novel kind of persuasion dialogue, based on examples rather than a belief base. The result has more in common with case based than rule based reasoning

• It avoids the need for knowledge representation effort

• The databases are not shared, enabling distinctive features of particular DBs to be identified and maintaining some level of privacy

• Where issues can be identified and resolved in preliminary dialogues, accuracy can be improved

The Talk Is Finished