View
43
Download
6
Category
Tags:
Preview:
DESCRIPTION
IUI 2004, Madeira, Portugal – Wed Jan 14, 2004. Designing Example Critiquing Interaction. Boi Faltings Pearl Pu Marc Torrens Paolo Viappiani. LIA. HCI. Outline. Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion. Motivation. - PowerPoint PPT Presentation
Citation preview
Designing Example Critiquing Interaction
Boi FaltingsPearl PuMarc TorrensPaolo Viappiani
IUI 2004, Madeira, Portugal – Wed Jan 14, 2004
LIA HCI
Wed Jan 14, 2004 Designing Example Critiquing Interaction 2
Outline
Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion
Wed Jan 14, 2004 Designing Example Critiquing Interaction 3
Motivation
Many real word applications require people to select a most preferred outcome from a large set of possibilities (electronic catalogs)
Users are usually unable to correctly state their preferences up front
People are greatly helped by seeing examples of actual solutions example critiquing
Wed Jan 14, 2004 Designing Example Critiquing Interaction 4
Mixed Initiative Interaction
initial preference
the system shows K solutions
The user critiques the solutions stating a new preference
The user picks the final choice
Wed Jan 14, 2004 Designing Example Critiquing Interaction 5
An implementation: reality
user critiques existing solutions
trade-off between different criteria
Wed Jan 14, 2004 Designing Example Critiquing Interaction 6
What to show?
Standard approach show the best solutions assumption: user model is complete and
accurate Does not in general stimulate new
preferences
Wed Jan 14, 2004 Designing Example Critiquing Interaction 7
New approach
Display_set = stimulate_set + optimal_set
Wed Jan 14, 2004 Designing Example Critiquing Interaction 8
What to show?
Stimulate set = solutions that make the user aware of attributes
diversity have high probability to become optimal if
new preferences are stated Optimal set = solutions that
are optimal given the current preferences
Wed Jan 14, 2004 Designing Example Critiquing Interaction 9
Outline
Introduction Stimulating expression of
preferences Guaranteeing optimal solutions Conclusion
Wed Jan 14, 2004 Designing Example Critiquing Interaction 10
Stimulating new preferences
Pareto optimality general concept does not involve weights
Dominated solution can become Pareto optimal if new preferences are stated show solutions that have higher
probability of becoming Pareto optimal
Wed Jan 14, 2004 Designing Example Critiquing Interaction 11
Dominance relation and Pareto optimality
P1 P2S1 0.3 0.4S2 0.4 0.2S3 0.6 0.5S4 0.3 0.9S5 0.9 0.7
3
6
3
9
6 9
Penalty table, 2 preferences
s1
s2
s3
s4
s5
S1 and S2 are Pareto optimal
S3 is dominated by S1 and S2
S4 is dominated by S1
S5 is dominated by S1, S2, S3.P1
P2
Wed Jan 14, 2004 Designing Example Critiquing Interaction 12
Pareto Optimal Filters
Estimate the probability that a dominated solution can become Pareto optimal when new preferences are stated
Different Pareto-filters: counting filter attribute filter probabilistic filter
Wed Jan 14, 2004 Designing Example Critiquing Interaction 13
Counting filter
We count the number of dominators
S1 and S2 are currently “optimal”
S4 more promising than S3 and S5
P1 P2 FilterS1 0.3 0.4 0S2 0.4 0.2 0S3 0.6 0.5 2S4 0.3 0.9 1S5 0.9 0.7 3
Counting Filter: number of Dominators
Wed Jan 14, 2004 Designing Example Critiquing Interaction 14
A new preference is added
New column with penalties S4 becomes Pareto optimal even if the new penalty
(0.6) is worse than for S3 (0.5) and S5 (0.4)
P1 P2 P3 Pareto Optimal
S1 0.3 0.4 0.7 YesS2 0.4 0.2 0.3 YesS3 0.6 0.5 0.5 NoS4 0.3 0.9 0.6 YesS5 0.9 0.7 0.4 No
The counting filter predict that S4 has better chances to become P.O. when a new preference is added.
Wed Jan 14, 2004 Designing Example Critiquing Interaction 15
Hasse diagrams
1 2
34
5
User Model={p1,p2}
Pareto Optimal
User Model={p1,p2, p3}
1 2
3
4
5
Adding preferences: Pareto optimal set grows, dominance relation becomes sparse
Wed Jan 14, 2004 Designing Example Critiquing Interaction 16
Attribute filter
Solution: n attributes: A1,..,An
D1,..,Dn domains for A1,..,An
a solution is a complete assignment Preferences modeled as penalty functions
defined on attribute domains Look at the attribute space
if two values are the same, any penalty function defined on these values will be the same
Wed Jan 14, 2004 Designing Example Critiquing Interaction 17
Price M2 Location TransportS1 650 28 West BusS2 700 24 North TramwayS3 700 24 West Bus
Attribute filter: motivation
S2 and S3 are both dominated by S1
If we add new preference on Location if North is preferred S2 will be Pareto
Optimal on Transport if Tramway is preferred to Bus then S2
will be P.O. S3 will always be dominated!!
Preferences: on price (to minimize), on M2 (to maximize)
Wed Jan 14, 2004 Designing Example Critiquing Interaction 18
Attribute Filter
For the new preference, dominated solution s must have lower penalty than all dominant solutions for discrete domain, attribute values must
be different for continuous domains, consider extreme
values
Wed Jan 14, 2004 Designing Example Critiquing Interaction 19
Probabilistic filter Directly estimate probability of becoming P.O. The bigger the difference on a specific attribute, the
more likely the penalties will be different
1 1
domain
pe
na
lty
pe
na
lty
domain
Wed Jan 14, 2004 Designing Example Critiquing Interaction 20
Experiments
Database of actual accommodation offers (room for rent, studios, apartments)
Random datasets 11 attributes of which
4 continuous (price, duration, square meters, distance to university)
7 discrete (kitchen, kitchen type, bathroom, public transportation, ..)
Wed Jan 14, 2004 Designing Example Critiquing Interaction 21
Results (accommodation dataset)
Wed Jan 14, 2004 Designing Example Critiquing Interaction 22
Results (random dataset)
0
0.1
0.2
0.3
0.4
0.5
0.6
1 2 3 4 5 6
CountProbAttRandom
Ave
rage
fra
ctio
n o
f co
rrec
t p
red
icti
ons
number of preferences known
Wed Jan 14, 2004 Designing Example Critiquing Interaction 23
Outline
Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion
Wed Jan 14, 2004 Designing Example Critiquing Interaction 24
Modelling True preference model P* (unknown)
P*={p*1, p*
2, .., p*
k}
st: target solution Estimated through a model P
P={p1,..,pk} pi are built-in standard penalty functions assume limited difference between p and p*
Penalty functions p
i(a
k): d
k -> R
write pi(s) instead of p
i(a
j(s))
Wed Jan 14, 2004 Designing Example Critiquing Interaction 25
Selecting displayed solutions
Dominance filters Utilitarian filters Egalitarian filters
Wed Jan 14, 2004 Designing Example Critiquing Interaction 26
Optimal Set Filters Properties
We want..
1. To show a limited number of solutions each filter selects k solutions to display
2. To ensure that a Pareto-optimal solution in D is Pareto-optimal in S
each filter satisfies this dominance filter (by definition), Utilitarian and
Egalitarian (theorem)
Wed Jan 14, 2004 Designing Example Critiquing Interaction 27
Optimal Set Filters Properties
3. To include target solutions only if target solution is included the user can
choose it! probability to include the target solution in D
depends on filter.
Assumption (1-ε)pi ≤ pi* ≤ (1+ε)pi
Wed Jan 14, 2004 Designing Example Critiquing Interaction 28
Dominance filter
Display k solutions that are not dominated by another one
Wed Jan 14, 2004 Designing Example Critiquing Interaction 29
Dominance filter: target solution
Plot of probability of target solution being included in D, as function of number of preferences |P|=3,..,12 K=30, 60 m=778, 6444
Wed Jan 14, 2004 Designing Example Critiquing Interaction 30
Utilitarian filter
We minimize the un-weighted sum of penalties
Efficiently computed by “branch & bound”
Pp
i
i
spsC )()(
Wed Jan 14, 2004 Designing Example Critiquing Interaction 31
Utilitarian filter: probability to find target solution
Does not depend on m, the number of total solutions (proved analytically)
Better than dominator filter
Wed Jan 14, 2004 Designing Example Critiquing Interaction 32
Egalitarian filter
Minimize F(s) In case of equality, use lexicographic order:(0.4, 0.2) preferred to (0.4, 0.4)
Target solution inclusion probability similar to that of the Utilitarian filter.
)(max)( spsF iPpi
Wed Jan 14, 2004 Designing Example Critiquing Interaction 33
Robustness against violated assumption
Fraction of PO solutions shown within the best k ones
Egalitarian, utilitarian filter
Wed Jan 14, 2004 Designing Example Critiquing Interaction 34
Outline
Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion
Wed Jan 14, 2004 Designing Example Critiquing Interaction 35
Conclusion
Optimal and stimulation set Example critiquing on firmer
mathematical ground Suggestions to system developer How to compensate an
incomplete/inaccurate user model Experimental evaluation on real and
random problems
Wed Jan 14, 2004 Designing Example Critiquing Interaction 36
Questions
Wed Jan 14, 2004 Designing Example Critiquing Interaction 37
Wed Jan 14, 2004 Designing Example Critiquing Interaction 38
Counting filter works already fairly well Attribute filter works very well when only
1 or 2 preferences are missing, but generally probabilistic is the best
Impact of correlation between attributes can affect performance
Pareto Filters: conclusions
Complexity
Counting Attribute ProbabilisticRandom
Wed Jan 14, 2004 Designing Example Critiquing Interaction 39
Attribute filter/2
Continuous domains: best values are the extremes
assumption: preference functions are monotonic
1OR
domain
pe
na
lty
Wed Jan 14, 2004 Designing Example Critiquing Interaction 40
1
θθ
1
domain domain
ai(o1) ai(o1)ai(o2) ai(o2)
1
θ
domain
ai(o1) ai(o2)
m
domain
ai(o1) ai(o2)
Wed Jan 14, 2004 Designing Example Critiquing Interaction 41
θθ
1
domain domainai(o)
gilisi
1
θdomain
ai(o) li
domain
ai(o) li
1
ai(o1)-t θ-t
m1
m2
ai(o)θ+t
Wed Jan 14, 2004 Designing Example Critiquing Interaction 42
Wed Jan 14, 2004 Designing Example Critiquing Interaction 43
Theorem
Given a set of m solutions S={s1,..,sm} and a set of penalties {p1,..,pd}
Let S’ be the best k solutions according to the utilitarian filter
A solution s in S’ not dominated by any other of S’, is Pareto Optimal in S.
Wed Jan 14, 2004 Designing Example Critiquing Interaction 44
Simplified Apartment Domain
A very simple example: A={Location, Rent, Rooms} DLocation={Centre, North, South, East, West}
DRent={x|x integer x>0}
DRooms={1,2,3..}
Preferences: location should be centre and rent less than 500
Wed Jan 14, 2004 Designing Example Critiquing Interaction 45
Penalty functions
P1:= if (Location==centre) then 0 else 1
P2:= If (Rent > 500) then K*(Rent-500) Else 0
Wed Jan 14, 2004 Designing Example Critiquing Interaction 46
Electronic catalogues
K attributes: A1,..,Ak
D1,..,Dk domains for A1,..,Ak
a solution is a complete assignment write aj(s), value of S for attribute j
Solution set S is a subset of D1 x D2 x D3 x D4 x ...
Preferences modeled as penalty functions defined on attribute domains
Wed Jan 14, 2004 Designing Example Critiquing Interaction 47
Counting filter*The Dominator set for a solution s
1, is the subset of S of
solution that dominates s1.
The counting filter orders
solutions on the size of
the dominator set.S
d(s
1)
s1
Wed Jan 14, 2004 Designing Example Critiquing Interaction 48
Probabilistic filter Directly estimate probability of becoming P.O. The bigger the difference on a specific attribute, the
more likely the penalties will be different
1
domain
pe
na
lty
Recommended