Embed Size (px)
DESCRIPTION
market based recommender system
Citation preview
B R A C H A S H A P I R A
B S H A P I R A @ B G U . A C . I L
B E N - G U R I O N U N I V E R S I T Y
Search Engines Personalization
Personalization
“Personalization is the ability to provide content and services tailored to individuals based on knowledge about their preferences and behavior” [Paul Hagen, Forrester Research, 1999];
Acceptance of Personalization
Overall, the survey finds that interest in personalization continues to be strong with 78% of consumers expressing an interest in receiving some form of personalized product or content recommendations.
ChoiceStream Research
Motivation for Search Engine Personalization
Trying to respond to the user needs rather than to her query
Improve ranking tailored to user’s specific needs
Resolve ambiguities
Mobile devices – smaller space for results – relevance is crucial
Search Engines Recommender Systems - Two sides of the same coin????
Search Engines
Goal – answer users ad hoc queries
Input – user ad-hoc need defined as a query
Output- ranked items relevant to user need (based on her preferences???)
Recommender Systems
Goal – recommend services of items to user
Input - user preferences defined as a profile
Output - ranked items based on her preferences
Search Engines Personalization Methods adopted from recommender systems
Collaborative filtering User-based - Cross domain collaborative filtering is required???
Content-based Search history – quality of results????
Collaborative content-based Collaborate on similar queries
Context-based Little research – difficult to evaluate
Locality, language, calendar
Social-based Friends I trust relating to the query domain
Notion of trust, expertise
Marcol- a collaborative search engine Bracha Shapira, Dan Melamed, Yuval Elovici
Based on collaborations on queries
Documents found relevant by users on similar queries are suggested to the current query
An economic model is integrated to motivate users to provide judgments.
MarCol Research Methods System Architecture
MarCol Example
MarCol Example
MarCol Example
MarCol Example
MarCol Example Ranking reward: up to 3
MarCol Ranking Algorithm • Step 1: Locate the set of queries most similar to the current user
query.
Where:
– a (“short”) query submitted by a user u
– the set of all (“long”) queries
),( iu LqSqSim – the cosine similarity between and QLqi
1t – a configurable similarity threshold
1),(' tLqSqSimQQ iu
uSq
uSq
},...,,{ 21 nLqLqLqQ
MarCol Ranking Algorithm • Step 2: Identifying the set of most relevant documents to the
current user's query.
Where:
– the set of all documents that have been ranked relevant to
queries in
– a configurable similarity threshold
2),()'()'(' tdSqSimQDQD iu
)'(QD
'Q
)'(QDdi
2t
MarCol Ranking Algorithm
),( iu dSqSim
• Step 3: Ranking the retrieved documents according to their
relevance to the user query.
The relevance of document to query :
Where:
– the average relevance judgment assigned to the set of the documents
– similarity between user query and the document.
)'(' QDdi
),( ou qSqSim – similarity between user query and documents’ query . )'( Qqo
),( oi qdJ
id for the query (measured in a 1..5 scale). oq
uSq
),(),(),(),( oiouiuiu qdJqSqSimdSqSimdSqrel
Experiment Results – first experiment Satisfaction
• There is not a significant difference between the modes
(p=0.822535) for a 99% confidence interval.
4.00
3.74
4.24
4.43
3.47
3.78
4.32
3.95
4.19
3.88
3.663.66
3.40
3.60
3.80
4.00
4.20
4.40
4.60
1 2 3 4 5 6
Sub-Stage
Sa
tis
fac
tio
n
MarCol Free MarCol
The properties of a pricing model
• Cost is allocated for the use of evaluation, and users are
compensated for providing evaluations.
• The number of uses of a recommendation does not affect its cost
(based on Avery et al. 1999). That value is expressed by the relevance of
a document to users query and the number of evaluations
provided for that document representing the credibility of
calculated relevance.
• Voluntary participation (based on Avery et al. 1999). The user decides
whether he wants to provide evaluations.
• The economic model favors early or initial evaluations.
Therefore, a lower price is allocated for early and initial
evaluations than for later ones and a higher reward is given for
provision of initial and early evaluations than for later ones.
Cost of document Calculation
• An item that has more evaluations has a higher price (until
reaching upper limit).
• An item that has few recommendations offers a higher reward for
evaluation.
• The price of an information item is relative to its relevance to the
current users query.
• The price is not affected by the number of information uses.
Document Cost Calculation for a query – the price of document id uq
Where:
– the number of judgments
– upper bound
( , ) min( , )( , )
5
u iu i
rel q dPay q d
),( iu dqPay
Reward Calculation reward – is the amount of MarCol points that a user is awarded for providing
an evaluation for document id uq
),( iu dq
that was retrieved for query
)1,min(
5
),(),(
iu
iu
dqreldqReward
Where:
– the number of judgments
– upper bound
Experiment Methods
Independent variable:
• The only variable manipulated in the experiment is an
existence of the economic model.
Mode Short description
With economic
model
Users should pay “MarCol points” to
access a document suggested by the
system. While submitting a judgment, they
will be awarded with “MarCol points”
Without economic
model
Users can freely access any suggested
document and are not awarded by
submitting their judgments
The following questions (tasks) were used (Turpin and Hersh 2001):
1. What tropical storms hurricanes and typhoons have caused property
damages or loss of life?
2. What countries import Cuban sugar?
3. What countries other than the US and China have or have had a
declining birth rate?
4. What are the latest developments in robotic technology and it use?
5. What countries have experienced an increase in tourism?
6. In what countries have tourists been subject to acts of violence
causing bodily harm or death?
Experiment Methods
• There were six equal subgroups, while every subgroup was given
its unique sequence of questions (a Latin square).
• There were six sub stages; on each sub stage the participants were
provided with a different question.
Experiment Procedure
654321
6243151
1324562
3561423
4652314
2135645
5416236
SubstageP
art
icip
an
ts
Su
bg
rou
p
Experiment Results – first experiment Performance
80%
82%
84%
86%
88%
90%
92%
94%
96%
98%
100%
1 2 3 4 5 6
Sub-Stage
Perf
orm
an
ce
MarCol Free MarCol
• There is a significant difference between the modes (p≈0) for a
99% confidence interval.
Experiment Results – second experiment Performance
40%
50%
60%
70%
80%
90%
100%
1 2 3 4 5 6
Sub-Stage
Perf
orm
an
ce
MarCol Free MarCol
• There is a significant difference between the modes (p≈0) for a
99% confidence interval.
Experiment Results – first experiment Participation
• There is a significant difference between the modes (p=0.008204)
for a 99% confidence interval.
1.67
1.21
2.71
2.332.25
2.92
1.63
1.17
2.29
2.63
3.00
2.50
1.00
1.50
2.00
2.50
3.00
3.50
1 2 3 4 5 6
Question
Part
icip
ati
on
MarCol Free MarCol
Experiment Results – first experiment Accumulated Participation
10.67
1.46
13.33
15.63
2.42
4.38
8.887.54
5.88
4.21
7.54
10.42
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
1 2 3 4 5 6
Sub-Stage
Part
icip
ati
on
MarCol Free MarCol
Experiment Results – first experiment Accumulated Participation
10.67
1.46
13.33
15.63
2.42
4.38
8.887.54
5.88
4.21
7.54
10.42
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
1 2 3 4 5 6
Sub-Stage
Part
icip
ati
on
MarCol Free MarCol
Experiment Results – second experiment Participation
• There is a significant difference between the modes (p=0.000164)
for a 99% confidence interval.
0.27
0.18
0.36
0.70
1.60
0.50
0.91
0.55
0.450.50
1.10
0.90
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
1 2 3 4 5 6
Question
Part
icip
ati
on
MarCol Free MarCol
Experiment Results – second experiment Accumulated Participation
2.73
1.10
4.30
5.30
0.55 1.00
2.362.00
1.36
1.90
3.10
3.80
0.00
1.00
2.00
3.00
4.00
5.00
6.00
1 2 3 4 5 6
Sub-Stage
Part
icip
ati
on
MarCol Free MarCol
Experiment Results – first experiment Satisfaction
• There is not a significant difference between the modes
(p=0.822535) for a 99% confidence interval.
4.00
3.74
4.24
4.43
3.47
3.78
4.32
3.95
4.19
3.88
3.663.66
3.40
3.60
3.80
4.00
4.20
4.40
4.60
1 2 3 4 5 6
Sub-Stage
Sa
tis
fac
tio
n
MarCol Free MarCol
Experiment Results – second experiment Satisfaction
• There is not a significant difference between the modes
(p=0.746576) for a 99% confidence interval.
3.50
1.25
2.22
3.833.673.25
2.382.39
2.90 3.33
4.00
3.54
0.00
1.00
2.00
3.00
4.00
5.00
1 2 3 4 5 6
Sub-Stage
Sa
tis
fac
tio
n
MarCol Free MarCol
• User performance is significantly better when using MarCol mode.
– The average superiority of is 6% in the first experiment, and 16% in the
second.
– The user performance superiority of MarCol increases as the task is more
difficult.
• User participation is significantly higher when using MarCol
mode.
– The average superiority of MarCol is 46% in the first experiment, and 96%
in the second.
– The user participation superiority of MarCol increases as the task is more
difficult.
– The participation grows constantly over time and so does the gap between
the MarCol and MarCol Free modes in both experiments.
• There is not any significant difference in user satisfaction between
the modes.
Summary of Results
Conclusions and Trends search engines personalization
Search engines already integrate personal ranking
Technology is yet to be developed to enahance personalization
Still needs evaluations to calibrate the degree of personalization
Privacy issues are to be considered
Paper: Dan Melamed, Bracha Shapira, Yuval Elovici: MarCol: A Market-Based Recommender System. IEEE Intelligent Systems 22(3): 74-78 (2007)
