Embed Size (px)
Citation preview
1
Relevance Ranking in the Scholarly Domain
Dr. Tamar Sadeh
LIBER Conference
Tartu, Estonia, June 2012
2
1 2 3The top three keys for success
1 Content
2 Speed
3 Relevance Ranking
8
Relevance is the measure of
correspondence between a document
and a query as determined by a user
Based on Saracevic, 1975
9
??
10
• System or algorithmic relevance
• Topical or subject relevance
• Cognitive relevance or pertinence
• Situational relevance or utility
• Affective relevance
• System or algorithmic relevance
• Topical or subject relevance
• Cognitive relevance or pertinence
• Situational relevance or utility
• Effective relevance
11
There is no absolute relevance
12
13
14
15
16
17
18
The ScholarRank
Project
19
The GoalEnhance the Primo relevance
ranking algorithm
The GoalEnhance the Primo relevance
ranking algorithm
20
Relevance ranking was not new to us.
28
MethodologyMethodology
29
• Setting up a team
• Building test environment, tools, and
procedures
• Defining metrics to evaluate our current
success and the improvements we make
• Defining measurements to assess the success
of the changes, once implemented
30
EvaluationEvaluation
31
• Working with researchers
• Researchers’ evaluation quantified
• Enhancements introduced and checked in the
lab, using defined metrics
• Enhancements launched and usage patterns
monitored
• Improvements are introduced on an ongoing
basis
32
How is relevance ranking
calculated?
How is relevance ranking
calculated?
33
??
34
abstract author date full text journal
language type publisher
subject title
citations downloads journal impact factor eigenfactor pagerank
35
academic degree discipline(s) language
location previous selections search history
36
broad-topic search currency exact-item search material type
narrow-topic search
??
37
Broad-topic query
Narrow-topic query
Author-related query
Known-item query
??
39
??
40
The match: traditional information retrieval
methods, adapted to the scholarly environment
42
1
20
100
43
??no. of citations; no. of selections; recency; type; peer review
45
1
20
100
46
??
Academic degree, discipline
47
48
Computer Science
49
Psychology
50
Mathematics
51
Physics
52
??
Author-related query, known-item query, broad-topic query…
53
Before
54
After
Tachycardia – Wikipedia the free encyclopediaTachycardia comes from the Greek words tachys (rapid or accelerated) and kardia (of the heart). Tachycardia typically refers to a heart rate that exceeds the normal range for a resting ...
55
56
??ScholarRank™
57
