Personalizing the Digital Library Experience

Nicholas J. Belkin, Jacek Gwizdka, Xiangmin Zhang

SCILS, Rutgers University

nick@belkin.rutgers.eduhttp://scils.rutgers.edu/imls/poodle

Goals of Personalization

• To make the user’s interaction with information as effective and pleasurable as possible

• To tailor the user’s interaction with information to the user’s characteristics, preferences, the specific circumstances of the interaction, and the user’s goals

Types of Personalization

• With respect to predictions of usefulness/ relevance of items, e.g.– modify query– re-rank results

• With respect to interaction, e.g.– different interface designs for different tasks– different interface designs for different

individuals

Facets of Personalization

• Viewing/saving/evaluating behaviors

• Task

• Problem state

• Personal characteristics

• Personal preferences

• Context/situation

Viewing, etc. Behaviors

• Implicit evidence (Kelly & Teevan)– Time on “page”– Click-through– Previous uses– Others like the interactant

• Explicit evidence– Relevance feedback (of various sorts)

Task

• “Everyday” or “leading” or “work” task– Complexity, difficulty, “type” (Bystrom, et al.)

• Information seeking task– Choice of strategies, sources (Bates, Pejtersen,

berrypicking)

• Information searching task– Moves, shifts (Bates; Xie)

Problem State

• What has been done before– Previous searches

• Stage in the Problem Solving Process (Kuhlthau; Vakkari)

• What is being done now– Immediately past behavior in searching, other

concurrent activities

Personal Characteristics

• Knowledge – of topic, of task

• Demography – gender, age

• Individual differences– Cognitive abilities– Affect

Personal Preferences

• For types of interaction– Mixed or single initiative

• For styles of interaction– Display, navigation

• For support for interaction– Active, passive– Integrated, separate

• For types of information– Genre, level

Context, Situation• Location

– Physical environment– Mobile, static

• Salience• Urgency• Time

– of day, of week, of month, of season, …

• Other interactants– Group conditions

• Social norms

Overall Goals for Personalization

• Determining significant aspects of each facet

• Determining means for identifying these aspects

• Determining means for implementation of support

• Integrating all facets of personalization into single system frameworks

Evidence for Personalization

• Explicit evidence, e.g.– Relevance judgments

– Statements of goals, problems, etc.

– Location; time of day, week, month, year

• Implicit evidence, e.g.– Dwell time

– Clickthrough

– Past searches, uses

– Concurrent activities

Interpreting Implicit Evidence

• Dwell time is evidence of usefulness / relevance / interestingness– But needs to be interpreted in terms of task (Kelly,

2004; White & Kelly, 2006)

– Is dependent on individual characteristics (Kelly, 2004)

• In general, evidence from any one facet could affect interpretation of evidence from any other facet

• All evidence is probably individual-dependent

Our Approach

• Investigate in depth aspects of specific facets, e.g.– Task– Domain knowledge– Cognitive characteristics

• Investigate the interactions among the different facets

• Implement and test within an integrative system framework

• Using a client-side “personalization assistant”

Initial Investigation

• Three months of logs of all computer use and searching behavior for each of seven Ph.D. students

• Judgments, by subjects, of usefulness of pages viewed as results of searching, with task type, duration and stage of task, topic, and familiarity with topic

• Both from Kelly (2004).

Data Analysis

• Exploratory analysis of relationships among dwell time and each of: task and topic familiarity; task stage; and, task duration, to determine most accurate dwell time value for predicting usefulness

• Exploratory analysis of current and past behavior as indicator of task type, task stage, and task/topic familiarity

Results to Date for Task Staget test res ults for dwell time (log) for useful and non-useful documents for task stages of each subject (UT = Usefulness threshold on 7-point scale) sub stage UT t df p Useful Non-useful d n M(SD) n M(SD) 5 5 6 2.99 31 .005 19 3.28(1.36) 14 1.99(1.02) 1.05 5 7 (6) 2.75 31 .01 18 3.28(1.39) 15 2.08(1.04) 0.96 6 1 5 -3.8 21 .001 21 0.58(0.71) 2 2.52(0.17) -2.81 3 6 (5) 2.14 255 .03 29 2.05(1.07) 228 1.60(1.07) 0.42 5 6 (5) 2.44a 9.13 .04 9 2.31(1.86) 221 1.58(1.12) 0.83 6 5 2.39a 129.94 .02 84 1.52(1.32) 172 1.13(0.99) 0.34 6 6 (5) 4.86 a 5.44 .004 4 1.92(0.24) 252 1.24(1.13) 1.23 7 1 6 -3.47 170 .001 114 4.03(0.92) 58 3.49(1.01) -0.56 4 6 -2.37 76 .02 57 3.79(1.00) 21 3.19(0.99) -0.60 5 6 -3.12 42 .003 37 3.97(0.83) 8 2.93(0.96) -1.22

Results to Date for Task Typet-test results for dwell time (log) for useful and non-useful documents for individual tasks o f each subject (UT = Usefulness Threshold on 7-point scale; task numbers are specific to each subject) sub task UT t df p Useful Non-useful d n M(SD) n M(SD) 1 2 6 2.21 125 .03 21 3.44(1.48) 106 2.60(1.61) 0.53 6 6 -2.74 14 .02 6 1.90(0.85) 10 3.11(0.85) -1.41 2 10 7 -2.03 153 .04 116 3.11(1.42) 39 2.59(1.35) -0.07 3 2 6 2.19 21 .04 17 3.48(1.17) 6 2.24(1.26) 1.04 3 6 2.82a 106.67 .006 90 3.04(1.48) 39 2.42(0.97) 0.54 3 5 (6) 3.16a 93.82 .002 100 2.99(1.48) 29 2.35(0.75) 0.67 9 6 -2.36 13 .04 9 1.67(1.08) 6 3.05(1.17) -1.24 9 5 (6) -2.30 13 .04 10 1.75(1.05) 5 3.17(1.27) -1.26 4 1 6 (7) -2.39 35 .02 25 1.97(1.49) 12 3.41(2.13) -0.84 5 5 (7) -2.52a 39.46 .02 40 2.70(1.45) 2 3.28(0.03) -1.83 6 3 5 2.28 141 .02 117 2.00(1.07) 26 1.48(0.90) 0.49 3 6 (5) 2.27 141 .03 11 2.59(0.62) 132 1.85(1.07) 0.71 12 6 (5) 1.92 164 .05 22 1.87(1.28) 144 1.39(1.06) 0.44 7 7 7 (6) 2.43 52 .02 32 3.78(0.89) 22 3.16(0.98) 0.67 7 6 2.76 52 .008 33 3.8(0.88) 21 3.10(0.96) 0.77 27 5 (6) 8.22a 5.76 .000 2 4.67(0.19) 14 2.86(0.65) 6.21 a - not assuming equal variance

Results to Date for Topic Familiarity

• The three-way interaction of individual*usefulness*topic familiarity was significant, meaning that considering both the individual and topic familiarity information may be helpful in predicting usefulness by display time.

Next Steps

• Begin three concurrent investigations of– Domain knowledge– Task– Cognitive characteristics

• Each investigation to consider one additional facet• Each to identify:

– Evidence for particular facet– Use of evidence for personalization– Interaction of main facet with one other

Then …

• Implement results from previous investigations in prototype system

• Experiments to test methods of identification of evidence, and the use of that evidence from all facets simultaneously

And Finally

• Move from experimental prototype to robust client-side personalization assistant

• Distribute assistant to subjects in a real work environment

• Compare performance, usability, acceptability, etc. between those with, and those without the personalization assistant

• Make the personalization assistant available as open source software