Contextual Support for Collaborative

Information RetrievalShuguang Han, Daqing He, Zhen

Yue and Jiepu Jiang

1

Shuguang Han, Daqing He, Zhen Yue, and Jiepu Jiang. 2016. Contextual Support for Collaborative Information Retrieval. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR '16). ACM, New York, NY, USA, 33-42. DOI=http://dx.doi.org/10.1145/2854946.2854963

MotivationCollaborative Information Retrieval (COL) VS. Individual Information Retrieval

(IND)Studies observed an increasing trend of COL, in which people search together for the same taskCOL provides unique & interesting contextual factors for search support

team project

ski trip

students

family

Communication & Coordination

Shared Search History

Search

Search2

Research Questions

RQ1: Are IND contextual support directly applicable in COL without handling complex COL interactions?

3

RQ2: Can partners’ search histories be employed for contextual search support in COL?

RQ3: Can team members’ chat content be applied for contextual search support in COL?

Contextual Support for COLSearch contexts

Five types (since chat is highly interactive and is not isolated, we merge self- and partner-chat)

4

Contextual search support modelsDocument re-ranking based on search contexts

Experiment setup for contextual search supportData about user search contextsGround-truth

Obtaining Data CollectionUser studyA self-developed collaborative web search system

Search Shared

Workspace

Shared Query HistoryCommunication &

Coordination5

Obtaining Data CollectionSearch Tasks

T1: Academic TaskWriting a report to summarize the development of SNS

T2: Leisure TaskPlanning a trip to Helsinki

User Study ProcedureConditions: Collaborative search (COL) & Individual search (IND)Within subject design

Each team (in COL) / individual(in IND) works 30 mins for each taskPost-task questionnaire for relevance rating of each saved document

User Study Data18 pairs (for COL) and 18 individuals (for IND)970 queries, 1,384 clicks and 909 saved documents

6

Ground-truth● For each team {U, V}, pooling document relevance

○ Aggregating document relevance by averaging relevance judgement from other users

● Personalize ground-truth for each query q○ Pooled relevant documents already-saved relevant documents

7

Utilizing Self Search Histories in COL RQ1: Are IND contextual support directly

applicable in COL?

8

ND

CG

@N

MA

P

G: pure google results (baseline)G + H_QUS: re-rank Google results using self query historyG + H_CLS: re-rank Google results using self click history

Take-away messages● Self search history is also useful in COL● QU > CL (different from [Shen et al. 2004])

○ may because of the task nature○ may because of that self search

behaviors in COL are influenced by collaboration

Utilizing Self Search Histories in IND

Academic Leisure

#chat messages

21.56 (20.98) 38.86 (27.24)

#queries 12.97 (6.98) 9.25 (5.30)

#click-through 24.00 (13.23)

14.44 (7.25)

search-intensive task collaboration-intensive task 9

ND

CG

@N

MA

P

Take-away messages● IND is consistent with COL for Academic task● IND is inconsistent with COL for Leisure task

○ Hypothesis: Strong collaboration effect in leisure task

G: pure google results (baseline)G + H_QUS: re-rank Google results using self query historyG + H_CLS: re-rank Google results using self click history

Utilizing Partners’ Search Histories in COL

ND

CG

@N

RQ2: Can partners’ search histories be employed for contextual search support in COL?

MA

P

10

G + H_QU: re-rank Google by both (self + partner) query historyG + H_CL: re-rank Google by both (self + partner) click history

Take-away messages● Incorporating search histories from

partners can provide a better contextual support

Utilizing Chat Histories in COL

Take-away messages● Chat-based approach performs the best in

the leisure task● Chat-based approach outperforms click-

based method in the academic task

● This finding goes beyond our expectation○ Chat contains a lot noise (our previous study

find that 30% of chats are irrlevant to tasks)

ND

CG

@N

RQ3: Can team members’ chat content be applied for contextual search support in COL?

G + H_CH: re-rank Google results by chat history

MA

P

11

Category Description

Task social (TS) Chats concerning attitude to obtained resources

Task coordination (TC) Chats about the coordination of a task

Task content (TT) Chats related to the content of the search task

Non-task (NT) Chats that are not related to the search task

G + H_X: re-rank Google with {TS, TC, TT, NT} chat history

ND

CG

@N

MA

PTake-away messages

● Any type of chat (even for NT) helps improve search

● Involving all chat messages achieves the best

Categorization schema for manual chat message coding [2]

12

Utilizing Chat Histories in COL

Analyzing Non-Task Chat in COL

Task NT (×10-

3)Others (×10-

3)

T1 p(X|C)

0.520 1.936

p(X|R)

0.509 2.422

T2 p(X|C)

0.333 2.120

p(X|R)

0.346 3.139

Why NT works?● They contain useful information. Each chat message refers to all the content a user typed in chat

box before she hits the submit button, which may cover multiple types of chat content. ● Noise in NT (e.g., lol, haha, okay) does not hurt result ranking

13

● p(NT|R) ≈ p(NT|C)● p(Others|R) > p(Others|

C)

p(X|C)

p(X|R)

corpus

relevant docs

C

R

Take-away MessagesContextual support method developed in IND can be directly applied in COL

Task may affect the utility of different search contexts

Partners’ search histories can help for contextual support in COLCombination of self and partners’ search histories provides better utility

Chat messages can help in for contextual support in COLNoise in chat messages may not affect the utility of chat-based context

14

Q & A

15

Task Socialwell, hello there

yeah! we are going to Helsinki!

everything looks great so far!

Task Coordinationyou do stats and I’ll do impacts on students and professionals

have you done impact yet?

Task Contentok so outdoor activities will be hard

in December they set up tons of markets and stuff in the streets

Non-taskCan we eat after this?

I wish there was a notification every time we saved a page

16

Examples of Different Chat Types