View
2.224
Download
0
Category
Preview:
DESCRIPTION
Presented at IUI2010 conference workshop on Social Recommender Systems http://mrtaggy.com
Citation preview
Information Seeking with Social Signals: Anatomy of aSocial Tag-based
Exploratory Search Browser
Ed H. Chi, Rowan Nairn
Palo Alto Research Center
Contact: echi@parc.comArea Manager, Augmented Social Cognition Area
2CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Social Search Survey
[Evans & Chi, CSCW2008]
150 user surveys Help understand the
importance of:– social cues and information
exchanges– vocabulary problems– distribution and
organization
3CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
TagSearch Exploratory Focus
3
3 kinds of search
navigational transactional
28% 13%
You know what you want and where it is You know what you want to do
Existing search engines are OK
informational
59%
You roughly know what you want
but don’t know how to find it
Difficult for existing search engines
Opportunity
4CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Research Motivation
5CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
6CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
MapReduce Implementation
Spreading Activation in a bigraph MapReduce computation over a large data set
– 150 Million+ bookmarks
Tags URLs
P(URL|Tag)
P(Tag|URL)
7CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Guide
Web
Howto
TipsHelp
Tools
Tip
Tricks
Tutorial
Tutorials
Reference
Semantic Similarity Graph
Use Semantic Analysis to Reduce Noise
8CHI2009 MrTaggy TagSEarch– © 2008 Palo Alto Research Center Inc.
TagSearch Architecture
• MapReduce: months of computation to a single day• Development of novel scoring function
9CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Baseline Interface
10CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Exploratory Interface
11CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Experiment Design
2 interface x 3 task domain design– 2 Interface (between-subjects)
» Exploratory vs. Baseline– 3 task domains (within-subjects)
» Future Architecture, Global Warming, Web Mashups
30 Subjects (22 male, 8 female)– Intermediate or advanced computer and web search skills– Half assigned Exploratory, half Baseline.
For each domain, single block with 3 task types:– Easy and Difficult Page Collection Task [6min each]– Summarization Task [12min]– Keyword Generation Task [2min]
12CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Page Collection Tasks [6min each]
13CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Summarization Tasks [12min each]
14CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Procedure [2 hours]
Prior Knowledge Test 1st Task Domain
– With easy and difficult page collection tasks, summarization and keyword generation task.
– NASA cognitive load questionnaire
2nd Task Domain– Same battery of tasks and cognitive load questionaire
3rd Task Domain Experimental Survey
15CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Results: Interaction Behaviors
Number of Queries– Effect of Interface on number of queries (p < .01)
» Exploratory (M=7.81) > Baseline (M=3.77)
Time Taken– Effect of Interface on time taken (p < .01)
» Exploratory (7.7min) > Baseline (6.6min)
16CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Results: Page Collection Task
0
2
4
6
8
10
12
14
16
task 1 task 2 task 1 task 2 task 1 task 2
global warming future architecture web mashups
nu
mb
er
of
pag
es c
olle
cte
d
Baseline Exploratory
– Effects of Task Domain (p<.01) and Task Difficulty (p<.05)
– Interaction effect of Interface by Task Domain (p<.05), with Exploratory interface performing better in the Web Mashup domain
– For relevance scores, similar patterns.
Measure of # of pages collected
17CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Results: Summarization Tasks
0
1
2
3
4
5
futurearchitecture globalwarming mashups
nu
mb
er
of
sum
mar
y a
sp
ect
s
baseline exploratory
– Quality of summarization scored (Cohen’s Kappa=0.7)
– ANCOVA with Prior Knowledge as covariate
– Exploratory Interface scored higher in Future Architecture (p<.05) and Global Warming (p<.05)
– For Web Mashup, Prior Knowledge correlated positively with performance (r=.51)
18CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Results: Keyword Generation Tasks
0
2
4
6
8
10
12
14
16
18
globalwarming futurearchitecture mashups
nu
mb
er o
f ke
yw
ord
s
baseline exploratory
– ANCOVA showed Exploratory > Baseline for Future Architecture (p<.05) and Web Mashups (p<.01), but not for Global Warming.
– Linear model between PK and # of keyword generated for Baseline showed mean slope = 0.32 and significant (p<.05)
19CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Results: Cognitive Load
0
20
40
60
80
100
futurearchitecture globalwarming mashups
co
gn
itiv
e l
oa
d
baseline exploratory
– Exploratory > Baseline (p<.05)
20CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Discussion
Exploratory interface users:– performed more queries, – took more time, – wrote better summaries (in 2/3 domains), – generated more relevant keywords (in 2/3 domains), and– had a higher cognitive load.
Suggestive of deeper engagement and better learning.
Some evidence of scaffolding for novices in the keyword generation and summarization tasks.
21CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Summary
Harnessing user-generated tags to enrich content for social search
Weaknesses of social tagging systems is Tag Noise and Inconsistency– Difficult to leverage for search– Use data mining techniques to normalize and reduce noise– Apply normalized tag data in new search algorithm
Study suggest deeper user engagement in exploration and better learning with MrTaggy
22CHI2009 MrTaggy TagSearch– © 2008 Palo Alto Research Center Inc.Try it now: http://mrtaggy.comTry it now: http://mrtaggy.com
Thanks!
http://mrtaggy.com
http://spartag.ushttp://wikidashboard.parc.com
Our Blog: http://asc-parc.blogspot.com
Contact:Ed H. Chi, Ph.D.Manager, Augmented Social Cognition Areaechi@parc.com
Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 625-634.
Research VisionResearch Vision
Augmented Social CognitionAugmented Social Cognition Cognition: the ability to remember, think, and reason; the
faculty of knowing. Social Cognition: the ability of a group to remember, think,
and reason; the construction of knowledge structures by a group.– (not quite the same as in the branch of psychology that studies
the cognitive processes involved in social interaction, though included)
Augmented Social Cognition: Supported by systems, the enhancement of the ability of a group to remember, think, and reason; the system-supported construction of knowledge structures by a group.
Citation: Ed H. Chi. The Social Web: Opportunities for Research. IEEE Computer, Sept 2008
232008-11-07 Ed H. Chi ASC Overview
Collective Intelligence
Augmented Social CognitionAugmented Social Cognition
24
Higher Productivity via Collective Intelligence
Intelligence that emerges from the collaboration and competition of many individuals
search
sharing
foraging
TagSearch: Mining social data for automatic data clustering and organization:
• Better organization via user-assigned tags
• Better UI for browsing interesting contents
• Recommendation instead of just search
Social Transparency create trust and attribution:
• Increase participation via attribution
• Increase credibility and trust with community feedback
• Reduce wiki risks
SparTag.us: sharing of interesting contents:
• A notebook that automatically organizes your reading
• Social sharing of important and interesting tidbits
• Viral sharing of highlighted and tagged paragraphs
Foundation:• Understanding of human
cognition and behavior• Data mining of social data
Generic benefits:• Greater trust• Better decision-making• Useful sharing of info• Auto-organization thru
social data
Recommended