Upload
brynn-evans
View
106
Download
1
Tags:
Embed Size (px)
DESCRIPTION
To what extent can social interactions augment people’s natural search experiences? What factors influence the decision to turn to a friend for help? This talk presents the preliminary results of a social sensemaking task that begin to address such questions by examining the cognitive consequences of social search.
Citation preview
April 9, 2009
EXPLORING THE COGNITIVE CONSEQUENCES OF SOCIAL SEARCH
CHI ’09
UC San Diego
PARCPARC
Brynn M. Evans
Sanjay KairamPeter Pirolli
Tuesday, April 14, 2009
MotivationResearch Goal
How do people make use of social resources during search tasks?
In what ways do social interactions cognitively benefit search?
Tuesday, April 14, 2009
As a study of social search, we are particularly interested in HOW people make use of social resources during search tasks.
Motivation
Traditional search fails with exploratory queries
Motivation
Tuesday, April 14, 2009
This is important because Google sometimes fails us.
In fact, exploratory queries are difficult to answer with traditional search alone (there were several sessions at CHI that have looked at this issue).
Social interactions are common during search
Motivation
Photo Credit: David Wild
MORRIS 2008; EVANS & CHI 2008
Tuesday, April 14, 2009
Yet prior work has shown that people do ask friends for help during search.
Social interactions are actually quite common during search (between half and two-thirds of searches may involve social interactions).
Motivation
Social interactions play an important cognitive role
Photo Credit: http://www.infodiv.unimelb.edu.au/tss/archive/history.html
HATCH & GARDNER 1993; KARASAVVIDIS 2002
Tuesday, April 14, 2009
Additionally, social interactions play an important cognitive role in problem solving, brainstorming, and learning tasks. People help each other think through problems and reframe issues.
This has been documented in classrooms, organizations, libraries, and other physical environments.
Study design
selected in pre-test survey for being:
• expert searchers
• highly social ‣ 55-1000 friends on primary social network
PARTICIPANTS:
PROTOCOL: • 2 Google-hard search tasks• Talk-aloud protocol• Video protocol analysis
(N=8)
Tuesday, April 14, 2009
To study the benefit of social interactions in search, we recruited 8 subjects who were expert searchers, highly social.
They performed 2 Google-hard search tasks in two seach conditions. We video recorded all interactions and asked them to talk-aloud while they were searching.
Study design: tasks
“If we lowered the speed limit nationally to 55 mph, how many fewer barrels of oil would the U.S. consume every year?”
55 MPH
“What role does pyrolytic oil (or pyrolysis) play in the debate over carbon emissions?”
PYROLYTIC OIL
Tuesday, April 14, 2009
These were our task questions.
Study design: conditions
NON-SOCIAL CONDITION
• search engines (Google, Yahoo)
•Wikipedia
Tuesday, April 14, 2009
Subjects received each task in one of two conditions.
In the non-social condition, they performed an otherwise typical web search...
Study design: conditions
SOCIAL CONDITION
• friends (email, phone, IM, etc.)
• social networks
•blogs
•Question-Answer sites
Tuesday, April 14, 2009
In the Social Condition, they were restricted to social resources only: friends, social networks, blogs, question-answer sites...
Three primary social strategies
TARGETED ASKING
NETWORK ASKING
SEARCHING
Photo Credit: Timothy Morgan
Tuesday, April 14, 2009
Our analysis is restricted to the social condition right now, since we are primarily interested in HOW people exploit their social enviroments.
We found that people used one of three primary social tactics to answer our task problems
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
Task performance
SS03
TIME
Photo Credit: Timothy Morgan
Tuesday, April 14, 2009
You can see these strategies reflected in the color coding of behaviors on one subject’s timeline.This person begins by searching over MetaFilter; then posts questions on MetaFilter & Twitter; and finally IMs with a friend to get additional help.But as you can also see, these activities are interspersed with many other behaviors (as expected) -- such as the gray bars: thinking/synthesizing the information he’s come across
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
Task performance
SS03
TIME
SEARCHING
[ ]
SEARCHING
Photo Credit: Timothy Morgan
Tuesday, April 14, 2009
You can see these strategies reflected in the color coding of behaviors on one subject’s timeline.This person begins by searching over MetaFilter; then posts questions on MetaFilter & Twitter; and finally IMs with a friend to get additional help.But as you can also see, these activities are interspersed with many other behaviors (as expected) -- such as the gray bars: thinking/synthesizing the information he’s come across
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
Task performance
SS03
TIME
NETWORK ASKING
[ ]
NETWORK ASKING
Photo Credit: Timothy Morgan
Tuesday, April 14, 2009
You can see these strategies reflected in the color coding of behaviors on one subject’s timeline.This person begins by searching over MetaFilter; then posts questions on MetaFilter & Twitter; and finally IMs with a friend to get additional help.But as you can also see, these activities are interspersed with many other behaviors (as expected) -- such as the gray bars: thinking/synthesizing the information he’s come across
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
Task performance
SS03
TIME
TARGETED ASKING
[ ]
TARGETED ASKING
Photo Credit: Timothy Morgan
Tuesday, April 14, 2009
You can see these strategies reflected in the color coding of behaviors on one subject’s timeline.This person begins by searching over MetaFilter; then posts questions on MetaFilter & Twitter; and finally IMs with a friend to get additional help.But as you can also see, these activities are interspersed with many other behaviors (as expected) -- such as the gray bars: thinking/synthesizing the information he’s come across
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
Task performance
SS03
TIME
THINKINGRESPONSES?OTHER
Photo Credit: Timothy Morgan
Tuesday, April 14, 2009
You can see these strategies reflected in the color coding of behaviors on one subject’s timeline.This person begins by searching over MetaFilter; then posts questions on MetaFilter & Twitter; and finally IMs with a friend to get additional help.But as you can also see, these activities are interspersed with many other behaviors (as expected) -- such as the gray bars: thinking/synthesizing the information he’s come across
Tuesday, April 14, 2009
Of course, subjects were highly diverse in the strategies they employed, as can be seen by the various color coding patterns in their search episodes. This shows patterns for all 8 subjects (in the social condition).Some didn’t have an expert friend on hand -- (they didn’t have the “lifeline” to call.) Or they didn’t usually post questions to their friends on Facebook, for example. For this reason, not everyone performed all three social strategies.
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
Task performance
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
each
lin
e =
= o
ne
fact
SS03
Tuesday, April 14, 2009
Next, we mapped the facts subjects found to the type of behavior they were engaged in.
...the size of the bubble represents the depth of processing of that one fact. And you can see that some facts do become synthesized more over time. [see fact #2, #3, #4, #5, #6]
Results
ID#social tactics
#facts discovered
S01 3 6
S02 1 2
S03 3 8
S04 1 1
S05 2 5
S06 2 1
S07 1 1
S08 3 3
Spearman R: 0.77
combining social tactics correlates better to performance on our tasks than:
• social network size• network diversity
(as measured by the position generator)
• background knowledge / intrinsic interest in topic
Tuesday, April 14, 2009
We found that any one social tactic used on its own didn’t produce as good performance outcomes as combining social tactics together.
This is reflected in both the total number of facts discovered in the session and how deeply users actually processed information (which isn’t elaborated on here).
This suggests that accessing people with different technologies makes a difference in their benefit to you.
Results
deeper cognitive processing of information while:
“Now, what could I say?”
“Let’s see...what do I really want to be asking?”
Composing the question Receiving Information
Network (Asking)
More thinking, contemplation while crafting query (3/5 users)
No pondering or mulling over of network responses (0/4 users)
Targeted (Asking)
Little thinking or reformulation of problem statement
(2/7 users)
More synthesis of information from friends’ replies (4/5 users)
Tuesday, April 14, 2009
And looking at the cognitive processing of information, this is absolutely what we saw.
However the story is more nuanced than that:
If we divided the search phase into the question and answering components:We found that users had more thinking and contemplation of their search problem while they were composing the question (e.g., when posting it to a social network...)
Composing the question Receiving Information
Network (Asking)
More thinking, contemplation while crafting query (3/5 users)
No pondering or mulling over of network responses (0/4 users)
Targeted (Asking)
Little thinking or reformulation of problem statement
(2/7 users)
More synthesis of information from friends’ replies (4/5 users)
Results
deeper cognitive processing of information while:
Tuesday, April 14, 2009
In contrast, users more deeply integrated and synthesized information from replies shared from friends who they interacted with one-on-one.
This could be due to a number of different things. For one, replies on social networks tended to be goofy, silly, or off-target. They didn’t contain any facts that helped subjects advance their understanding of the search question. (In fact, when we followed up with these respondents, they reported wanting to start a conversation with our users rather than help with a substantive reply).
Replies from targeted friends, on the other hand, were content-heavy. Even replies shared through asynchronous channels (an email, a single IM) caused subjects to pause and really think about the information they just received. Respondents here reported wanting to help our users develop content; they were less interested in the “social” aspect of the communication.
Results
deeper cognitive processing of information while:
!"
#"
$"
%"
&"
'"
("
)"
*"
+"
#!"
##"
#$"
#%"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"
!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"
SS03
More synthesis of information from friends’
replies (4/5 users)
Tuesday, April 14, 2009
We can see that users paused to think about shared information from targeted asking in one subject’s timeline: The green (asking friends) and gray bands (thinking episodes) pattern each other. This is only illustrated for one subject, but was true for most of the subjects who engaged in targeted asking.
Conclusions & implications
• People serve as more than information sources; they also provide cognitive support.
• But different types of social engagement benefit the search process in different ways:
• Problem reformulation while posting questions on social networks
• Integration of information from friends’ replies.
IMPLICATIONS: Can we design new tools that leverage these cognitive benefits during search tasks?
CONCLUSIONS:
Tuesday, April 14, 2009
So we have seen how different types of social engagement benefit the search process.
People serve as more than information resources; they also provide cognitive benefits!
In particular we saw problem reformulation while posting questions on social networks and when integrating information from friends’ replies.
Could we design an integrated search tool that takes this into account? For example, if users posted questions in social networking sites, could we route their question to domain experts from their personal network who then might feel more compelled to reply? Could a model like this help maximize the cognitive benefits for searchers during both questioning and answering phases of the search query?
At the same time, we should learn more about why people are motivated to respond appropriately to targeted questions, but seem to provide less relevant replies on social networks.
Acknowledgements
our participants
SRC judges (CHI ’09)
Augmented Social Cognition group at PARC!
BRYNN M. EVANS [email protected] http://brynnevans.com
A GREAT THANKS TO:
Tuesday, April 14, 2009