Upload
harrikipper
View
1.582
Download
8
Tags:
Embed Size (px)
Citation preview
1
Copyright © C. Dellarocas 2008-09
The Many Faces of Reputation:
Towards a discipline of Web 2.0 reputation system design
Prof. Chris DellarocasRobert H. Smith School of BusinessUniversity of [email protected]
Copyright © C. Dellarocas 2008-09
Suddenly everything Web 2.0 became so sexy…
2
Copyright © C. Dellarocas 2008-09
Except, perhaps, reputation systems…
RepuNomics
How Online Reputation
Enables Markets
Sustains Communities
and Creates
New Business
Opportunities
This book
does not exist
Copyright © C. Dellarocas 2008-09
But reputation systems are everywhere
eCommerceeBayAmazonEpinions
Web 2.0YelpSlashdot
CrowdsourcingYahoo! AnswersiStockPhoto
GamingXbox Live
3
Copyright © C. Dellarocas 2008-09
Reputation Systems are the Unsung Heroes
of the Web
Copyright © C. Dellarocas 2008-09
Outline of this talk
Reputation Systems Serve Business Objectives
Four areas where practice runs ahead of research
Our challenge: Developing a rigorous discipline of real-life reputation system design
4
Copyright © C. Dellarocas 2008-09
Some Definitions
Reputation = a summary of one’s past actionsdefined within the context of a specific community
Reputation system = a system that mediates and automates the process of assessing one’s reputation
Keeps track of a user’s actionsAggregates and displays summary statistics
Copyright © C. Dellarocas 2008-09
Bird’s eye view of a reputation system
Aggregation
Outputs
Inputs
Users
(Inter)actions
Artifacts
COMMUNITY
5
Copyright © C. Dellarocas 2008-09
Reputation Systems serve a variety of
Business Objectives
Copyright © C. Dellarocas 2008-09
Why do communities need reputation systems?
TrustEncourage “good” and discourage “bad” behavior
QualityProvide incentives for quality contributions Recognize best contributors
Matching and FilteringAssist users in finding suitable partnersReduce information overload
Participation and loyaltyGive user reasons to join and stay in your community
6
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
7
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
8
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
Four examples
++++++++Yahoo!Answers
+++++++++++Yelp
++++++++Amazon
+++++++eBay
LoyaltyFilteringQualityTrust
9
Copyright © C. Dellarocas 2008-09
The design space of Reputation Systems
is very rich
Copyright © C. Dellarocas 2008-09
Four dimensions of reputation system designwhere practice runs ahead of research
1. What inputs should be solicited2. What outputs should be presented3. How transparent should the rules be4. How should reputation evolve over time
10
Copyright © C. Dellarocas 2008-09
What inputs should be solicited?
What internal actions to keep track ofWhat external feedback to solicitRatings of artifacts vs. ratings of people
Copyright © C. Dellarocas 2008-09
11
Copyright © C. Dellarocas 2008-09
Determining inputs
Perform an audit of your systemList all actions a user can takeAssess how each action relates to each of the four classes of objectivesTrack/report actions that:
you want to encourage/discourageprovide the most information related to one or more of your key objectives
Copyright © C. Dellarocas 2008-09
A story of a site that got it wrong: Consumating.com
12
Copyright © C. Dellarocas 2008-09
An interesting question: Should you allow ratings of users….
… separately from the artifacts they create?
Pluses:Makes a person feel like a person… which might increase site loyalty
Minuses:Can distract from quality of one’s workMight encourage personal attacks
Copyright © C. Dellarocas 2008-09
13
Copyright © C. Dellarocas 2008-09
Four dimensions of reputation system designwhere practice runs ahead of research
1. What inputs should be solicited2. How outputs should be presented3. How transparent should the rules be4. How should reputation evolve over time
Copyright © C. Dellarocas 2008-09
What outputs should be presented?
Simple statisticsStar ratingsScoreNamed levelsAchievement badgesSocial network
14
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
Simple Statistics
15
Copyright © C. Dellarocas 2008-09
Statistics
Score
Copyright © C. Dellarocas 2008-09
SocialNetwork
16
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
17
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
18
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
To what extent should the system allow comparison across users?
Display of relative performance statisticsTop N lists“Leaderboards”
19
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
20
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
21
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
22
Copyright © C. Dellarocas 2008-09
To compare or not to compare?
PlusesIncreases incentives to contributeEnhances filtering role of reputation
MinusesInstills a culture of competitionObsession with rankings might lead to manipulative, disruptive behaviorLow ranked users might be discouraged and exit
Copyright © C. Dellarocas 2008-09
A cautionary tale
Digg.com
23
Copyright © C. Dellarocas 2008-09
How you display reputation information affects the spirit of your community
Reputation system design must be consistent with the community’s overall culture
Statistical evidence
RankingLeaderboards
Point-based scoreBadges
NamedLevels
CORDIAL COMPETITIVE
Copyright © C. Dellarocas 2008-09
Some ideas
Implement multiple ways of ranking usersNo single measure that people obsess onAllows users with different qualities to feel goodParticularly relevant in systems where matching and filtering is an important objective
24
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
Four dimensions of reputation system designwhere practice runs ahead of research
1. What inputs should be solicited2. What outputs should be presented3. How transparent should the rules be4. How should reputation evolve over time
25
Copyright © C. Dellarocas 2008-09
How transparent should be the aggregation rules?
Transparency breeds trust
… but also facilitates gaming
Copyright © C. Dellarocas 2008-09
26
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
27
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
Four dimensions of reputation system designwhere practice runs ahead of research
1. What inputs should be solicited2. What outputs should be presented3. How transparent should the rules be4. How should reputation evolve over time
28
Copyright © C. Dellarocas 2008-09
How should reputation evolve over time?
Accumulate and growDecay and refresh
TradeoffsAccumulation breeds complacencyAccumulation discourages new entrants
BUTAccumulation promotes loyalty and lock-in
Copyright © C. Dellarocas 2008-09
Accumulation
Decay and Refresh
29
Copyright © C. Dellarocas 2008-09
A case study:Yelp
Copyright © C. Dellarocas 2008-09
30
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
31
Copyright © C. Dellarocas 2008-09
Copyright © C. Dellarocas 2008-09
The challenge:Mapping Business Objectives
to Design
32
Copyright © C. Dellarocas 2008-09
Observation #1: Everything should be driven from a clear understanding of objectives
Trust building/Community PolicingIncentives for Quality Matching and FilteringSite Loyalty
It is important for the designer to be very clear with respect to how these goals are prioritizedIt is important to understand how every aspect of design affects each of these goals
Copyright © C. Dellarocas 2008-09
Observation #2: The design of a reputation system can affect a community’s culture
Reputation system design must be consistent with the community’s overall culture
Statistical evidence
RankingLeaderboards
Point-based scoreBadges
NamedLevels
CORDIAL COMPETITIVE
33
Copyright © C. Dellarocas 2008-09
Observation #3: Intangible aspects of reputation matter
Users care about their reputation beyond the tangible benefits it confers to themReputation as pure status
Status is zero-sumUsers who are not recognized might get upsetSo, introduction of a reputation system might make some users worse off and cause them to leave
Copyright © C. Dellarocas 2008-09
Observation #4: A reputation system can be a site’s competitive weapon
A way to lock-in users to our site
Lock-in properties of reputation system can sometimes be in conflict with its other objectives
Understanding the design of competitors’reputation mechanisms matters
34
Copyright © C. Dellarocas 2008-09
In summary
Much of what we know about reputation system design in Web 2.0 communities is still anecdotal
We need research-driven guidelines on how to design such systems
A lot of open areas where theoretical and experimental work needs to be done
Copyright © C. Dellarocas 2008-09
In the meantime…
Reputation systems designers will be the poor lonesome cowboys of the WebNot necessarily such a bad place to be (hey, it can get you to places like Gargonza)