Regulating Behavior KSE 652 Social Computing Systems Design Uichin Lee Sept. 26, 2013 Regulating behavior in online communities S. Kiesler, A. Kittur,

Regulating Behavior

KSE 652 Social Computing Systems DesignUichin Lee

Sept. 26, 2013

Regulating behavior in online communitiesS. Kiesler, A. Kittur, R. E. Kraut, P. Resnick, 2012

Encouraging Commitment: Summary

• Affective Commitment– Identity-based affective commitment: e.g., social identity, group

norms, homogeneous groups, naming, common fate/goal/task, in vs. outgroups

– Bond-based affective commitment: e.g., leveraging social ties, social interaction mechanisms, profiles, pseudonyms

– Reducing repelling forces: e.g., large group, diversity, off-topic

• Normative Commitment– purpose, testimonies (social proof), reciprocity

• Needs-based Commitment– matching benefits with motivation, competitors, lock-in (sunk cost)

Contents

• Introduction to normative behavior• Limiting effects of bad behavior• Coerced compliance: limits on bad behavior• Encouraging voluntary compliance– Making norms clear and salient– Enhancing compliance – Rewards and sanctions

• Summary

Normative Behavior

• Normative Behavior– Rough consensus emerges about the range of behaviors that

the managers and most members consider acceptable– Ex) A news article about “a rape in a cyberspace” describes

an example of unacceptable behavior• Communities differ on which behaviors are normative– Wikipeida expects writers to adopt a neutral point of view

when writing articles– Huffington Post (www.huffingtonpost.com) expects guest

bloggers to express view points

http://www.huffingtonpost.com/

Normative Behavior

• Help community to achieve its mission– Being supportive, on-topic, etc.

• Behavioral norms of handling conflicts– Edit wars in Wikipedia three-revert rule

• Destructive behavior: troll and manipulators– Troll/griefers like to destruct the community

• E.g., inflammatory comments, corpse camping in games

– Manipulators like to produce some particular outcome• E.g., pumping up ratings of a particular venue in Yelp.com

using multiple “shill” accounts

Normative Behavior• Insiders “do” violate

– New comers do not know much about rules– Some people may have some cognitive/social impairments

• Those who have hard time inferring rules

– Social dilemma• Public goods problem: everyone is better off if everyone contributes some efforts to community,

but there is a temptation to free-ride on others’ contributions• Public bad problem (common pool resource; or tragedy of the common problem) • Online community: “people’s attention” is the limited (common pool) resource

• Learning design principles from institutions that have successfully self-governed common pool resources (Ostrom 1990)– Participation of rule making, monitoring, graduated sanctions, conflict-resolution

mechanisms• Regulation describes any efforts to decrease the frequency of non-normative

behaviors or lessen their impacts on the community– Law, norms, markets, technology; in online communities, hard to deal with laws; our focus

is on the latter three categories

Contents


• Summary

Limiting effects of bad behavior

• Design Claim 1: Moderation systems that pre-screen, degrade, label, move, or remove inappropriate messages limit the damage they cause.– Ex) deleting messages, disemvoweling, labeling/rating

messsages (e.g., rating in slashdot)

• Design Claim 2: Redirecting inappropriate posts to other places will create less resistance than removing them.– People may not agree with moderator’s decision– It’s better for people to take it outside, rather than removing

it (thus, not really watching flaming)

Limiting effects of bad behavior• Design Claim 3: Consistently applied moderation criteria, a chance to

argue one’s case, and appeal procedures increase the legitimacy and thus the effectiveness of moderation decisions.– People are more satisfied if they go though fair procedure (source of legitimacy

comes from procedural justice)– Interestingly, people likes to take a more severe punishment after they had had

their day in court than a milder punishment without any hearing (Tyler 1990)

• Design Claim 4: Moderation decided by people who are members of the community, are impartial, and have limited or rotating power will be perceived as more legitimate and thus be more effective– Moderator: community selected; those with expertise– Ex) Slashdot “promote quality, discourage crap”; it also has “meta-moderation

system to address issue of unfair moderators

Limiting effects of bad behavior

• Design Claim 5: Revision tools limit the damage disrupters can inflict in production communities.

• Design Claim 6: Filters or influence limits can limit the damage of shill raters in recommender systems, but only at the cost of ignoring some useful information from honest raters.– Algorithms of detecting suspicious patterns (e.g., too many in a

short term or insufficient variability)– Influence limiter partially discounts contribution (discount declines

as the system gains confidence)• Design Claim 7: A widely followed norm of ignoring trolls will

limit the damage they can do.– E.g., “DNFTT: do not fee the troll”

Contents


• Summary

Coerced Compliance

• Design Claim 8: Activity quotas allow people to participate in a community, but prevent repetitive, spam-like activity

• Design Claim 9: Gags and bans can limit the continuing damage of a bad actor, but only if it is hard for the bad actor to use a different account (“expensive” account) or if the ban is disguised

• Design Claim 12: Limiting fake accounts with CAPTCHAs or identity checks limits automated attacks.

Coerced Compliance

• Design Claim 10: Consistently applied criteria for gags and bans, a chance to argue one’s case, and appeal procedures increase the legitimacy (“procedural justice”) and thus the effectiveness of gags and bans.

• Design Claim 11: Paying to take actions in the community with currency accumulated through normal participation will reduce the ability for trolls and manipulators to act.– Ex) OSS’s vetting process (access ladders), Omidyar’s internal

currency for participation, Influence limiter for recommender systems (reputation currency)

Contents


• Summary

Encouraging voluntary complianceMaking norms clear and salient

• How to learn community norms?– Observing other people and consequences of their behavior– Seeing instructive generalizations or codes of conduct– Behaving and directly receiving feedback

• Descriptive (one’s perception) vs. injunctive (others’ perception) norms• People may want to fit in by doing what others are doing (some cases,

descriptive ~ injunctive) • Power of observing others (how a group learn norms)

– Auto-kinetic effect (Sherif 1936): each person speak out distance, and people’s perception of illusion converged to 3 inches

– Presentation of self in everyday life (Goffman 1959) explains people tend to act predictably within norms in typical social interactions (e.g., hospital, parties, elevator, etc.)

Encouraging voluntary compliance Making norms clear and salient Enhancing compliance Rewards and sanctions


• Design Claim 13: Publicly displaying examples of appropriate behavior on the site will show members what is expected and increase their adherence to those expectations.– Making people’s normative behavior (descriptive norms) to

all members of community• Ex) reducing binge drinking among students by publishing accurate

information about how much the typical student drinks (Perkins et al., 1999)

– Online communities have many ways of explicitly displaying descriptive norms (rather than random encounters): e.g., post of the week in an online forum



• Design Claim 14: Publicly contrasting examples of inappropriate behavior in the context of a descriptive norm of appropriate behavior will highlight the descriptive norm and increase people’s adherence to it.– Cialdini’s focus theory (1991): people learn norms from salient

behaviors—actions that stand out (e.g., negative behavior) and point out to people what is appropriate to do in a situation • Ex) seeing littering behavior in dirty vs. clean environments

– Online communities: e.g., moderator flag, disemvowel • Design Claim 15: Publicly displaying many examples of

inappropriate behavior on the site will lead members to believe this is common and expected.



• Design Claim 16: Displaying feedback of members to others increases members’ knowledge of community norms and compliance with them; formal feedback is more effective than informal feedback– Observing common online behavior (descriptive norms) vs. observing

feedback to such behaviors teaches injunctive norms (i.e., approved/disapproved behaviors)

– Informal (e.g., “thank you” comments) vs. formal feedback (e.g., points, stars); formal feedback is known to work better in practice

• Design Claim 17: In large communities, displaying statistics that highlight the prevalence of normative behavior will increase members' adherence to normative behavior – Ex) low percentage of messages flagged for violating the rule



• Design Claim 18: Explicitly stating rules and guidelines increase the ability for community members to know the norms, especially when it is less clear what others think is acceptable.


(Motorists returning to their cars were given a hand bill with different statements; Cialdini, Kallgren, Reno 1991)


• Then how prominently to display guidelines and rules?– Design Claim 19: prominently displayed or excessively detailed

rules may also convey the wrong descriptive norm• Reddit.com’s “reddiquette”; articles that describe “norms/rules” in detail• But this has negative impression (e.g., newcomers)

• Design Claim 20: Offering people reminders at the point of an action that may violate norms will reduce the number of offenses– E.g., Eudora’s MoodWatch s/w automatically cautions users who

are about to send an email containing flame words (Shankland 2000)


Encouraging voluntary complianceEnhancing compliance

• Design Claim 21: In more cohesive groups to which members are more committed, members will be more likely to spontaneously comply with the norms.– How to improve group cohesion?

• Design Claim 22: Community influence on rule making (i.e., participatory rule design; or collective choice) will increase compliance with the rules.– In general, externally imposed rules and monitoring tend to be

viewed as unfair and to lead to conflict (Ostrom 2000) – Management’s top-down “play nice” rules was not that helpful in a

multi-player game, Everquest (Yee 2001)


Encouraging voluntary complianceEnhancing compliance

• Design Claim 23. Face-saving ways to correct norm-violations will increase compliance.– Violators are often willing to stop bad behavior and

correct errors if they can do so without having to admit that they deliberately violated the norms; provide an opportunity of face saving

– Ex) “UYA notice in MIT network”; “someone using your account” “did you notice that…”

– Ex) Notification like “You may not have been aware of this guideline, but we have a stated policy of […]. No big deal, but please stick to this in the future (plead ignorance)


Encouraging voluntary complianceRewards and Sanctions

• Deterrence theory in criminology– (rational decision process) People with criminal disposition will

violate the rules only when it pays; i.e., by performing informal cost-benefit analysis (punishment vs. benefit)

– For spammers, manipulators and trolls, simply limiting effectiveness of their actions reduces the incentives to participate

• How to deter anti-normative actions?– Reducing benefits: spam protection, disemvoweling, etc. – Costly actions: social approbation, reputation management,

price/bonds/bets on actions, expensive pseudonyms– Monitoring/sanctioning: peer reporting + automatic detection,

graduated sanction



• Design Claim 24: (link spamming) Telling search engines not to follow links will discourage spammers from posting links.– E.g., nofollow attributes in Blog comments to prevent search engine spamming

• Design Claim 25: Verified identities and pictures will reduce the incidence of norm violations.– Social approbation (people are very sensitive to public impression); Goffman 1959;

under-reporting lying, stealing, drug use with “in-person” interviews (vs. online survey)– Anonymity and larger group size both lead to anti-normative behavior

• De-individuation in a group as being submerged in it (Festinger 1952); individuals are not seen or paid attention to as individuals (and not stand out)

• People are more likely to violate norms when they can conceal their identities under white robes (Zimbardo 1969)

– Relative anonymity of online communications reduces normative pressure (Bordia 1997)

• Design Claim 26: Reputation systems, which summarize the history of someone’s online behavior, help to encourage good behavior and deter norm violations.– Reputation may influence future interactions as well (e.g., posting articles, etc.)



• Design Claim 27: Prices, bonds, or bets that make undesirable actions more costly than desirable actions will reduce misbehavior.

• Handling cheap pseudonyms:– Design Claim 28: Increasing the benefits of participating with a long-

term identifier (due to reputation or some other reasons) will increase the community’s ability to sanction misbehavior.

– Design Claim 29: Imposing costs for or preventing pseudonym switching increases the community’s ability to sanction misbehavior.

– Design Claim 30: Forcing newcomers to post bonds that may be forfeited if the newcomers misbehave or forcing newcomers’ sponsors to stake their own reputations increases the community’s ability to sanction misbehavior



• Design Claim 31: Graduated sanctions increase the legitimacy and thus the effectiveness of sanctions.– Sanctions proportional to the offence are perceived as fair and legitimate (Ostrom 1990)– More persuaded to comply through expertise and judgment than commands and force

(Koslowsky 2001)– Graduated sanctions that begin with persuasion based on expertise and judgment and

proceed to more forceful measure can be effective

• Design Claim 32: Peer reporting or automatic detection of violations increases the deterrent effect of sanctions.– A mild but certain punishment is more effective than a severe but uncertain

punishment• Design Claim 33: Increased community cohesion, graduated sanctions, explicit

rules, identifiable perpetrators, formal sanctioning roles, and anti-retaliatory measures increase the likelihood that sanctions will be applied and thus increase the deterrent effect of sanctions.


Summary

• Limiting effects of bad behavior– Moderation, effective moderation, revision tools, filters/influence

limiters (reputation systems)• Coerced compliance: limits on bad behavior

– Activity quota, gags/bans, CAPTCHAs• Encouraging voluntary compliance

– Making norms clear and salient• Descriptive (explicit display) vs. injunctive norms (feedback)

– Enhancing compliance • Cohesive groups, collective choice, face-saving

– Rewards and sanctions• Link spam protection, identifiability, reputation, handling cheap pseudonym

Documents

Regulating Behavior KSE 652 Social Computing Systems Design Uichin Lee Sept. 26, 2013 Regulating behavior in online communities S. Kiesler, A. Kittur,