Eran Banin. Background User-generated content is critical to the success of any online platforms...
51
Antisocial Behavior in Online Discussion Communities Eran Banin
Eran Banin. Background User-generated content is critical to the success of any online platforms (CNN, Facebook, StackOverflow). These sites engage their
Why Robots CommunicateEran Banin
1
Background
User-generated content is critical to the success of any online
platforms (CNN, Facebook, StackOverflow).
These sites engage their users by allowing them to contribute and
discuss content, strengthening their sense of ownership and
loyalty.
While most users tend to be civil, others may engage in antisocial
behavior, negatively affecting other users and harming the
community.
2
Examples-
3
Mechanisms for preventing
Moderator - person who is responsible for a newsgroup or mailing
list on the Internet and for checking the messages before sending
them to the group.
Up & Down voting.
Blocking users.
Yet, antisocial behavior is a significant problem that can result
in offline harassment and threats of violence.
4
Past research
Despite its severity and prevalence, surprisingly little is known
about online antisocial behavior.
Most research reports qualitative studies that focus on
characterizing antisocial studying the behavior of a small number
users in specific communities.
In order to create new methods for identifying undesirable users,
large-scale and longitudinal analysis (of the phenomenon) is
required.
5
Present:
Past:
Rely on a community and its moderators to decide who they consider
to be disruptive and harmful
Antisocial behavior has been widely discussed in past literature in
order to formally define the behavior.
6
Present:
Past:
Large-scale data-driven analysis with the goal of obtaining
insights and developing tools for the early detection of trolls.
Moreover, the research provide prior study of the effects of
community feedback on user behavior.
Research has tended to be largely qualitative, generally involving
deep case study analyses of a small number of manually-identified
trolls.
7
Present:
Past:
Predict whether individual users will be subsequently banned from a
community based on their overall activity. Also show how our models
generalize across multiple communities.
Focus on detecting vandalism with respect for their language and
reputation. Other works tried to identified undesirable comments
based on their relevance to the discussed article and the presence
of insults.
8
Questions
1. Are there users that only become antisocial later in their
community life, or is deviant behavior innate?
2. Does a community’s reaction to users’ antisocial behavior help
them improve, or does it instead cause them to become more
antisocial?
3. Can antisocial users be effectively identified early on?
9
Contents
Source research
18 months, 1.7 million users contributed nearly 40 million posts
and more than 100 million votes.
Members that repeatedly violate community norms are eventually
banned permanently. Such individuals are clear instances of
antisocial users, and constitute “ground truth” in our
analyses.
11
List of posts in the same article
In addition, complete time-stamped trace of user activity from
March 2012 to August 2013, as well as a list of users that were
banned from posting in these communities.
12
Measuring undesired behavior
On a discussion forum, undesirable behavior may be signaled in
several ways:
down-vote
comment
ban a user from ever posting again in the forum.
However, down-voting may signal disagreement rather than
undesirability. Further, one would need to define arbitrary
thresholds needed to label a user as antisocial.
In contrast, we find that post deletions are a highly precise
indicator of undesirable behavior, as only community moderators can
delete posts.
13
Measuring undesired behavior
Bans are similarly strong indicators of antisocial behavior, as
only community moderators can ban users.
Thus, we focus on users who moderators have subsequently banned
from a community .
While such an approach does not necessarily identify every
antisocial user, this results in a more precise set of users who
were explicitly labeled as undesirable.
Filter by-
Apart from insults and profanity,
these include repeated attempts to bait users (“Ouch, ya
got me. What’s Google?”), provoke arguments (“Liberalism
truly breeds violence...this is evidence of that FACT”), or
derail
discussions (“All I want to know is...was there a broom
involved in any shape or form?”).
14
Matching FBUs and NBUs
We note that FBUs tend to post more frequently than average
users.
For example, on CNN, a typical FBU makes 264 posts, but an average
user makes only 22 posts.
To control for this large disparity, we use matching - statistical
technique used to support causality claims in observational
studies, to control for the number of posts a user made and the
number of posts made per day.
15
Unbiased measure of the quality or appropriateness of a post
Dictionary-based approaches may miss non-dictionary words
classifier trained on the text of deleted posts may confound post
content and community bias
Thus, we instead obtained human judgments of the appropriateness of
a specific post.
Collect post of random FBU & NBU and asked workers for evaluate
how appropriate is a post (on a scale of 1 to 5). Each post was
labeled by three independent workers, and their ratings
averaged.
131 workers completed these tasks, and they rated deleted posts
significantly lower than non-deleted posts (2.4 vs. 3.0)
16
Measuring text quality
Using these labeled posts, we then trained a logistic regression
model on text bigrams to predict the text quality of a post. Posts
with a rating higher than 3 were labeled as appropriate, and the
other labeled as inappropriate.
Under ten-fold cross validation, the AUC attained by this
classifier was 0.70.
This suggests that while the classifier is able to partially
capture this human decision making process and allow us to observe
overall trends in the data, other factors may play a significant
role in determining whether a post is appropriate.
17
Understanding Antisocial Behavior
To understand antisocial behavior in the context of a discussion
community, we first characterize how users who are banned differ
from those who are not in the terms of how they write and how they
act in a community.
Then, we analyze changes in behavior over the lifetimes of these
users to understand the effects of post quality, community bias,
and excessive censorship.
18
Differences in how FBUs & NBUs write
The similarity of a post to previous posts in a same thread may
reveal how users are contributing to a community.
Here we compare the average text similarity of a user’s post with
the previous three posts in the same thread
FBUs make less of an effort to integrate or stay on-topic
Post deletion is weakly negatively correlated with text similarity,
suggests that off-topic posts are more likely to be deleted
Differences in how FBUs & NBUs write
Next, we measure each post with respect to several readability
tests, including the Automated Readability Index (ARI), which are
designed to gauge how understandable a piece of text is.
FBUs appear to be less readable than those written by NBUs
20
Differences in how FBUs & NBUs write
Prior research also suggests that trolls tend to make inflammatory
posts.
FBUs are less likely to use positive words
Use less conciliatory language
More likely to swear
How do FBUs generate activity around themselves
Do FBUs purposefully try to create discussions, or
opportunistically respond to an on-going discussion?
While FBUs may either create or contribute to (already existing)
discussions depending on the community, they generally get more
replies from other users, and concentrate on fewer threads.
22
Evolution over time
how does FBU’s behavior and the community’s perception of them
change over time?
The rate of post deletion increases over time for FBUs, but is
effectively constant for NBUs.
23
Evolution over time
The increase in the post deletion rate could have two causes-
1) A decrease in posting quality
2) An increase in community bias
Text quality is decreasing over time, suggesting that 1) may be
supported
24
Evolution over time
Random Select of 400 FBUs & NBUs and from each user sampled a
random post from the first 10% or last 10% of their entire posting
history.
The posts presented to examiners who rated the appropriateness off
each post
FBUs start out writing worse than NBUs and worsen more than NBUs
over the course of their life.
25
Community tolerance over time
For testing it, random posts with similar predicted text quality
were matched to pairs. Each pair contain one post from the first
10% of a user’s life, and one from the final 10%.
Wilcoxon Signed-rank performed for check which post is more likely
to be deleted.
Results - among FBUs, found a significant effect of post time in
all communities, mean that a post was made in the last 10% of an
FBU’s life is more likely to be deleted in contrast to NBUs.
26
Does draconian post deletion policy could exacerbate undesirable
behavior?
For testing it, users with at least 10 posts were taken and divided
into 2 groups : 4/5 post s deleted among their 5 first posts and
the others.
Pairs were matched based on mean text quality of their 5 first
posts and the compared by the last 5.
Here, a Wilcoxon Signed-rank test shows a significant effect of the
deletion rate – mean that “unfairly” deletion cause worsen
writing.
27
Types of Antisocial Users
The following figure suggest that there are 2 kinds of FBUs-
Hi-FBU - proportion of deleted posts above 0.5
Lo-FBU - proportion of deleted posts below 0.5
Across all communities, the number of users in each population is
split fairly equally between the two groups.
28
use language that is less accommodating
receive more replies
Their post deletion rate starts high and remains high
In contrast, Lo-FBUs tend to have a constant low rate, until the
second half of their life.
29
Increasing reasons
The text similarity of these users’ posts with other posts in the
same thread is significantly lower across their last five
posts.
Additionally, they start to post more frequently, and in fewer
threads later in their life.
Thus, a combination of a large number of less relevant posts in a
short period of time potentially makes them more visible to other
members of the community.
30
Antisocial Behavior in Two Phases
Attempt to characterize this change over a user’s lifetime by
splitting it into two halves, with the goal of understanding how
users may change across them.
Fit two linear regression lines to a user’s post deletion rate over
time, one for each half of the user’s life obtained by bucketing
posts into tenths .
Computing (m1,m2) – the slope in first and second half
respectively.
31
Antisocial Behavior in Two Phases
By plotting the points we can identify quadrants corresponding to
how the deletion rate of a user’s posts changes over time:
fraction of users who are getting worse is higher for FBUs
fraction of users who are improving is higher for NBUs
32
When looking only on users with high initial deletion rates:
high proportion of NBUs -
many users who should be banned are in fact not
fraction of users who are improving is higher for NBUs
33
Main goal
In the communities we studied, FBUs tend to live for a long time
before actually getting banned . communities response slowly to
toxic users.
For example , on CNN, FBU write an average of 264 posts (over 42
days), with 124 posts deleted before they are finally banned.
Therefore, our currently main goal, is to build tools for automatic
early identification of potently FBUs.
34
Factors for identify FBUs
Motivated by our observations and insights, we could begin by
designing features for identification –
Post: Number of words, Readability metrics (ARI), LIWC - content
analysis and text-mining software.
Activity: number of posts per day/thread ,fraction of replied
posts, votes up & down.
Community: votes received per post, fraction of up-votes received,
fraction of posts reported, number of replies per post.
Moderator: fraction of posts deleted, slope and intercept of linear
regression lines (i.e. m1,m2).
35
36
Rationale – the combination of learning models increase the
classification accuracy.
Main idea – To average noisy and unbiased models to create a model
with low variance .
Random forest tree works as a large collection of decorrelated
decision trees.
40
Random forest classifier
Algorithm- assume we have set of features (F1,…,Fn) and dataset for
training (X1,…,Xm)-
Create the following matrix :
Chose subset of rows from the matrix and create decision
tree.
For testing X’ , train on all the trees and average the
results.
F1(x1) F2(x1) … Fn(x1) X1
K-fold cross validation
The original sample is randomly partitioned into k equal
sized subsamples.
One of the k subsamples is retained as the validation
data for testing the model, and the remaining k − 1
subsamples are used as training data.
The k results from the folds can then be
averaged.
42
Area under the ROC curve
In order to measure the classifier performance AUC technique been
used:
43
Performance seem to peak near to 10 post.
In case user make to mach posts, it’s difficult to predict.
45
Prediction performance change over “distance” from getting
banned
It becomes increasingly difficult to predict whether a user will
get banned the further in time the examined posts are from when the
user gets banned
46
47
Conclusions
The paper presents a data-driven study of antisocial behavior in
online discussion communities by analyzing users that are
eventually banned from a community.
Leads to characterization of antisocial users and to an
investigation of the evolution of their behavior and of community
response
Also, the paper proposes a typology of antisocial users based on
post deletion rates.
Finally, introduces a system for identifying undesired users early
on in their community life.
48
Potentially improvements
A more fine-grained labeling of users may reveal a greater range of
behavior and lead to better performance.
A better analysis of the content of posts, and of the relation
between the posts in a thread.
Expend the analysis to temporary banned users and different
communities.
Research for understanding how antisocial users may steer
individual discussions
49
While we present effective mechanisms for identifying and weeding
antisocial users, taking extreme action against small infractions
can exacerbate antisocial behavior.
Though average classifier precision is relatively high (0.80), one
in five users identified as antisocial are nonetheless
misclassified.
Possible solutions -
Trading off overall performance for lower rate of such
mistake.
Employ a human moderator for approve any bans.
Giving antisocial users a chance to redeem themselves.
50