Copyright 2011 Digital Enterprise Research Institute. All rights reserved.

Digital Enterprise Research Institute www.deri.ie

Enabling Networked Knowledge

Arguments about deleting Wikipedia content

Jodi Schneiderjschneider@pobox.com

Vendredi 19th April 2013

1

Télécom ParisTech

Is Wikipedia Sustainable?

Deletion threatens Wikipedia

• 1 in 4 new Wikipedia articles is deleted – within minutes or hours

• Demotivating! – 1 in 3 newcomers start by writing a new article– 7X less likely to stay if their article is deleted!

• Can we support editor retention?

Ph.D. case study: argumentative dialogues about deleting Wikipedia articles

• Goals:– Understand collaboration & coordination– Identify “pain points” & new IT support opportunities

• Approaches:– Net-ethnography

• Interviews of community members• Embedded participation• Reading essays, policies, & written dialogues• Analysing article history, user contributions

– Content analysis• Departure point: grounded theory or existing categories. With multiple annotators,

iteratively refined annotation manual to achieve strong interannotator agreement.• Decision factors (WikiSym 2012)• Walton’s argumentation schemes (CSCW 2013)

– Prototyping & iterative design• Design (WikiSym 2012 demo)• User study (reported in dissertation)

Corpus

• Article deletion dialogues from English Wikipediastarted on a typical-volume day

• 72 dialogues (94 A4 pages)

Findings: pain points of article deletion

• Article creators• Novices visiting or newly joining Wikipedia• No-consensus dialogues

Article creators

• Misunderstand policy– “I do understand that articles on wikipedia need to be

sourced… it is due to have two [sources] once [our website goes] live”

• Express high levels of emotion– “To be honest it's been a real turn off adding articles to WP

and I don't think I will add articles again. So smile and enjoy.”

• Learn from discussions– “much as it would break my heart … it is perhaps sensible

that the piece is deleted.”Net-ethnography in 8th International Symposium on Wikis and Open Collaboration (WikiSym 2012)

Novices’ arguments

• Structurally different to experts’ arguments• More problematic arguments from novices

– Personal preference– Requesting a favor– Analogy to other cases– No harm in keeping an article– Large number of search engine hits

Argumentation schemes content analysis in 16th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2013)

No consensus discussions

“What works well is simply the community agreeing on a verdict.”

Otherwise:• Time-consuming & difficult to judge a case• Same case may get raised repeatedly• Emotional upset is more likely

– “messy”, “full of hate and pain” when overturned

Net-ethnography & interviews in 8th International Symposium on Wikis and Open Collaboration (WikiSym 2012)

Articulate criteria

Decision factors content analysis in 8th International Symposium on Wikis and Open Collaboration (WikiSym 2012)

4 Factors cover– 91% of

comments– 70% of

discussions

Factor Example (used to justify `keep')

Notability Anyone covered by another encyclopedic reference is considered notable enough for inclusion in Wikipedia.

Sources Basic information about this album at a minimum is certainly verifiable, it's a major label release, and a highly notable band.

Maintenance …this article is savable but at its current state, needs a lot of improvement.

Bias It is by no means spam (it does not promote the products).

Other I'm advocating a blanket "hangon" for all articles on newly- drafted players

Use criteria to augment interface

Prototype design (RDFa; custom ontology based on FOAF, SIOC)in WikiSym 2012 Demos

84% prefer our system

“Information is structured and I can quickly get an overview of the key arguments.”

“The ability to navigate the comments made it a bit easier to filter my mind set and to come to a conclusion.”

“It offers the structure needed to consider each factor separately, thus making the decision easier. Also, the number of comments per factor offers a quick indication of the relevance and the deepness of the decision.”

Based on a formative evaluation user study with 20 novice usersin dissertation “Enabling reuse of arguments and opinions from online social disputes”