Argumentation Mining in Persuasive Essays and Scientific Articles fromthe Discourse Structure Perspective

Christian Stab†, Christian Kirschner†‡, Judith Eckle-Kohler†‡ and Iryna Gurevych†‡

†Ubiquitous Knowledge Processing Lab (UKP-TUDA)Department of Computer Science, Technische Universitat Darmstadt

‡Ubiquitous Knowledge Processing Lab (UKP-DIPF)German Institute for Educational Research

www.ukp.tu-darmstadt.de

Abstract

In this paper, we analyze and discuss ap-proaches to argumentation mining fromthe discourse structure perspective. Wechose persuasive essays and scientific ar-ticles as our example domains. By an-alyzing several example arguments andproviding an overview of previous workon argumentation mining, we derive im-portant tasks that are currently not ad-dressed by existing argumentation miningsystems, most importantly, the identifica-tion of argumentation structures. We dis-cuss the relation of this task to automateddiscourse analysis and describe prelimi-nary results of two annotation studies fo-cusing on the annotation of argumentationstructure. Based on our findings, we derivethree challenges for encouraging future re-search on argumentation mining.

1 Introduction

Argumentation mining is a recent research areawhich promises novel opportunities not only forinformation retrieval, educational applications orautomated assessment tools but also aims at im-proving current legal information systems or pol-icy modeling platforms. It focuses on automat-ically identifying and evaluating arguments intext documents and includes a variety of sub-tasks like identifying argument components, find-ing accepted arguments and discovering argumen-tation structures. Researchers have already inves-tigated argumentation mining in several domains.For instance, Teufel (1999) aims at identifyingrhetorical roles of sentences in scientific articlesand Mochales-Palau and Moens (2011) identifyarguments in legal documents. Also, Feng andHirst (2011) investigated argumentation schemesin newspapers and court cases and Florou et al.

(2013) applied argumentation mining in policymodeling.

However, current approaches mainly focus onthe identification of arguments and their compo-nents and largely neglect the identification of ar-gumentation structures although an argument con-sists not only of a set of propositions but also ex-hibits a certain structure constituted by argumenta-tive relations (Peldszus and Stede, 2013; Sergeant,2013). We argue in this paper that identifying ar-gumentative relations and the argumentation struc-ture respectively is an important task for argu-mentation mining. First, identifying argumenta-tive relations between argument components en-ables the identification of additional reasons for agiven claim and thus allows the creation of valu-able knowledge bases e.g. for establishing newinformation retrieval platforms. Second, it is im-portant to recognize which premises belong to aclaim, since it is not possible to evaluate argu-ments without knowing which premises belong toit. Third, automatically identifying the structure ofarguments enables novel features of applications,such as providing feedback in computer-assistedwriting (e.g., recommending reasonable usage ofdiscourse markers, suggesting rearrangements ofargument components) or extracting argumenta-tion structures from scientific publications for au-tomated summarization systems.

In this paper, we analyze several examples ofargumentative discourse from the discourse struc-ture perspective.1 We outline existing approacheson argumentation mining and discourse analysisand provide an overview of our current work onargumentation structure annotation in scientific ar-ticles and persuasive essays. We conclude this pa-per with a list of challenges for encouraging future

1The examples are taken from persuasive essays whichare either collected from the writing feedback section ofhttp://www.essayforum.com or from the corpuscompiled by Stab and Gurevych (2014)

research on argumentation mining.

2 Background

Philosophy and Logic proposed a vast amount ofargumentation theories (e.g. Toulmin (1958), Wal-ton et al. (2008), Freeman (2011)).2 The major-ity of these theories generally agree that an ar-gument consists of several argument componentswhich can either be a premise or a claim. The sim-plest form of an argument includes one claim thatis supported by at least one premise (figure 1).

Claim Premisesupports

Figure 1: Illustration of a simple argument

The claim3 is the central component of an ar-gument that can either be true or false. Thus, theclaim is a statement that should not be accepted bythe reader without additional reasons. The secondcomponent of an argument, the premise4, under-pins the plausibility of the claim. It is usually pro-vided by the proponent (writer) for convincing thereader of the claim. Examples (1) and (2) illustratetwo simple arguments, each containing a claim (inbold face) and a single premise (underlined):

(1) “It is more convenient to learnabout historical or art items online.With Internet, people do not need totravel long distances to have a real lookat a painting or a sculpture, which prob-ably takes a lot of time and travel fees.”(2) “Locker checks should be mademandatory and done frequently be-cause they assure security in schools,make students healthy, and will makestudents obey school policies.”

These examples illustrate that there exist argu-ment components both on the sentence level andon the clause level.

Argumentative relations are usually directed re-lations between two argument components andrepresent the argumentation structure. There ex-ist different types like support or attack (Peldszus

2A review of argumentation theory is beyond the scope ofthis paper. A survey can be found in Bentahar et al. (2010)

3also called conclusion (Mochales-Palau and Moens,2009)

4sometimes called support (Besnard and Hunter, 2008) orreason (Anne Britt and Larson, 2003)

and Stede, 2013) which indicate that the source ar-gument component is a reason or a refutation forthe target component. For instance, in both of theexamples above, an argumentative support relationholds from the premise to the claim. The follow-ing example illustrates a more complex argumentincluding one claim and three premises:

(3) “Everybody should study abroada.It’s an irreplaceable experience if youlearn standing on your own feetb sinceyou learn living without depending onanyone elsec. But one who is livingoverseas will of course struggle withloneliness, living away from family andfriendsd.”

Figure 2 shows the structure of the argument in(3). In this example, premiseb supports the claima

whereas premised attacks the claima.

a b c d supports

supports

attacks

Figure 2: Argumentation structure of example (3).

This example illustrates three important proper-ties of argumentation structures:

1. Argumentative relations can hold betweennon-adjacent sentence/clauses, e.g. the ar-gumentative attack relation from premised tothe claima.

2. Some argumentative relations are signaled byindicators, whereas others are not. For in-stance, the argumentative attack relation frompremised to the claima is indicated by the dis-course marker ‘but’, whereas the argumenta-tive support relation from premiseb to claima

is not indicated by a discourse marker.

3. Argumentative discourse might exhibit rea-soning chains, e.g. the chain constituted be-tween argument components a, b, and c.

3 Argumentation Mining

Previous approaches on argumentation miningcover several subtasks including the separation ofargumentative from non-argumentative text units(Moens et al., 2007; Florou et al., 2013), theclassification of argument components (with dif-ferent component classes) (Rooney et al., 2012;

Mochales-Palau and Moens, 2009; Teufel, 1999;Feng and Hirst, 2011), and the identificationof argumentation structures (Mochales-Palau andMoens, 2009; Wyner et al., 2010).

3.1 Separation of Argumentative fromNon-argumentative Text Units

The first step of an argumentation mining pipelinetypically focuses on the identification of argu-mentative text units before analyzing the compo-nents or the structure of arguments. This taskis usually considered as a binary classificationtask that labels a given text unit as argumenta-tive or non-argumentative. One of the first ap-proaches was proposed by (Moens et al., 2007).They focus on the identification of argumentativetext units in newspaper editorials and legal doc-uments included in the Araucaria corpus (Reedet al., 2008). The annotation scheme utilized inAraucaria is based on a domain-independent ar-gumentation theory proposed by Walton (1996).A similar approach is reported by Florou et al.(2013). In their experiments, they classify textsegments crawled with a focused crawler as eithercontaining an argument or not. They focus on theidentification of arguments in the policy model-ing domain for facilitating decision making. Forthat purpose, they utilize several discourse mark-ers and features extracted from the tense and moodof verbs.

Although the separation of argumentative fromnon-argumentative text units is an important stepin argumentation mining, it merely enables the de-tection of text units relevant for argumentation anddoes not reveal the argumentative role of argumentcomponents.

3.2 Classification of Argument ComponentsThe classification of argument components aimsat identifying the argumentative role (e.g. claimsand premises) of argument components.

One of the first approaches to identify argumentcomponents is Argumentative Zoning proposed by(Teufel, 1999). Each sentence is classified as oneof seven rhetorical roles including e.g. claim, re-sult or purpose using structural, lexical and syn-tactic features. The underlying assumption of thiswork is that argument components extracted froma scientific article provide a good summary of itscontent. Rooney et al. (2012) also focus on theidentification of argument components but in con-trast to the work of Teufel (1999) their scheme is

not tailored to a particular genre. In their exper-iments, they identify claims, premises and non-argumentative text units in the Araucaria corpus.Feng and Hirst (2011) also use the Araucaria cor-pus for their experiments, but focus on the identi-fication of argumentation schemes (Walton, 1996)which are templates for arguments (e.g. argumentfrom example or argument from position to know).Since their approach is based on features extractedfrom mutual information of claims and premises,it requires that the argument components are re-liably identified in advance. Mochales-Palau andMoens (2009) report several experiments for clas-sifying argument components. They solely focuson the legal domain and in particular on legal courtcases from the European Court of Human Rights(ECHR). They consider the classification of argu-ment components as two consecutive steps. Theyutilize a maximum entropy model for identifyingargumentative text units before identifying the ar-gumentative role (claim and premise) of the identi-fied components using a Support Vector Machine.

3.3 Identification of ArgumentationStructures

Currently, there are only few approaches aimingat the identification of argumentation structures.For instance, the approach proposed by Mochales-Palau and Moens (2011) relies on a manuallycreated context-free grammar (CFG) and on thepresence of discourse markers for identifying atree-like structure between argument components.However, the approach relies on the presence ofdiscourse markers and exploits manually createdrules. Therefore, it does not accommodate ill-formatted arguments (Wyner et al., 2010) and isnot capable of identifying implicit argumentationstructures which are common in argumentativediscourse. Indeed, Marcu and Echihabi (2002)found that only 26% of the evidence relations inthe RST Discourse Treebank (Carlson et al., 2001)include discourse markers.

Another approach was presented by Cabrio andVillata (2012). They identify relations between ar-guments of an online debate platform for identify-ing accepted arguments and to support the interac-tions in online debates. In contrast to the work ofMochales-Palau and Moens (2011), this approachaims at identifying relations between arguments(macro-level) and not between argument compo-nents (micro-level).

4 Argumentation and Discourse Analysis

Discourse analysis aims at identifying discourserelations that hold between adjacent text units withtext units being sentences, clauses or nominaliza-tions (Webber et al., 2012). Since text units mightbe argument components and discourse relationsare often closely related to argumentative rela-tions, previous work in automated discourse anal-ysis is highly relevant for argumentation mining.

4.1 Discourse Relations and ArgumentativeRelations

Most previous work in automated discourse anal-ysis is based on corpora annotated with generaldiscourse relations, most notably the Penn Dis-course Treebank (PDTB) (Prasad et al., 2008)and the Rhetorical Structure Theory (RST) Dis-course Treebank (Carlson et al., 2003). WhereasRST represents the discourse structure as a tree,the PDTB allows more general graph structure.For the annotation of discourse relations in thePDTB, two different types of discourse relationswere distinguished: implicit and explicit relations.Whereas explicit discourse relations are indicatedby discourse markers, implicit discourse relationsare not indicated by discourse markers and theidentification of those relations requires more so-phisticated methods.

Take as an example the argumentation structurediscussed in section 2.

“Everybody should study abroada. It’san irreplaceable experience if you learnstanding on your own feetb since youlearn living without depending on any-one elsec. But one who is living over-seas will of course struggle with lone-liness, living away from family andfriendsd.”

Whereas the argument components b and c, aswell as c and d are related through the discoursemarker ‘since’ (signalling an explicit CAUSE rela-tion) and ‘but’ (signalling an explicit CONTRASTrelation), the discourse relation JUSTIFY betweena and b is an implicit relation.

Existing approaches of discourse analysis pro-posed different sets of discourse relations, andthere is currently no consensus in the literatureabout the ‘right’ set of discourse relations. Forinstance, the RST (Mann and Thompson, 1988)

uses a different set of discourse relations than thePDTB (Prasad et al., 2008).

It is still an open question how the proposed dis-course relations relate to argumentative relations.Although, there are preliminary findings that indi-cate that there are certain similarities (Cabrio etal., 2013), approaches like RST and PDTB aimat identifying general discourse structures and arenot tailored to argumentative discourse.

The difference of the relations is best illustratedby the work of Biran and Rambow (2011), whichis to the best of our knowledge the only approachthat focuses on the identification of distinct argu-mentative relations. The authors argue that exist-ing definitions of discourse relations are only us-able as a building block for argumentation miningand that there are no distinct argumentative rela-tions included in existing approaches. Therefore,they combine 12 relations from the RST DiscourseTreebank (Carlson et al., 2001) to a single argu-mentative support relation for identifying justifi-cations in online discussions.

4.2 Discourse Markers and Indicators ofArgumentative Relations

There is a large body of previous research in lin-guistics on the role of discourse markers, sig-nalling discourse relations (e.g.‘because’, ‘there-fore’, ‘since’, etc.) in discourse analysis. Mostprevious investigations of discourse markers arebased on the PDTB (Prasad et al., 2008) and on theRST Discourse Treebank (Carlson et al., 2003).

However, a critically discussed question in thiscontext is the definition of discourse markers. Arediscourse markers in the sense of indicators mark-ing discourse relations just words like ‘because’,‘therefore’, ‘since’? Taboada (2006) investigatesthe role of discourse markers in corpora annotatedwith discourse relations according to the RST. Inher discussion of related work on discourse mark-ers in linguistics, she concludes that there aremany lexical and linguistic devices signalling dis-course relations beyond discourse markers, suchas the mood (e.g. indicative or conjunctive) or themodality (e.g. possibility, necessity) of a sentence.

In particular, for argumentative discourse, therole of indicators, such as discourse markers, is notwell-understood yet, which is due to the lack ofcorpora annotated with argumentation structures.Recently, Tseronis (2011) summarized interme-diate results of a corpus-based analysis of argu-

mentative moves, aiming at the identification oflinguistic surface cues that act as argumentativemarkers. According to Tseronis (2011), any sin-gle or complex lexical expression can act as anargumentative marker, and it can either mark anargumentative relation (i.e., connecting two argu-ments or argument components) or signal a certainargumentative role, such as a claim or a premise.Moreover, he observed that also sequential pat-terns of argumentative markers indicate particularargumentative moves, for instance, first stating thecommon ground (e.g., using the marker it is un-derstandable ...) and then presenting an attack tothis common ground (e.g., using a marker such asnevertheless).

5 Argumentation Structure Annotation

Our research in argumentation mining is mo-tivated by the (1) information access and (2)computer-assisted writing perspective. Currently,we are conducting two annotation studies, focusedon analyzing argumentation structures in scientificarticles and persuasive essays. In the followingsubsections we provide an overview of the (pre-liminary) results.

5.1 Argumentation Structures in ScientificArticles

One of the main goals of any scientific publica-tion is to present new research results to an expertaudience. In order to emphasize the novelty andimportance of the research findings, scientists usu-ally build up an argumentation structure that pro-vides numerous arguments in favor of their results.The goal of this annotation study is to automati-cally identify those argumentation structures on afine-grained level in scientific publications in theeducational domain and thereby to improve infor-mation access. A potential use case could be anautomated summarization system creating a sum-mary of important arguments presented in a scien-tific article.

Up to now only coarse-grained approaches likeArgumentative Zoning (Teufel et al., 2009; Li-akata et al., 2012; Yepes et al., 2013) have beendeveloped for argumentation mining in scientificpublications. These approaches classify argumentcomponents according to their argumentative con-tribution to the document (see section 3.2) but theydo not consider any relations between the argu-ment components. To the best of our knowledge,

there is no prior work on identifying argumenta-tion structures on a fine-grained level in scientificfull-texts yet (see section 3.3).

Due to the lack of evaluation datasets, we areperforming an annotation study with four annota-tors, two domain experts and two annotators whodeveloped the annotation guidelines. Our datasetconsists of about 20 scientific full-texts from theeducational domain. For the annotation study,we developed our own Web-based annotation tool(see figure 3 for a screenshot). The annotationtool allows to label argument components directlyin the text with different colors and to add differ-ent relations (like support or attack) between ar-gument components. The resulting argumentationstructure is visualized as a graph (see figure 3).

Next, we plan to develop weakly supervisedmachine learning methods to automatically anno-tate scientific publications with argument compo-nents and the relations between them. The firststep will be to distinguish non-argumentative parts(for example descriptions of the document struc-ture) from argumentative parts (see section 3.1).The second step will be to identify support and at-tack relations between the argument components.In particular, we will explore lexical features, suchas discourse markers (for example ‘hence’, ‘so’,‘for that reason’, ‘but’, ‘however’, see section 4),and semantic features, such as text similarity ortextual entailment.

5.2 Identifying Argumentation Structures forComputer-Assisted Writing

The goal of computer-assisted writing is to pro-vide feedback about written language in orderto improve text quality and writing skills of au-thors respectively. Common approaches are forinstance focused on providing feedback aboutspelling and grammar, whereas more sophisti-cated approaches also provide feedback about dis-course structures (Burstein et al., 2003), readabil-ity (Pitler and Nenkova, 2008), style (Burstein andWolska, 2003) or aim at facilitating second lan-guage writing (Chen et al., 2012; Huang et al.,2012).

Argumentative Writing Support is a particu-lar type of computer-assisted writing that aims atproviding feedback about argumentation and thuspostulates methods for reliably identifying argu-ments. Besides the recognition of argument com-ponents, the identification of the argumentation

Figure 3: Screenshot of the annotation tool for argumentation structure annotation in scientific full-texts: The left side includes the text of a scientific article and the argument components marked withdifferent colors and labels (a1-a7). The graph visualization on the right side illustrates the argumentationstructure. Each node represents an argument component connected with several relations (‘support’,‘attack’, ‘sequence’).

structure is crucial for argumentative writing sup-port, since it would open novel possibilities forproviding formative feedback about argumenta-tion. On the one hand, an analysis of the argu-mentation structure would enable the recommen-dation of more meaningful arrangements of argu-ment components and a reasonable usage of dis-course markers. Both have been shown to increaseargument comprehension and recall, and thus thequality of the text (Anne Britt and Larson, 2003).On the other hand, by identifying which premisesbelong to a claim, it would be possible to advicethe author to add additional support in her/his ar-gumentation to improve the persuasiveness.

Following this vision, we conducted an anno-tation study with three annotators to model ar-gument components and the argumentation struc-ture in persuasive essays at the clause-level. Thecorpus includes 90 persuasive essays which weselected from essayforum.com. Our annotationscheme includes three argument components (ma-jor claim, claim and premise) and two argumen-tative relations (support and attack). For definingthe annotation guidelines and the annotation pro-cess we conducted a preliminary study on a cor-pus of 14 short text snippets with five non-trainedannotators and found that information about the

topic and the author’s stance is crucial for anno-tating arguments. According to these findings,we defined a top-down annotation process start-ing with the major claim and drilling-down to theclaims and the premises so that the annotators areaware of the author’s stance and the topic beforeannotating other components. Using this strategy,we achieved an inter-rater agreement of αU =0.725 for argument components and α = 0.81for argumentative relations indicating that the pro-posed scheme and annotation process successfullyguides annotators to substantial agreement. Formore details about this annotation study, we re-fer the interested reader to (Stab and Gurevych,2014), which includes a detailed description of theannotation scheme, an analysis of inter-annotatoragreements on different granularities and an er-ror analysis. The corpus as well as the annotationguidelines are freely available to encourage futureresearch.6

5We used Krippendorff’s αU (Krippendorff, 2004) formeasuring the agreement since there are no predefined mar-bles in our study and annotators had also to identify theboundaries of argument components.

6http://www.ukp.tu-darmstadt.de/data/argumentation-mining

6 Challenges

Existing approaches of argumentation miningmainly focus on the identification of argumentcomponents (section 3). Based on the examplesanalyzed in section 2 and on the experience gainedin our annotation studies (section 5), we identifiedthe following challenges for future research in ar-gumentation mining that have not been addressedadequately by previous work.

Segmentation: Most of the existing approachesare based on the sentence-level. However, for an-alyzing arguments, a more fine-grained segmenta-tion is needed (Sergeant, 2013). Apart from thesentence level, in real world data argument com-ponents exist on the clause level or can spread overseveral sentences. For instance, example (4) il-lustrates that a single sentence can contain multi-ple argument components (claim in bold face andpremise underlined) (see also example (2) in sec-tion 2). In example (5) the premise consists of twosentences, because both sentences are needed torepresent and support the “different opinions” inthe claim.

(4) “Eating apples is healthy which hasto do with substrates which prevent can-cer and other diseases.”(5) “There are different opinions aboutcoffee. Some people say they need it tostay awake. Other people think it’s un-healthy.”

It is an open question if existing segmentationapproaches can be used for reliably identifying theboundaries of argument components. In example(4) we find two times the word “which”. Thismakes it hard for a segmenter to split the sentencecorrectly in only two parts. On the other hand,the combination of sentences (example (5)) alsorequires more elaborated techniques that are ableto identify sentences that are related and only formin combination the support of a particular claim.

Context Dependence: The context is crucialfor identifying arguments, their components andargumentation structures. As illustrated by Staband Gurevych (2014), it is even a hard task for hu-man annotators to distinguish claims and premiseswithout being aware of the context. For instance,the following three argument components consti-tute a reasoning chain in which c is a premise forb and b a premise for a:

(6) “Random locker checks should bemade obligatory.a Locker checks helpstudents stay both physically and men-tally healthy.b It discourages studentsfrom bringing firearms and especiallydrugs.c”

In this argumentation structure, a can be clas-sified as a claim. However, without being awareof the argument component a, b becomes a claimwhich is supported by premise c. The same situa-tion can be found in example (3) in section 2. If welook at the argument components b and c in isola-tion, we can classify b as claim. However, lookingat the whole example, the argument component ais the claim, supported by the premise b. The sameholds for the argument components c and a whichwould be connected by a support relation if theyare considered in isolation. Both examples illus-trate that the context is crucial for classifying ar-gument components as claims or premises and foridentifying the argumentation structure. Although,Stab and Gurevych (2014) proposed an annotationprocess that facilitates these decisions in manualannotation studies of persuasive essays, it is stillan open issue how to model the context in order toimprove the performance of automatic argumenta-tion mining methods.

Ambiguity of Argumentation Structures:The most important challenge for identifying argu-mentation structures is ambiguity, since there areoften several possible interpretations of argumen-tation structures which makes it hard or even im-possible to identify one correct interpretation. Inprevious examples, we have already seen that theclassification of argument components depends onthe context and the considered argument compo-nents respectively. However, even if we considerall components of an argument, there might beseveral reasonable interpretations of its structure.For instance, the structure of example (6) can beinterpreted in three different ways (figure 4). In thefirst interpretation, the argument component c sup-ports argument component b and argument com-ponent b supports argument component a, whereasin the second interpretation argument componentsb and c both support argument component a. Thethird interpretation contains all possible argumen-tative relations from the first and second interpre-tation combined, and thus represents a graph struc-ture (in contrast to a tree structure).

The ambiguity of argumentation structures rep-

a

b c supportsupport

a

b c supportsupport

support

a

b

c

support

support

Figure 4: Several interpretations of the argumen-tation structure of example (6).

resents a major challenge for argument anno-tation studies and consequently the creation ofreliable gold standards for argumentation min-ing. In all annotation studies we know, exactlyone annotation is considered to be correct whichmeans that other possibly correct interpretationsare considered as incorrect and therefore down-grade the results for the inter annotator agree-ment and the performance of automatic classi-fiers. Consequently, it might be interesting toexplore different evaluation methods. For in-stance, evaluation schemes used in automatic textsummarization could be considered as an alterna-tive. In text summarization, inter annotator agree-ment for human-generated summaries is particu-larly low, and hence, each human-generated sum-mary is considered valid for evaluating an auto-matic summarization system (Nenkova and McK-eown, 2012).

7 Conclusion

In this paper, we showed that existing approachesto argumentation mining mainly focus on the iden-tification of argument components and largely ne-glect the identification of argumentation struc-tures, although this task is crucial for manypromising applications, e.g., for building novel ar-gument related knowledge bases. By examiningseveral examples, we derived characteristic prop-erties of argumentation structures. We discussedthe relation of discourse analysis and argumen-tation structure and showed that previous worksin discourse analysis are not capable of identify-ing argumentation structures, because discourserelations do not cover all argumentative relationsand are limited to relations between adjacent textunits. Based on our observations, we derived threechallenges for encouraging future research, i.e.,(i) identifying the boundaries of argument compo-nents, (ii) modeling the context of argument com-ponents and argumentative relations, and (iii) ad-

dressing the problem of ambiguous argumentationstructures. In particular, the ambiguity of argu-mentation structure poses an important issue forfuture work.

ReferencesM. Anne Britt and Aaron A. Larson. 2003. Construct-

ing representations of arguments. Journal of Mem-ory and Language, 48(4):794–810.

Jamal Bentahar, Bernard Moulin, and MichelineBelanger. 2010. A taxonomy of argumentationmodels used for knowledge representation. Artifi-cial Intelligence Review, 33(3):211–259.

Philippe Besnard and Anthony Hunter. 2008. El-ements of argumentation, volume 47. MIT pressCambridge.

Or Biran and Owen Rambow. 2011. Identifying jus-tifications in written dialogs by classifying text asargumentative. International Journal of SemanticComputing, 05(04):363–381.

Jill Burstein and Magdalena Wolska. 2003. TowardEvaluation of Writing Style: Finding Overly Repet-itive Word Use in Student Essays. In Proceedingsof the Tenth Conference on European Chapter of theAssociation for Computational Linguistics - Volume1, EACL ’03, pages 35–42, Budapest, Hungary.

Jill Burstein, Daniel Marcu, and Kevin Knight. 2003.Finding the WRITE Stuff: Automatic Identificationof Discourse Structure in Student Essays. IEEE In-telligent Systems, 18(1):32–39.

Elena Cabrio and Serena Villata. 2012. Natural lan-guage arguments: A combined approach. In Pro-ceedings of the 20th European Conference on Artifi-cial Intelligence, ECAI ’12, pages 205–210, Mont-pellier, France.

Elena Cabrio, Sara Tonelli, and Serena Villata. 2013.From discourse analysis to argumentation schemesand back: Relations and differences. In Joao Leite,TranCao Son, Paolo Torroni, Leon Torre, and Ste-fan Woltran, editors, Computational Logic in Multi-Agent Systems, volume 8143 of Lecture Notes inComputer Science, pages 1–17. Springer Berlin Hei-delberg.

Lynn Carlson, Daniel Marcu, and Mary EllenOkurowski. 2001. Building a discourse-tagged cor-pus in the framework of rhetorical structure theory.In Proceedings of the Second SIGdial Workshop onDiscourse and Dialogue - Volume 16, SIGDIAL ’01,pages 1–10, Aalborg, Denmark.

Lynn Carlson, Daniel Marcu, and Mary EllenOkurowski, 2003. Building a discourse-tagged cor-pus in the framework of rhetorical structure theory,chapter 5, pages 85–112. Springer.

Mei-Hua Chen, Shih-Ting Huang, Hung-Ting Hsieh,Ting-Hui Kao, and Jason S. Chang. 2012. FLOW:A First-language-oriented Writing Assistant Sys-tem. In Proceedings of the ACL 2012 SystemDemonstrations, ACL ’12, pages 157–162, Jeju Is-land, Korea.

Vanessa Wei Feng and Graeme Hirst. 2011. Classi-fying arguments by scheme. In Proceedings of the49th Annual Meeting of the Association for Com-putational Linguistics: Human Language Technolo-gies - Volume 1, HLT ’11, pages 987–996, Strouds-burg, PA, USA. Association for Computational Lin-guistics.

Eirini Florou, Stasinos Konstantopoulos, AntonisKoukourikos, and Pythagoras Karampiperis. 2013.Argument extraction for supporting public policyformulation. In Proceedings of the 7th Workshop onLanguage Technology for Cultural Heritage, SocialSciences, and Humanities, pages 49–54, Sofia, Bul-garia, August. Association for Computational Lin-guistics.

James B. Freeman. 2011. Argument Structure: Repre-sentation and Theory, volume 18 of ArgumentationLibrary. Springer.

Chung-chi Huang, Ping-che Yang, Mei-hua Chen,Hung-ting Hsieh, Ting-hui Kao, and Jason S. Chang.2012. TransAhead: A Writing Assistant for CATand CALL. In Proceedings of the Demonstrations atthe 13th Conference of the European Chapter of theAssociation for Computational Linguistics, EACL’12, pages 16–19, Avignon, France.

Klaus Krippendorff. 2004. Measuring the Reliabilityof Qualitative Text Analysis Data. Quality & Quan-tity, 38(6):787–800.

Maria Liakata, Shyamasree Saha, Simon Dobnik,Colin Batchelor, and Dietrich Rebholz-Schuhmann.2012. Automatic recognition of conceptualizationzones in scientific articles and two life science ap-plications. Bioinformatics, 28(7):991–1000.

William C. Mann and Sandra A. Thompson. 1988.Rhetorical structure theory: Toward a functional the-ory of text organization. Text, 8(3):243–281.

Daniel Marcu and Abdessamad Echihabi. 2002. Anunsupervised approach to recognizing discourse re-lations. In Proceedings of the 40th Annual Meetingon Association for Computational Linguistics, ACL’02, pages 368–375.

Raquel Mochales-Palau and Marie-Francine Moens.2009. Argumentation mining: The detection, classi-fication and structure of arguments in text. In Pro-ceedings of the 12th International Conference on Ar-tificial Intelligence and Law, ICAIL ’09, pages 98–107, New York, NY, USA. ACM.

Raquel Mochales-Palau and Marie-Francine Moens.2011. Argumentation mining. Artificial Intelligenceand Law, 19(1):1–22.

Marie-Francine Moens, Erik Boiy, Raquel MochalesPalau, and Chris Reed. 2007. Automatic detectionof arguments in legal texts. In Proceedings of the11th International Conference on Artificial Intelli-gence and Law, ICAIL ’07, pages 225–230, NewYork, NY, USA. ACM.

Ani Nenkova and Kathleen McKeown. 2012. A surveyof text summarization techniques. In Mining TextData, pages 43–76.

Andreas Peldszus and Manfred Stede. 2013. FromArgument Diagrams to Argumentation Mining inTexts: A Survey. International Journal of Cogni-tive Informatics and Natural Intelligence (IJCINI),7(1):1–31.

Emily Pitler and Ani Nenkova. 2008. Revisitingreadability: A unified framework for predicting textquality. In Proceedings of the Conference on Em-pirical Methods in Natural Language Processing,EMNLP ’08, pages 186–195.

Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and BonnieWebber. 2008. The Penn Discourse TreeBank 2.0.In Proceedings of the Sixth International Conferenceon Language Resources and Evaluation (LREC’08),pages 28–30, Marrakech, Morocco.

Chris Reed, Raquel Mochales-Palau, Glenn Rowe, andMarie-Francine Moens. 2008. Language resourcesfor studying argument. In Proceedings of the SixthInternational Conference on Language Resourcesand Evaluation, LREC ’08, pages 2613–2618, Mar-rakech, Morocco.

Niall Rooney, Hui Wang, and Fiona Browne. 2012.Applying kernel methods to argumentation min-ing. In Proceedings of the Twenty-Fifth Interna-tional Florida Artificial Intelligence Research So-ciety Conference, FLAIRS ’12, pages 272–275,Marco Island, FL, USA.

Alan Sergeant. 2013. Automatic argumentation ex-traction. In Proceedings of the 10th European Se-mantic Web Conference, ESWC ’13, pages 656–660,Montpellier, France.

Christian Stab and Iryna Gurevych. 2014. Annotat-ing argument components and relations in persua-sive essays. In Proceedings of the 25th InternationalConference on Computational Linguistics (COLING2014), page (to appear), Dublin, Ireland, August.

Maite Taboada. 2006. Discourse Markers as Sig-nals (or Not) of Rhetorical Relationsteu. Journal ofPragmatics, 38(4):567–592.

Simone Teufel, Advaith Siddharthan, and Colin Batch-elor. 2009. Towards discipline-independent ar-gumentative zoning: evidence from chemistry andcomputational linguistics. In Proceedings of the2009 Conference on Empirical Methods in NaturalLanguage Processing, pages 1493–1502.

Simone Teufel. 1999. Argumentative Zoning: Infor-mation Extraction from Scientific Text. Ph.D. thesis,University of Edinburgh.

Stephen E. Toulmin. 1958. The uses of Argument.Cambridge University Press.

Assimakis Tseronis. 2011. From connectives to argu-mentative markers: A quest for markers of argumen-tative moves and of related aspects of argumentativediscourse. Argumentation, 25(4):427–447.

Douglas Walton, Chris Reed, and Fabrizio Macagno.2008. Argumentation Schemes. Cambridge Univer-sity Press.

Douglas N Walton. 1996. Argumentation schemes forpresumptive reasoning. Routledge.

Bonnie Webber, Mark Egg, and Valia Kordoni. 2012.Discourse structure and language technology. Natu-ral Language Engineering, 18:437–490, 10.

Adam Wyner, Raquel Mochales Palau, Marie-FrancineMoens, and David Milward. 2010. Approaches totext mining arguments from legal cases. In SemanticProcessing of Legal Texts, volume 6036 of LectureNotes in Computer Science, pages 60–79. Springer.

Antonio Jimeno Yepes, James G. Mork, and Alan R.Aronson. 2013. Using the argumentative structureof scientific literature to improve information access.In Proceedings of the 2013 Workshop on BiomedicalNatural Language Processing, pages 102–110.