R. Galgani 2014

Writing feedback & assessment using meta-language and can-do statements

Riccardo Galgani

R. Galgani 2014

Introduction

• Aims.• Motivation.• Outline.• Tasks.

R. Galgani 2014

Importance of Writing

• writing is generally recognised as being the de facto need at university (Donohue, 2012)

• students need to write academic texts in order to be successful at university (Hyland, 2006)

• writing is what, invariably, presents most difficulties for them [students] (Evans & Morrison, 2011:391)

R. Galgani 2014

Importance of Writing

• We've put this Economic and Social History writing skills site together because how you write will impact on your degree grade.....developing good writing skills is crucial to success at university and beyond.

http://khios.dcs.gla.ac.uk/writing/course/view.php?id=18

R. Galgani 2014

Importance of Writing

• How important on a scale of 0-5 are the following for success in your studies at Glasgow? 5 is very important and 0 is not important at all.

Writing __ Speaking __ Listening __ Reading __

R. Galgani 2014

Importance of Writing

0 1 2 3 4 5

Writing 1 2 11

Speaking 2 2 5 4 1

Listening 4 1 1 5 2 1

Reading 1 6 5 2

R. Galgani 2014

What writing?

• An academic text is not a unitary item (Van de Poel & Gasiorek, 2012: 295).

Task 2

R. Galgani 2014

Writing at University

R. Galgani 2014

The essay

• ‘Idea of the nature of the essay will influence the questions he or she (students) asks (or omits to ask) before tackling it’ (Thorp, 1991:113).

Task 3

R. Galgani 2014

IELTS essay

• IETLS task 2 “typically requires students to draw upon their personal experience for examples to support claims that they make ....” (Banerjee & Wall, 2006; 54)

R. Galgani 2014

The academic essay

• IELTS 2 “typically requires students to draw upon their personal experience for examples to support claims that they make rather than to draw on reading sources or primary data” (Banerjee & Wall, 2006; 54).

• Writing an essay using sources (Delaney, 2008)

R. Galgani 2014

The academic essay

• Moore & Morton (2005) concluded that the essay, understood as ‘the presentation of an argument in response to a given proposition or question’ (ibid: 50), is the most common task (60%).

• ‘a process of argumentation, a connected series of statements intended to establish a position and implying response to another (or more than one) position’ (Toulmin, Reike, and Janik 1984: p. 14).

• ‘to analyse and evaluate content knowledge’ (Wu, 2006: 330)

R. Galgani 2014

The academic essay

• “relevant components of writing ability” (Alderson, 2005:155-156) so as to teach them more effectively.

• “each formative judgement contributes to the summative checklist” (Banerjee 2006; 64).

R. Galgani 2014

The academic essay

• How/when are these taught?

• Formative

• Writing essay with target features.

• Summative

Target Language

Features of Academic

EssayAims

Priorities/Objectives

Feedback

R. Galgani 2014

Feedback

• Little consensus as to how to identify, let alone measure (assess), these components (Kim, 2011; Fox, 2009).

R. Galgani 2014

Feedback• Written corrective feedback (CF), otherwise known as error correction or

grammar correction (see Truscott, 1996 , 2007 ), has been a controversial topic in second language (L2) teaching for several years. .... From a theoretical or research perspective, there are ongoing disagreements about methodology, terminology, and interpretation of results. Meanwhile, real-world teachers struggle to help their students write more effectively, and, in some instances, students fail to meet practical goals because of their lack of progress in producing more linguistically accurate texts.

• Sub-questions include how many features (and which ones) should be examined in one treatment or study, whether the feedback should be implicit or explicit, and, if explicit, how much meta-linguistic explanation is necessary. (Ferris, 2010)

Task 4

R. Galgani 2014

Feedback Decisions

• Explicit.

• Meta-language.

• Focusing on specific items - recycling.

• Cognitively challenging.Task 5

R. Galgani 2014

Classroom Research 1

• Gave students a criterion-referenced checklist at the beginning of the course.

• Alerting students to features of target context by introducing meta-language.

• Establishing class objectives based on priorities.

R. Galgani 2014

Classroom Research 2

• Revision of original assessment of ability, that is, how accurate had they been given knowledge acquired during course - learning.

• What they thought on exit about current ability - improvement.

R. Galgani 2014

Results 1

R. Galgani 2014

Results 1

R. Galgani 2014

Results 1 – Priorities

General Organisation Use of sources Sentence Variety Academic Style0

2

4

6

8

10

12

14

16

18

20

a

b

c

d

e

R. Galgani 2014

Results 2 – Revisiting Checklist

R. Galgani 2014

Improvement Results

1. General

2. Orga

nisation

3. Use

of source

s

4. Sentence

varie

ty

5. Aca

demic sty

le02468

10121416

abcde

R. Galgani 2014

Learning Results

1.General

2. Orga

nisation

3. Use

of source

s

4. Sentence

varie

ty

5. Aca

demic Sty

le02468

1012141618

abcde

R. Galgani 2014

Discussion - Improvement

1. Students were generally very positive about what they felt they could do at beginning.

2. First drafts didn’t reflect this level of confidence.

3. Divergence between their sense of what they thought they could do and what their actual levels were.

R. Galgani 2014

Discussion - Learning

• Knowledge of meta-language – learning.

• Revised down.

• Exit level of ability – improvement.

R. Galgani 2014

Conclusions

• Where learning had taken place.

• Showed that learning had taken place at least in terms of awareness of what these terms meant.

• Ability.

• Limits – control group/hybrid.

R. Galgani 2014

Conclusion

• “Broad (2000) argues against absolute reliability in writing assessment because language is too complex to be narrowed down to objective analyses of measurable moves” (Baker, 2013).

• “a higher return to language teacher input can be achieved by a shift in teaching emphasis towards wider questions of essay structure in general and the art of logical presentation in particular” (Wall et al 1998)

R. Galgani 2014

Bibliography• Alderson, J. Charles, Clapham, C & Wall, D, 1995. Language Test Construction and

Evaluation. Cambridge: CUP.• Banerjee, Jayanti & Dianne Wall. 2006. “Assessing and reporting performance on pre-

sessional EAP courses: Developing a final assessment checklist and investigating its validity”. Journal of English for Academic Purposes 5: 50-69.

• Carroll, S. and Swain, M. 1993. Explicit and implicit negative feedback. Studies in Second Language Acquisition, 15 (03), pp. 357--386.

• Delaney, Yuly Asencion. 2008. “Investigating the reading-to-write construct”. Journal of English for Academic Purposes 7: 140-150.

• Donohue, James P. & Elizabeth J. Erling. 2012.”Investigating the Relationship between the use of English for Academic Purposes and Academic Attainment”. Journal of English for Academic Purposes 11: 210-219.

• Evans, Stephen & Bruce Morrison. 2011. “The first term at university: implications for EAP”. ELT Journal 65 (4): 387-397. Journal of English for Academic Purposes 8: 26-42.

• Ferris, D. 2010. Second language writing research and written corrective feedback in SLA. Studies in Second Language Acquisition, 32 (2), pp. 181--201.

R. Galgani 2014

Bibliography• Fox, J. D. 2009. “Moderating top-down policy impact and supporting EAP curricular

renewal: Exploring the potential of diagnostic assessment.” • Hyland, Ken & Marina Bondi (Eds). 2006. Academic discourse across disciplines. Bern:

Peter Lang.• Lyster, R., & Ranta, L. (1997). Corrective feedback and learner uptake: Negotiation of

form in communicative classrooms. Studies in Second Language Acquisition, 19, 37-66.• Kim, Youn-Hee. 2011. “Diagnosing EAP writing ability using a Reduced Reparameterized

Unified Model”. Language Testing 28(4): 509-541.• Moore, Tim & Janne Morton. 2005. “Dimensions of difference: a comparison of

university writing and IELTS writing”. Journal of English for Academic Purposes 4: 43-66.• Van de Poel, Kris & Jessica Gasiorek. 2012. “Effects of an efficacy-focused approach to

academic writing on students’ perceptions of themselves as writers”. Journal of English for Academic Purposes 11: 294-303.

• Yilmaz, Y. (2012), The Relative Effects of Explicit Correction and Recasts on Two Target Structures via Two Communication Modes. Language Learning, 62: 1134–1169