ASSESSING WRITING (2)

Lecture 9

Teaching Writing in EFL/ESL

Joy Robbins

TODAY’S SESSION

The portfolio system of assessment

What is portfolio assessment? What are the pros and cons of portfolio

assessment? How do I set up portfolio assessment?

We’ll also look at some recent studies of students’ attitudes toward portfolio assessment

2

WHAT IS THE PORTFOLIO SYSTEM OF ASSESSMENT?

A portfolio is a collection of writing, normally a selection of the student’s writing that they have done over their course. Hyland (2003) defines portfolios as ‘multiple writing samples, written over time, and purposefully selected from various genres to best represent a student’s abilities, progress, and most successful texts’ (p.233).

The pieces of writing which go into the portfolio are selected from the whole of the student’s work

The portfolio is evaluated by the teacher (see Yancey 1992)

What do you think are the advantages and disadvantages of using a portfolio system to assess your students’ writing, rather than using traditional exams?

3

ADVANTAGES:PORTFOLIOS AND PROCESS

WRITING (1) Fits in much better with process writing

pedagogy than exams or writing tests. With exams, students have little chance to draft and redraft. Using portfolio assessment, teachers can ask students to write multiple drafts, and the final drafts can go into the portfolio.

Portfolios therefore encourage good (process) writing habits: like revision. Sommers (1991) puts it well: ‘The portfolio itself tends to encourage students to revise because it suggests that writing occurs over time, not in a single sitting, just as the portfolio itself grows over time and cannot be created in a single sitting’ (p.153-4)

4

Another feature that portfolios share with process writing is they both promote dialogue

The portfolio system also fits in with the process approach’s emphasis on peer collaboration and reflection (Burnham 1986).

Nunes (2004) argues that portfolios should ‘facilitate on-going interaction between teacher and students. If the portfolio is to be conceived as an instrument of dialogue, it cannot be written in one session and handed in at the end of the academic year. On the contrary, it has to be continually in the making and document work in progress’ (p.328) 5

ADVANTAGES:PORTFOLIOS AND PROCESS

WRITING (2)

Nunes (2004) also argues that portfolios encourage students to reflect on their writing

In her article, Nunes describes how she encouraged her learners to include reflections on their writing practices in their portfolios, as well as assignments

6

ADVANTAGES:PORTFOLIOS AND PROCESS

WRITING (3)

ADVANTAGES:PORTFOLIOS AND CONTINUITY

Portfolios allow students to write a series of texts that build on each other. Exam writing is disconnected (Black et al 1994; Yancey 1992)

Portfolios allow writers to view the progress they are making and their development as writers (Bailey 1998; Burnham 1986)

7

ADVANTAGES:PORTFOLIOS AND EVALUATION

Portfolio advocates also often believe that the way writing is evaluated is more in line with process pedagogy, because students’ work is commented on throughout the drafting stages

8

ADVANTAGES:PORTFOLIOS AND EVALUATION (2)

‘Portfolio evaluation establishes a writing environment rather than a grading environment in the classroom’ (Burnham 1986:139).

More constructive teacher feedback is possible using a portfolio system. Because the teacher does not have to assign grades early in the writing process, comments can be more constructive than the comments used in grading writing exams (Burnham 1986; Elbow 1993; Myers 1996; Peterson 1995).

9

ADVANTAGES:PORTFOLIOS AND EVALUATION (3)

‘By delaying the appraisal of written products…to the end of an academic term, a portfolio approach promotes revision, encouraging students to assume responsibility for their learning by giving them control over how they manage their time’ (Ferris & Hedgcock 2005: 320-1)

And other researchers claim that this delayed assessment takes the anxiety out of writing for some students who fear evaluation (e.g. Johnston 1983) 10

ADVANTAGES: PORTFOLIOS AND THE

TEACHER—STUDENT RELATIONSHIP

Summarizing the work of a number of scholars (e.g. Condon & Hamp-Lyons 1991; Smit et al 1991; Sommers 1991), Baker (1993) says that ‘The teacher seems to be cast in a different light—that of a mentor or coach—instead of the idiosyncratic authority figure who assigns grades. Teachers are less likely to put grades on papers written in a portfolio classroom than in a standard process-centered classroom, and they instead focus on comments which students are more likely to read and follow because students are given additional opportunities for revision’ (p.156) 11

ADVANTAGES: PORTFOLIOS AND THE

TEACHER—STUDENT RELATIONSHIP (2)

So the idea is that teachers become readers of students’ writing, making constructive comments, rather than assessors…

This may lead to a better, more respectful classroom atmosphere…

12

ADVANTAGES:PORTFOLIOS AND STUDENT

RESPONSIBILITY A portfolio ‘places a large measure of control

over success into the learner’s hands’ (Hamp-Lyons & Condon 2000:35)

Ferris & Hedgcock (2005) make the point that the delay in evaluation until the end of term means that the learners’ destiny is, to some extent, in their own hands, because they have the time to work on their drafts as much or as often as they wish

13

PORTFOLIOS: DISADVANTAGESWORKLOAD

A common argument put forward against the portfolio system (see Smit et al 1991; Sommers 1991) is that it may result in more work for teachers

For instance, when Baker (1993) recalls how she asked some writing teachers at her university to begin assessing students by portfolio, she notes that some teachers ‘expressed concern that their paperload might increase dramatically’, ‘[p]erhaps envisioning students writing five or six drafts of each essay in an attempt to improve their work’ (p.162)

14

PORTFOLIOS: DISADVANTAGESGRADE INFLATION

Baker (1993) also mentions that some of the teachers feared the portfolio system would result in ‘grade inflation’, where higher grades become more common because everyone has the chance to revise their work a number of times (p.162)

So some teachers may feel a portfolio system encourages dumbing-down, and environment where everybody passes, and there are lots of A grades

15

PORTFOLIOS: DISADVANTAGESTEACHER UNEASE AT THEIR LACK

OF CONTROL?

And some researchers also claim that some teachers won’t feel comfortable with the more student-centred classroom that will result from the portfolio system

16

PORTFOLIOS: DISADVANTAGES (2)

Ensuring reliability is a massive problem: would two teachers give the same grades for the same piece of work in a portfolio?

Reminder: Reliability ‘reliability refers to the consistency with which

a sample of student writing is assigned the same rank or score after multiple ratings by trained evaluators’. (Ferris & Hedgcock 1998: 230)

In other words, if we’re marking an essay out of 20, the test will be far more reliable if two markers both award an essay the same grade (or more or less the same grade), say 16 or 17. However, if one marker awards 10 and the other awards 15, the test isn’t reliable.

17

DISADVANTAGES (3) Still on the reliability issue…How can teachers

reliably arrive at a single mark which fairly assesses the whole of the student’s portfolio? The portfolio is likely to feature writing of a number of different genres…

How can two students’ portfolios which feature totally different genres of writing be fairly compared? (Grabe & Kaplan 1996)

There are always potential problems with plagiarism and peer collaboration: ‘How will the portfolio raters know that the students actually wrote all the pieces in the portfolio, and when is editing and revising assistance from other too extensive to represent the student’s own writing abilities’ (Grabe & Kaplan 1996: 417)? 18

PORTFOLIO ASSESSMENT AND TEACHING CONDITIONS

Lam & Lee (2010) explain why portfolios aren’t popular in Hong Kong:‘the exam-oriented culture in Hong Kong has made it difficult for innovative pedagogical ideas, such as process pedagogy to flourish…. Multiple drafting is considered a luxury because teachers are hard-pressed to cover the syllabus to help students prepare for…exams. Second, most practising teachers have not received training in the implementation of…portfolio programmes. They tend to think that asking students to document all their drafts in a folder and grading it summatively amounts to [portfolio assessment].’ 19

This reminds us that a great deal of care and preparation will be needed if it is decided to switch over to the portfolio

system…

20

DECIDING TO CHANGE TO PORTFOLIO ASSESSMENT

Hyland (2003: 236-7) offers a number of useful questions the teacher needs to be able to answer if they’re going to move to a portfolio system of assessment. Here are a few of them:1.Who will choose which pieces of writing go into the portfolio? Teachers only? Students only? Teacher and student together?2.Should the entries receive a preliminary initial grade or the portfolio only be graded as a whole?3.What part will students’ reflections and self-assessments play in the assessment?4.How will consistent scoring and feedback be achieved—what rater training is needed?5.How many people will grade the portfolio and how will scoring disagreements be resolved?6.What mechanisms should be set up for evaluating the program and making changes to it?

If you were moving over to a portfolio system, what would your answers be to these questions?

21

PORTFOLIOS & THE IMPORTANCE OF LEARNER TRAINING

As Hyland (2003) goes on to argue, if you do decide to move over to portfolio assessment, learner training is vital. Students need to be trained how to:

Select items to put into their portfolios Write reflective comments on their choices.

The teacher will probably ask for these comments to be included in the portfolio

How could you ensure students knew how to write reflective comments? 22

RESEARCH ON THE USE OF PORTFOLIOS

What does it tell us about portfolios’

effectiveness, and about attitudes

towards the portfolio system of

assessment?

23

STUDENTS’ ATTITUDES TOWARDS PORTFOLIO ASSESSMENT

Baker (1993) notes that students ‘praised the use of portfolios and often mentioned their changed attitude toward writing’ which was a result of the introduction of portfolios (p.167)

24

STUDENTS’ COMMENTS

One student said portfolios ‘give students a feeling of accomplishment’; another student said he ‘felt more confident in what [he] wrote’; another said the portfolio classroom environment ‘provided a more relaxed atmosphere for me to write in’ (p.167)

There was also evidence that students appreciated the fact that grades were now delayed: one student commented

‘I think the portfolio method helps students out. Instead of receiving a bad grade on a paper a student has a chance to revise their paper’ (p.167)

25

STUDENTS’ COMMENTS When Baker (1993) asked the students

whether teachers should continue using the portfolio system, over 90% said yes.

However, it should be noted that the students in Baker’s study were

undergraduate native speakers…

As Hamp-Lyons & Condon (2000) point out, what’s noticeable is that there

have been few studies of portfolios in EFL/ESL contexts…

26

PORTFOLIOS IN HONG KONG

But Lam & Lee (2010) is an EFL study investigating how students and

teachers reacted once portfolios had been introduced to their classroom.

They report positive attitudes…

27

THE OPPORTUNITY TO CONFERENCE

‘I like the teacher consulting section most. It is because it motivates students to seek teachers’ feedback about their performances. Students and teachers can have a chance to exchange their ideas’. (Lam & Lee 2010: 59)

28

THE BENEFITS OF A PROCESS-ORIENTED, SUPPORTIVE WRITING

APPROACH (1)

‘…in the past, we wrote the composition for the sake of passing the exam and we had only an hour to write it. Now, we have more time and support such as multiple drafting, peer feedback, conferences and so forth…’ (Lam & Lee 2010: 59)

29

THE BENEFITS OF A PROCESS-ORIENTED, SUPPORTIVE WRITING

APPROACH (2)

‘In secondary school, as teachers adopted [a] one-shot approach to writing, I was only expected to produce one draft and there wasn’t much time for deeper thinking’. (Lam & Lee 2010: 60)

30

ANOTHER STUDY OF PORTFOLIOS IN EFL/ESL CONTEXTS

Song & August (2002) is another of the few EFL/ESL studies available, and is quantitative

As a result of comparing non-native university writers’ pass rates on a timed exam and a portfolio, with the portfolio producing a higher pass rate, Song & August (2002) argue that timed writing can unfairly penalize non-native writers

31

TIMED WRITING VS. PORTFOLIOS

‘These exams particularly handicap ESL students because they not only test them on unfamiliar genres and tasks, but also require them to meet standards of excellence in grammatical and mechanical accuracy they cannot reach on a first draft in 50 minutes…’ (Song & August 2002: 61)

32

Another study:Hirvela & Sweetland (2005)

33

SETTING OF THE STUDY

Two undergraduate writing programmes in an American university which required students to produce a portfolio

The portfolios were only worth 5% of the students’ overall grade for one course, and 10% for the other, in an effort to get students to focus less on grades, and more on lecturers’ formative assessment (and students’ self-assessment) of the writing process

34

THE REFLECTIVE LETTER

For both courses, ‘the students were required to write an introductory letter in which they were to discuss the writing contained in the portfolios by reflecting on what the samples revealed about their development as writers’ (Hirvela & Sweetland 2005: 196)

The idea was to find out about the students’ experiences of, and attitudes

towards, the two different ways portfolios were used on the courses… 35

METHODOLOGY

Data about students’ experiences and attitudes was collected via

interviews analysis of their reflective letter (students’ pieces of writing in their portfolios

was also examined)

Two students took part in the study…

36

STUDENT 1: SHIM

Korean, doing a business degree Had never been asked to keep a portfolio

of writing before

37

STUDENT 2: MOTO

Japanese, doing an Aeronautical Engineering degree

Had moved with family to the US three years before he started university, so had some experience of doing portfolio writing at his American high school

38

SHIM’S REFLECTIVE LETTER (1)

‘The students were instructed to (a) identify, and provide a rationale for, the pieces of writing they had chosen to include in their portfolio, and (b) reflect on the writing growth they had experienced during the course’. (p.201)

However, it was clear that Shim didn’t really engage with the task

in this way…39

SHIM’S REFLECTIVE LETTER (2)

Shim explained the reflective letter as follows:“I think the…letter is kind of, as you know, when you make thesis in graduate school, before we submit thesis, we have to write introductory letter, right? I think this letter is like that, abstract for my portfolio, and thanks for teaching me, you are a nice teacher—I think the introductory letter contains this kind of information”. (p.201)

So Shim’s letter mainly summarized the writing samples which he put in his portfolio and thanked his teacher…

40

THE PURPOSE OF A PORTFOLIO: FORMATIVE OR SUMMATIVE? (1)

In the interviews, Shim‘maintained his belief…that a portfolio is more appropriately used as the basis for teachers to assess students than for students to reflect upon their learning as writers. In other words, he conceptualized the portfolio…as summative in nature despite its stated formative intentions’. (p.202)

Note: Students were provided with lengthy written explanations about

the formative nature of the portfolio!

41

THE PURPOSE OF A PORTFOLIO: FORMATIVE OR

SUMMATIVE? (2) Moto, like Shim, saw the portfolio as

summative: it ‘was just another piece of schoolwork that teachers used to “keep students busy”.’ (p.204)

42

THE PURPOSE OF A PORTFOLIO: FORMATIVE OR

SUMMATIVE? (3) Moto didn’t see any point in selecting

work, and said he didn’t benefit from doing so

He also felt the reflective letter was a waste of time and ‘boring’ (p.205)‘Perhaps not surprisingly, then, he claimed no sense of ownership for the submitted portfolio…and did not feel proud of it’ (p.204)

However, it turned out that Moto kept his own portfolio…

43

MOTO’S OWN PORTFOLIO

This consisted of all the pieces of writing Moto had done for a number of years, and‘he reported “enjoying” reading them once in while now that he had become a more mature and experienced…writer…he reported that someday…he thought that his future job could require him to do some of the types of writing required [on the university] course, in which case the work in his own portfolio could serve as models of these tasks. […] [Moto] felt that he would need an extended period of time as well as a comprehensive collection of his work to assess his growth as a writer.’ (pp.204-5)

44

THE PURPOSE OF A PORTFOLIO: FORMATIVE OR

SUMMATIVE? (4) ‘Because [Moto] believed that future

reflection was more valuable than more immediate reflection, he saw no reason (beyond getting a higher grade) to spend time trying to make the portfolio look good […] He felt that students need more than the few months a writing course typically covers in order to fully internalize what they’re learning about writing, and so they aren’t in a position to reflect meaningfully on their growth while compiling a portfolio’. (p.206)

45

REFLECTIONS ON WRITING

‘…development of writing ability is a long-term process, whereas classroom portfolios operate in the short-term realm of a writing course. This may well be why the introductory letters weren’t richer in terms of their reflective comments. The participants were being asked to reflect on their growth as writers before they felt they were ready to do so…’ (p.208)

46

A HANDS-OFF APPROACH TO PORTFOLIOS

Shim and Moto’s lack of enthusiasm may also be connected to the way the

portfolio system was implemented ‘…very little class time was devoted to

discussing portfolios in general and the portfolio assignments …. The students were pretty much left to their own devices in terms of interpreting the portfolio intentions and requirements […] It appears…that the…instructors failed…to nurture an ongoing portfolio culture that would enable students to better understand what the portfolio pedagogies were meant to achieve…’ (p.209)

47

A FORMATIVE CULTURE…IN THEORY

Although the portfolios were meant to be formative rather than summative, Hirvela & Sweetland (2005) note that the classroom culture wasn’t really formative…

So it’s not really surprising that Shim and Moto saw the portfolios as summative…

48

WRITING ASSESSMENT: PREFERRED METHODS

Based on last week’s lecture and this week’s session, would you prefer to assess your students’ writing ability by means of exams or the portfolio system? Why?

Do you think the portfolio system may be culturally more acceptable to western students than to students from, say, south-east Asia? Why (not)?

If you are a language teacher, would your students be happy with being assessed by means of the portfolio system? Why (not)?

If you were learning a foreign language, would you prefer your writing to be assessed by exam or by portfolio? Why?

49

REFERENCESBailey KM (1998) Learning about Language Assessment: Dilemmas,

Decisions, and Directions. Boston: Heinle & Heinle.Baker NW (1993) The effect of portfolio-based instruction on composition

students’ final examination scores, course grades, and attitudes toward writing. Research in the Teaching of English 27(2): 155-174.

Black L et al (1994) New Directions in Portfolio Assessment. Portsmouth: Heinemann.

Burnham C (1986) Portfolio evaluation: room to breathe and grow. In C. Bridges (ed.), Training the New Teacher of College Composition. Urbana: National Council of Teachers of English, pp.125-139.

Condon W & Hamp-Lyons L (1991) Introducing a portfolio-based writing assessment: progress through problems. In P.Belanoff & M.Dickson (eds.), Portfolios: Process and Product. Portsmouth: Boynton Cook, pp.231-247.

Elbow P (1993) Ranking, evaluating, and liking: sorting out three forms of judgment. College English 55: 187-206.

Ferris D & Hedgcock JS (1998) Teaching ESL Composition: Purpose, Process, and Practice. Mahwah: Lawrence Erlbaum.

Ferris D & Hedgcock JS (2005) Teaching ESL Composition: Purpose, Process, and Practice (2nd edition). Mahwah: Lawrence Erlbaum.

Grabe W & Kaplan RB (1996) Theory and Practice of Writing. London: Longman.

Hamp-Lyons L & Condon W (2000) Assessing the Portfolio: Principles for Practice, Theory and Research. Cresskill: Hampton Press.

Hirvela A & Sweetland YL (2005) Two case studies of L2 writers’ experiences across learning-directed portfolio contexts. Assessing Writing 10: 192-213.

50

REFERENCES (2)Hyland K (2003) Second Language Writing. Cambridge: Cambridge

University Press.Johnston B (1983) Assessing English. Sydney: St Clair Press.Lam R & Lee I (2010) Balancing the dual functions of portfolio

assessment. ELT Journal 64: 54-64.Myers M (1996) Sailing ships: a framework for portfolios in formative and

summative systems. In R. Calfee & P. Perfumo (eds.), Writing Portfolios in the Classroom: Policy and Practice, Promise and Peril. Hillsdale: Lawrence Erlbaum Associates, pp.149-178.

Nunes A (2004) Portfolios in the EFL classroom: disclosing an informed practice. ELT Journal 58(4): 327-335.

Peterson R (1995) The Writing Teacher’s Companion: Planning, Teaching, and Evaluating in the Composition Classroom. Boston: Houghton Mifflin.

Smit D et al (1991) Implementing a portfolio system at Kansas State University. In P.Belanoff & M.Dickson (eds.), Portfolios: Process and Product. Portsmouth: Boynton Cook, pp.46-56.

Sommers N (1991) Bringing practice in line with theory: using portfolio grading in the composition classroom. In P.Belanoff & M.Dickson (eds.), Portfolios: Process and Product. Portsmouth: Boynton Cook, pp.153-164.

Song B & August B (2002) Using portfolios to assess the writing of ESL students: a powerful alternative? Journal of Second Language Writing 11: 49-72.

Yancey KB (1992) Portfolios in the Writing Classroom. Urbana: National Council of Teachers of English. 51

THIS WEEK’S READINGChapter 8 of:Ferris D & Hedgcock JS (1998 or 2005) Teaching

ESL Composition: Purpose, Process, and Practice. Mahwah: Lawrence Erlbaum.

If you’re interested in portfolios, also look at what Ken Hyland (2003) says about portfolios in his book,

Second Language Writing,and:

Hamp-Lyons L & Condon W (2000) Assessing the Portfolio: Principles for Practice, Theory and Research. Cresskill: Hampton Press.

Hirvela A & Sweetland YL (2005) Two case studies of L2 writers’ experiences across learning-directed portfolio contexts. Assessing Writing 10: 192-213.

52

NEXT WEEK

In the final session, we’re going to have a look at using technology to teach writing

So next week’s class will be in computer lab G

53