Assessing Writing– assessing tools and teaching action

Barbeiro, Luís Filipe

LITERACY ACROSS THE CURRICULUM IN DIFFERENT LANGUAGES AND CONTEXTS.

Functional approaches to reading and writingMadrid

17-19 Oct 2013

Ferreira, Alexandra Oliveira, Ana CristinaBarbeiro, Célia

Correia Mateus School

School of Education and Social Sciences,‒ Polytechnic Institute of Leiria

■ In this presentation, we will compare the potential and use of different

writing assessment tools: the evaluation grids in use at schools

participating in the TeL4ELE project, in Leiria, and criteria that have been

adopted since the implementation of the R2L programme.

2

Introduction●

► Enable teachers to see the progress students are making, and how far they need to grow to achieve their grade standard;

► Provide a record of achievement for students, their family and the state,

► Show what language resources and skills students have, so teachers can plan teaching program, and explicitly discuss language and skills with them.

3

Introduction●► Background

♦ Purposes of writing assessment

► Rose (2010)

► Portuguese Ministry of Education has adopted a policy of national

examinations in order to evaluate students’ achievement (writing

included), at different school years: 4th, 6th, and 9th.

► Accompanying the national exams, the Ministry of Education provides

assessment guidelines and criteria to be used by the teachers who are

involved in the process of marking students’ exams.

4

Introduction●► Background

♦ Portuguese context

► Extension

► Theme and Typology

► Coherence and Content

Information

► Structure and Cohesion

► Syntax

► Vocabulary

► Punctuation

► Spelling5

Introduction●► Background

♦ The evaluation grid of the Ministry of Education (national examinations)

With slight modifications over the years, this grid has adopted the following criteria:

0/1-4/5

6

Introduction●► Background

▪ Since these guidelines and criteria were created by the Ministry of

Education and since they are used in national examinations, they were

adopted by many schools and teachers, aiming to keep up to date and

improve students’ preparation for the national examinations.

▪ Therefore, in Portuguese schools, these criteria have acquired a

regulatory dimension, in relation to writing assessments and ultimately in

teaching (functioning as guidelines to teachers’ evaluation and feedback

to students).

It is important to reflect on the potential and limitations of these criteria in comparison with other grids

► The assessment grids adopted in the schools were also largely based on the criteria of the Ministry of Education:

7

♦ … Schools where TeL4ELE was implemented, in Leiria

■ Correia Mateus School(5th - 9th)

▪ Type of text and theme

▪ Coherence and pertinence

of information

▪ Structure and cohesion

▪ Morphology and syntax

▪ Vocabulary

▪ Spelling

▪ Punctuation

■ Vale Sumo School(1st - 4th)

▪ (Extension)

▪ Theme

▪ Sequencing

▪ Ideas / Creativity

▪ Vocabulary

▪ Spelling

▪ Punctuation

Var.0-2/3/40-3

8

♦ During the implementation of TeL4ELE, we applied the criteria of R2L:

Levels Criteria Quick judgement Scores

CONTEXT

Purpose 0-3

Staging 0-3

Phases 0-3

Field 0-3

Tenor 0-3

Mode 0-3

DISCOURSE

Lexis 0-3

Appraisal 0-3

Conjunction 0-3

Reference 0-3

GRAMMAR 0-3

GRAPHIC FEATURES

Spelling 0-3

Punctuation 0-3

Presentation 0-3

• The most salient differences refer to the higher levels of text

construction: the level of the relationship with Context and the Discourse

organization

• In the previous grids, the criteria referring to higher levels merge

different levels (e.g. Structure and Cohesion; Sequencing; Coherence

and pertinence of information)

• In contrast, the R2L grid offers more specific criteria/parameters to deal

with text, at the higher levels of text adequacy to objectives and text

organization beyond the sentence

9

► Contrasting the grids

• Do they result in different global evaluations of students’ texts?

• Do the more specific parameters of R2L grid, namely at Context and Discourse levels, reflect on scores?

• Do R2L criteria reveal the need for specific action at these levels?

10

Study●

► Method

■ We compared the application of the two grids:

► Comparing the total score obtained with each one of the grids(text genre: narrative; scores were assigned by teachers of the classes)

► Comparing particular texts under the perspective (and scores) of the different grids

11

Results●► Correlations

♦ Very high correlation between the total scores of the two grids

Levels r Sig.

All cases (n=37) 0,863 ** (p<,01)

5th (n=25) 0,985 **

4th (n=6 ≡ 2H+2M+2L) 0,951 **

3rd (n=6 ≡ 2H+2M+2L) 0,984 **

12

♦ Both grids discriminate high, medium and low achieving students.

Levels Grid 1(av.)

R2L(av.)

High achievement 92 % 91 %

Medium 71 % 70 %

Low achievement 55 % 56 %

► The purposes of writing assessment tools are not limited to evaluating

students’ achievement at the end of each term or school year (to inform the

families and the educational authorities).

▪ Two other major purposes are► to show the language resources and skills that students have, so teachers

can plan teaching program;

► to function as tools of interaction in order to explicitly discuss language and skills with students.

► Working with R2L proposals and criteria provided us the

specific terms to guide students to improve their texts at the

level of Text organization and Discourse;

► At the level of Text organization / Context, the work on

Stages and Phases became prominent.

13

● ♦ Implications of R2L criteria for teachers’ practice

► During the implementation of TeL4ELE, namely through the R2L writing assessment criteria, we became aware that some of the criteria related to CONTEXT and DISCOURSE were not specifically present in our assessment grid and, as a consequence, did not acquire prominence in our instruction or our interaction with children;

► A significant example concerns text genre and its organization through Stages and Phases;

► Talking about text organization, the prominent terms were the common and general terms Introduction; Development and Conclusion;

14

● ♦ Implications of R2L criteria for teachers’ practice

► Working Stages and Phases benefited the students at all levels of achievement;

► The students who already performed at the highest levels found new challenges and new fields to improve their writing:

◊ the challenge of developing their text, according to the Stages of a specific genre (the terms “Orientation”, “Complication”, “Resolution”, for example, becoming usual in the classroom interaction, and allowing to establish the differences between narratives and recounts);

◊ the challenge of incorporating specific phases, in order to improve their texts — and, by doing this, finding the pathway to creativity, as modelled in the texts of the authors;

► The students at lower achievement levels took part in the classroom interaction and revealed (namely through explicit guidelines and the strategies of Rewriting and Joint Construction) textual and linguistic devices to improve their writing.

15

● ♦ Implications of R2L criteria for teachers’ practice

16

Examples●♦ Sara’s narrative “Adventure on the

beach“• Sara, 5th grader, is a high-achievement

student (level 5/5);• Very often, her texts receive the

maximum scores;

• The first evaluation of her text, using the usual Grid, assigned it the maximum score. The text does not present formal problems: spelling, syntax, etc. are OK. We cannot say it is not organised or it has sequencing, coherence and cohesion problems;

• R2L criteria revealed that the text could be improved through a better management of Stages (namely working on Complication and Resolution) and Phases (inserting a real Problem, Reactions, Solution…) — subsequent work allowed to improve even more Sara’s writing, according to the genre challenges.

An adventure at the beachIt was a beautiful sunny day.Carlota, the female crab, woke up with the sound of ocean waves.Carlota was very curious and as she was still small, she did not know well the beach where she lived.After having breakfast, Carlota was calling her friend Diogo, the hermit crab, to play with her by the seaside.They were both entertained playing a game, when suddenly, they heard a whistle. The whistle came from behind a rock, and Carlota and Diogo went out to see what was happening. They were amazed when they saw a fish trapped between the rocks whistling. The three became great friends and they played altogether until sunset. After sunset, they went back to their homes.Carlota never forgot that fantastic adventure at the beach.

17

Examples●♦ Patricia’s story “The horse“• Patricia, 5th grader, is a low-

achievement student (level 2/5);

• The first evaluation of her text, using the usual Grid, penalized the formal problems (8/20 ≡ 40%);

• R2L criteria confirmed the formal (spelling, …), discursive problems (Lexis, Reference, Appraisal), and genre problems (although a rough mastery of Stages) — 19/42 ≡ 45%).Instead of relying on generic concepts (“organization”, “structure”, …) it showed the way of the specific work with Stages and Phases, and the strategies of Rewriting to improve Patricia’s writing at the levels of Discourse and Context.

18

Examples●

♦ During the implementation, the work with Stages acquired prominence.

Orientations attained a high level of elaboration, even at the lower school

grades:

3rd grade

4th grade

Once upon a time, there was a beautiful and colourful mermaid of the sea.The mermaid had got blue eyes and her tail had the colors of the rainbow.The sea where she lived was quiet and beautiful, with blue water and many fish.

On a cloudy day, the sea was rough, and down there in the deep ocean was a beautiful mermaid with pink hair and a dark green tail, the most beautiful colour of that ocean.

19

Examples●

♦ Complications and Resolutions present a lower level of elaboration.

However, they became present and established the difference with the

genre Recount (which previously occupied almost all the work with stories).

3rd grade

4th grade

Examples of Resolutions

Then the elf became the wise.

On the big day she won and became amazed /openmouthed with the prize.

Thereafter there was never lack of food in that place.

Conclusion

20

■ Writing assessment criteria▪ are a powerful tool to regulate and influence the process of teaching writing skills

■ Portuguese language teachers ▪ have adopted the criteria of the Ministry of Education for national examinations;▪ this grid does not provide enough and adequate criteria to assess students’ texts at the levels above the sentence (some of the criteria merge different aspects or levels)

■ Working with R2L criteria▪ in association to a genre based pedagogy, revealed some of the weaknesses that

students’ writing commonly still present, at the level of text organization,▪ and guided us in the process of improving writing, focusing on genres, Stages and

Phases, benefiting students at all achievement levels.

▪ Alongside with Lexis, this was one of the most remarkable improvement of students’ writing, in our TeL4ELE classes.

21

Thank you for your attention!