Upload
muriel-doyle
View
261
Download
2
Embed Size (px)
Citation preview
Assessing Writing
Contents
Micro-skills vs. macro-skillsTypes of writingScoring writing
Holistic scoringAnalytic scoringObjective scoring
Genres of written language
1. Academic writing
2. job-related writing
3. Personal writing
Theses, shopping list, phone messages email, diaries, advertisements, compositions
Micro-skills vs. Macro-skills Micro-skills:
Phoneme-grapheme correspondences Mechanics such as spelling and punctuation Word order patterns, grammatical systems, & rules Cohesive devices in written discourse
Macro-skills: Rhetorical modes, communicative functions & purposes Main/supporting ideas, new/given info.,
generalization/exemplification Literal & implied meanings Culturally specific references Prewriting devices, paraphrases, synonyms, drafting,
feedback, revising, and editing
Types of Writing (1) Imitative (micro-skills):
Mechanics of writing: fundamental, basic tasks of writing letters, words, punctuation, and short sentences
Focus on form.Handwriting letters, words, and punctuation
• Copying• Listening cloze selection task
Spelling tasks and detecting phoneme• Spelling/ multiple-choice reading-writing spelling tasks• Matching phonetic symbols (p. 225)
Types of Writing
Intensive/controlled (micro-skills):Sentence production: proper vocabula
ry, collocations, idioms, grammarStill focus on form, with some concern
about meaning and context
Intensive Writing Assessment Dictation and Dicto-com Grammatical Transformation Tasks (p. 226)
Change the tenses in a paragraph Change the statement to yes/no questions
Picture-cued tasks Short sentences Picture description Picture sequence description
Vocabulary Assessment Tasks Reordering words in a sentence Short-answer and sentence completion tasks
Dictocomp A variation on the traditional dictation
procedureT reads while Ss listen carefullySs write a summary (not word for word as
in dictation)Decide scoring criteria (Bailey 151)
Rationale:Meaning-orientedConveying & capturing meaning = key
component of communication
Types of Writing (2)
Responsive (macro-skills): Paragraph writing: narratives, descriptions, short reports, lab
reports, summaries, responses to reading, interpretations of charts/graphs
Guided question and answer Paragraph construction tasks Emphasis on context and meaning
Extensive (macro-skills): All-purpose essay writing
• Term paper, research project report, thesis Process of multiple drafts
Scoring Writing
Holistic scoring Analytical scoring Objective scoring
Holistic Scoring (1)
A single scale is used to describe different levels of writing performance.
Not to think about the individual components of the writing skill or to count the number of errors in the writing.
The rater reacts to the writing as a whole. A single score is given.
• E.g., on a scale of 1 ~ 10. Focuses more on communication; overall
impression; e.g.,TOEFL’s TWE (Brown 239; Bailey 187)
Holistic Scoring (2)
Advantages• Fast• Higher inter-rater reliability• Easily understood by teachers and students alike• Widely applicable to writing across different disci
plines• Emphasizes the writer’s strengths
Holistic Scoring (3)
Disadvantages (Brown 242)
One score masks differences across individual composition or the subskills within each score.
No diagnostic info. Is available (no washback).The scale may not apply equally well to all genre
s of writing.Raters need to be extensively trained to use the
scale accurately.
Training Procedure
Norming process: Familiarization with the scale Read several benchmark papers & discussing h
ow they relate to the scale Raters independently read a mixed set of paper
s and score them. Comparison of scores awarded by all raters Discuss discrepancies Another set of papers is read & scored.
Primary Trait Scores
Focuses on “how well students can write within a narrowly defined range of discourse”
A four-point scale ranging from zero to 4. The accuracy of the account of the original
(summary) The clarity of the steps of the procedure and
the final result (lab report) The description of the main features of the
graph (the graph description) The expression of the writer’s opinion
(response to an article)
Analytic Scoring (1) “Embodies hypotheses about the underlying
constructs that comprise a given skill,” or evaluate separately various components
Assessing Ss’ performances on a variety of categories E.g., 5 components:
• Content 13 ~30 points• Organization 7 ~ 20 points• Vocabulary 7 ~20 points• Language use 5 ~25 points• Mechanics 2 ~5 points
Analytic Scoring (2)
Why called “analytical scoring?”Because scale designers have analyzed
the hypothesized components of the writing skill, and it’s these components that make up the categories used in scoring
Analytic Scoring (3)
Evaluating separately various components:Points-off, or weighted score, or split gradeScoring profile: based on five components
• Content• Organization• Vocabulary• Language use• Mechanics