bENEMERITA uNIVERSIDAD AUTONOMA DE PUEBLA Facultad de Lenguas Maestra en la Enseanza del Ingls Implementing English Writing Rubrics in a Mexican University: Designers, Professors and Students beliefs by Carina Flores Calyeca

E) WHY?D) WHEN?A) WHO?B) WHAT? F) HOW?C) WHERE?Cultural, political, administrative, educational and institutional subsystems.

Cooper, 1982Markees Theoretical FrameworkThe implementation of the rubrics to assess writing at this public university was carried out in two phases.

In 2008 coordinators provided teachers with an analytic rubric to evaluate different types of writing tasks.

2. In 2011 two multiple-trait rubrics to assess writing tasks were implemented. FrameworkRubrics

Descriptive scoring schemes that are developed by teachers or other evaluators to guide the analysis of the products or processes of students efforts (Brookhart, 1999).

Theoretical FrameworkCenter-periphery & Problem-solving models.

Coordinator: we were concerned about the number of teachers we had at that time. We were thinking that some teachers were very flexible in their grading systems and that other teachers were were being too strict and too hard on students and none of the two extremes was fair. FrameworkFrameworkThis study has been carried out in the English Language Department of a large public university in the center of Mexico where English language is taught as a mandatory course for undergraduate students and who at the end of the last and forth course are expected to achieve an A2 language level based on the Common European Framework.

There are 75 teachers who belong to this English Language Teaching department and who teach at different schools and different timetables every quarter and; therefore, they do not meet frequently. Framework Types of rubrics

1. Holistic rubrics

2. Analytic rubrics3. Primary trait rubrics4. Multiple trait rubrics

TraitExceptionalAcceptableAmateurUnsatisfactorySentences and paragraphsUsage of sophisticated sentence patterns.Paragraphs indicate shift in thought and are used to make sequence of events clear.Simple and some complex sentences are used.Some paragraphing to show sequence of events/ideas.Sentence structure is usually correct.Simple sentences are used.Little attempt made to paragraph writing.Sentences do not make sense.No paragraphing.Word choiceWords are used correctly and precisely.Acceptable vocabulary. Words are technologically appropriate. Simple vocabulary.Incorrect vocabulary.SpellingSpelling is correct, including complex and irregular words.Spelling is generally accurate.Frequent spelling errors.Spelling errors interfere with understanding.PunctuationA range of punctuation including commas, apostrophes, colons and semicolons is used accurately and effectively.Periods and capitals are used correctly and punctuation is beginning to be used within the sentence.Frequent punctuation errors.Insufficient or lacks punctuation.Incorrect use of capital letters.Primary Trait Scoring Rubric for Writing Mechanics

Letter-writing : Advising a friend about workCategory4321Salutation and closingSalutation and closing have no errors in capitalization and punctuation.Salutation and closing have 1-2 errors in capitalization and punctuation.Salutation and closing have 3 or more errors in capitalization and punctuation.Salutation and/or closing are missing.FormatComplies with all the requirements for a friendly letter.Complies with almost all the requirements for a friendly letter.Complies with several of the requirements for a friendly letter.Complies with less than 75% of the requirements for a friendly letter.Content and accuracyThe letter contains the 3 accurate facts about the topic.The letter contains 2 accurate facts about the topic.The letter contains 1 accurate facts about the topic..The letter contains no accurate facts about the topicGrammar and Spelling (conventions)Writer makes no errors in grammar or spellingWriter makes 1-2 errors in grammar and/or spelling.Writer makes 3-4 errors in grammar and/or spellingWriter makes more than 4 errors in grammar and/or spellingAdvantages of the use of the rubrics

Timely feedback (Steven and Levi, 2005)

Detailed feedback: strenghts and weaknesses (Steven and Levi, 2005)

3. Encourage critical thinking (Goodrich, 2000 & Steven and Levi, 2005)

4. Self and peer evaluation (OMalley & Valdez, 1998; Saddler & Andrade, 2004)5. Teachers expectations (Stevens and Levi, 2005; INTEL 2012)

6. Refine teaching skills(UNSW, 2014)

7. Objective assessment (Goodrich, 2010)Disadvantages of the use of the rubrics

Task-specific evaluative criteria(Pophman, 1997) General evaluative criteria (Pophman, 1997)

3. The excessive lenght in a rubric (Pophman, 1997) 4. Rubric administration (Kohn, 2006)

5. Rubrics are not self-explanatory. (Goodrich, 2010)

Method

Case study based on a mix methods exploratory sequential desing (Cresswell, 2007).

This type of approach focuses first on collecting qualitative data and then collecting quantitative data in order to test or generalize the initial findings (Creswell, 2011)

Context of Research: A public University in Central Mexico. English was established as a mandatory course as part of curricula.

Participants: Rubric designers, teachers, and students.

Instruments: Semi-structured interviews and semi-structured questionnaires.

Data collection and analysis: Content analysis principles. (Cohen,Manion, & Morrison, 2000)

Results

The implementation of rubrics to assess writing skills: Markees Theoretical Framework.

2. Designers rationale underlying rubrics implementation.

3. Teachers beliefs about the use of the rubrics including what they think about the A2 language level required to students.

4. Students beliefs about the use of the writing rubrics summarizing strengths and limitations they reported.

E) WHY?D) WHEN?A) WHO?B) WHAT? F) HOW?C) WHERE?Markees Theoretical Framework2. Designers rationale underlying rubrics implementation.

First, to standardize evaluation since every teacher had their own grading system and some of them were described as flexible and some others as too strict; Second, to be fair with students work and give effectively feedback to students because as Stevens and Levi (2005) claim that rubrics are useful tools to give effectively feedback if it is given immediately. And third, evaluation based on competences should be carried out through rubrics (OMalley & Valdez,1998).

3. Teachers perspectives about the use of the rubrics:

Purposes

Strengths

WeaknessesStrenghtsWeaknesses#when they used the rubricsAdvantagesDisadvantages1Before and after writingA guide to know what to writeWriting style limited2Before writingA guide to know what to writeThey contain a lot of academic vocabulary3Required by the teacher, before writingTo be informed about aspects to be evaluated4A rare use: self-learningTo be informed about aspects to be evaluated

Lack of use5Before writing and I understand what to doA guide to know what to writeLack of use6Before writingTo be informed about aspects to be evaluated4. Students perceptions about the rubrics: advantages and disadvantagesGeneral conclusions

1. There is little experience with rubrics in this university.2. Center-periphery and Problem-solving models for a top down decision made by the coordinators and a group of teachers to standardize assessment , give feedback and evaluate competences. 3. Third, more than 50% of the teachers, in this study, adopted the rubrics mainly to grade and evaluate.

General conclusions

From 40 to 45% of the teachers did not adopt the rubrics mainly because they considered they were time consuming, confusing and inadequate for the context.

Students reported this was the first experience with rubrics and identified more strengths than weaknesses.

Implications

Raise teachers awareness about rubrics not only as assessment tools but also as learning and teaching tools for self and peer evaluation.

Directions for further research

First, to research on how teachers who did not use the rubrics usually evaluate their students writing tasks. Second, to continue exploring teachers best practices on the use of the rubrics in this context.

Thank you!ACTORSadoptersresistersChange agentsclientsimplementerssuppliersFrameworkTeachers who adopted the rubric: adopters Teachers who sometimes resisted: laggards Framework