View
8.969
Download
1
Category
Tags:
Preview:
Citation preview
This presentation was developed for the exclusive use of students enrolled in:
Educational Testing & Grading, Professor Gregory E. Stone.
© 2004 Gregory E. Stone. All rights reserved. This presentation may not be reproduced in any form, in part or as a whole, without the express written permission of the author.
Essays& other non-objective items
We’ll include:
Essays, Short answers andother items that can fitwithin the general parameters
Mathematical problem solving
Other partial credit items
Allow for free expressionwithout artificial restriction
and without prompting.
Extended Response
Restricted Response
Two Fundamental Types:
Restricted Response
Limit the ways in which you willpermit the students to answer.
Restricted Response
Limit the ways in which you willpermit the students to answer.
� There ARE correct answers. � We allow students to express
the answer in their own words.
Restricted Response
“Write a brief essay comparing andcontrasting the term analysis and
synthesis as they relate toconstructing (a) objective items
and (b) essay items.”
example
Restricted Response
“What is the poet’s attitudetoward literature as it isapparent in lines 1 to 8?what words in these linesmake this apparent.
example
Restricted Response
“A car traveling 50 mph leaves Chicago at 9am. A train traveling at 70 mph leaves Milwaukee at 10 am. Who will arrive in Toledo (250 miles away) first? Show your work.”
example
Short Answer
Restricted Response
Predicated on the notion that students supply the answers rather than selecting the answer from a group options.
Extended Response
Allow students to express their ownideas and interrelationships amongideas and use their own strategy fororganization.
� No “correct” answers but reasonable/un logic..
Extended Response
Sometimes people write just for the fun of it. This is a chance for you to have some fun writing. Pretend that you are a pair of tennis shoes. You’ve done all kinds of things with your owner in all kinds of weather. Now you are being picked up again by your owner. Tell what you, as the tennis shoes, think about what’s going to happen to you. Tell how you feel about your owner. Space it provided below and on the next page.
example
Extended Response
Devise a plan to determine whether the democrats or republicans are evenly distributed throughout the city, or whether the supporters of each party are concentrated in certain wards.
Design an experiment to calculate the height of a redwood tree.
Create an original story.
example
Extended Response
Because the focus is on logicalargument and reasoned answeringthe teacher must be open andaccepting of uncomfortable responses.
Extended Response
Based on the creative process of the student - often including reasoning and factual presentations.
Essays have a unique ability to assess multiple taxonomic levels at once - while
maintaining a good understanding of strengths and weaknesses.
Look at the family photo attached. Describe the mood or feeling in the photo as well as the body
language of the people. Use metaphors or similes to make these descriptions.
Analysis Knowledge
Comprehension
Is your essay a good essay?
Checklist
1. Does the item test an important aspect of the learning objective?
Essays should ideally be targeted at no less than Application!
Is your essay a good essay?
Checklist
2. Does the item match the learning objective in terms of performance, emphasis and “number of points” (on the exam)?
Is your essay a good essay?
Checklist
3. Does the item require the students to apply their knowledge and skill to solve a new and novel problem?Essays should ideally be targeted at no less than Application!
Is your essay a good essay?
Checklist
4. When viewed in relation to the other items on the test, does this item measure new information covering the range of content and behavior specified in the learning objectives?
Is your essay a good essay?
Checklist
5. Is the item focused? Does it define a task with specific directions, rather than leave the assignment so broad that almost any answer would be acceptable?
Is your essay a good essay?
Checklist
6. Is the task defined by the item within a level of complexity that is appropriate for the intellectual maturity of the students?
Is your essay a good essay?
Checklist
7. To get a good grade on the item, is the student required to demonstrate more than facts, ideas, lists, definitions, generalizations, etc.
Is your essay a good essay?
Checklist
8. Is the item worded in a way that leads all students to interpret the assignment in the way you intended.
Is your essay a good essay?
Checklist
9. Does the wording of the item make clear to the students: Length, purpose, time needed and the basis for evaluation?
Is your essay a good essay?
Checklist
10. If the essay asks students to state and support their opinions on controversial matters, does the wording of the item make it clear that students’ assessments will be based on the logic and evidence supporting their arguments, rather than on the actual position or opinion?
Creating scoring rubrics
Essential for ensuring fair,reliable, valid and meaningfulassessments.
One of the hardest problems!
Rubrics come in 2 varieties:
Holistic (logic of argument)
Analytic (correct answersprovided in the student’s ownwords)
Holistic (logic of argument)
Best used for Extended Response Essays
Grading essays on how well the argument/essay is written in
general.
Analytic (correct answers)
Best used for Restricted Response Essays
Grading essays on whether or not the student has provided the
correct answer.
Analytic rubrics require thestudent to provide specific
information in their answerin order to receive credit.
No “bluffing” allowed!
To construct a rubric:
(1)Ask - what is your objective?(2)Ask - what is the taxonomy?(3)Ask - what pieces of the puzzle do I expect the student to provide?
Example: Comprehend Vertical Leap
Concepts: Potential/Kinetic energy
Describe the concepts of Potential and Kinetic energy. Give an example of each in your description.
Describe the concepts of Potential and Kinetic energy. Give an example of each in your description.
1. Student correctly defined terms 2 points: Answer was clear and fully correct. 1 point: One or more answers were nearly correct, but student missed key concept. 0 points: Student failed to provide correct answer.2. Examples were clear and correct 2 points: Both examples were appropriate. 1 point: One example was appropriate. 0 points: Neither example was appropriate.
Creating rubrics is essential.
Using the rubrics in a standard, systematic fashion is vital!
The loss of reliability and validity are the leading causes of essay
derailment.
May be caused by poorly written essays; or
May be caused by poor use of the created rubrics.
Inconsistent Standards
Raters of essays may vary greatly. Individuals may grade essays differently on one day versus the next.
(Reliability loss)
Rater Drift
Raters tend to either fail to pay attention to criteria or change them - midstream - over time.
(Reliability loss)
Changes in Topic and Prompt
Raters often grade essays of different topics differently. (e.g. I like this topic more so I’m easier)
(Validity loss)
Reader Bias
Halo Effect
Grading based on non-graded topics. (e.g If the student uses good grammar, grade the essay higher, even if the student doesn’t fully answer the item)
Reader Bias
Carryover Effect
If question 1 was answered poorly, the rater already has a bad feeling about the student when theybegin grading question 2.
Or vice versa!
Scoring Suggestions!
You’ve created a rubric, so USE it!
Score one item at a time.
Score essays “anonymously”
Score subject-matter correctness separately.
Provide your students with feedback!
Essays& other non-objective items
Recommended