Upload
ipt652
View
2.676
Download
0
Tags:
Embed Size (px)
Citation preview
Assessment in Schools
Complex Achievement:Scoring Performance Based Assessments
Question (choose the best answer)
Which statement provides the best description of an analytic scoring rubric?
A. Rating is based on the overall performance compared to exemplars.
B. Rating is based on a sum of scores for the individual parts of the performance.
C. Rating is based on the student’s analysis of their performance.
D. All the above.
Types of Performance
1. Alternative – something other than traditional paper and pencil tests requiring students to demonstrate
2. Authentic – practical application of a task in real world conditions/setting(usually only approximated/simulated)
Assessing Performance
Why use performances in assessment?
Why do we score/measure performance? Communication Comparison
Assessing Performance
All Claims about the value of performance assessments rest on the assumption that performance can be accurately observed and reliably rated.
Conducting Music Assessment
Scoring Challenge
No one correct or best answer/solution
Many different performances or solutions might be judged as excellent (or poor)
Requires expert judgment and clearly specified criterion to assess properly.
Scoring Limitations Scoring can be inconsistent (unreliable Rating)
To compare scores fairly, Task (learning outcome) must be clearly
defined and communicated to students Scoring criteria/rubrics must be well
defined. Time consuming to complete
Must have reasonable amount of time to do Limits the number of tasks that can be
done
Scoring Issues
What are you assessing?
Process – approach used, methods & procedures, instrument use, etc.
Product – complete performance or resulting artifact
Scoring Decisions
What assessment instruments will be used?
Rubrics, Rating scales, Checklists …
How will the results be used/reported?
What will you do to make sure the results are accurate (reliable)?
Scoring Decisions
Who will do the assessment?
Teacher, student, peers, others
How will they be trained?
Guidelines and Suggestions
1. Focus on the learning outcomes that require complex cognitive skills and performances
2. Select tasks that represent important content and skills
3. Minimize the dependence of irrelevant skills not directly related to learning outcome
Guidelines and Suggestions
5. Provide scaffolding as needed6. Construct task directions that
clearly explain what students are expected to do
7. Clearly communicate performance expectations (how performance will be judged)
Scoring Issues
What are your expectations?
Criteria – ideas about what is good or desirable when we judge adequacy; also used to defend that judgment.
Scoring Criteria Issues
Floating Criteria – wait until you see the performance to determine acceptability
Ask yourself – Do you know what your are looking for? Can you define and describe the quality of a
performance (both good and bad)? Can you provide a defensible basis for
rating good and bad performance?
Scoring Criteria Issues
Criteria – define what is acceptable and unacceptable
in ways the student can understand communicate the goal or standards not useful when vague or ambiguous make public what is considered important
[9-30] characteristics
Instruments
Rubrics Rating Scales Checklists
Scoring Rubrics
Rubrics are a set of guidelines that explain the criteria by which performance will be judged or rated (may include a rating scale).
Rubrics outline performance standards
Scoring Rubrics
Rubrics can be
Analytic – individual aspect of the task are judged and used to determine overall score
Holistic – the performance or product is judged as a whole, compared to models or exemplars
[see chapter 10]
Scoring Rubrics
Rubrics typically provide a description of how the rater should determine the quality of various performances at specific levels.
Examples [9-29,9-33, pg 272]
[9-32 rubric development]
Group Task
Create an rubric you might use to rate or score the Leading Music performance.
Rating Scales
They provide a convenient recording method, common frame of reference, and focus the raters attention on specific important aspects of the performance
Used for (limited to) make quality judgments Requires additional information regarding
performance expectations Examples [9-34, pg 274]
Rating Scales
These take many forms (numerical and descriptive) but are used to provide a uniform way to score performances along a continuum (at least ordinal, preferably interval).
[9-42 types of rating scales]
How often do you (meant to record
frequency)
Response scale 1
Daily 2-3 times per week Once a week 2-3 times a month Once a month Less than once a
month
Response scale 2
Once a day Once a week More than once a
day More than once a
week As seldom as
possibleNot Ordinal, Not IntervalOrdinal but Not Interval
Question (choose the best answer)
Which of the following is NOT a good principle for constructing a graphic rating scales?
A. Characteristics should be directly observable.
B. Use 3 to 7 points on the scale.C. Points on the scale must form an
ordinal continuum.D. Each point on the scale must be
defined clearly.
Rating vs. Ranking
Ranking requires a person to place in relative order
Raters assigns a specific score
Why might you rank instead of rate?
Checklists More appropriate for analytic rubrics where
you can easily divide the task into a series of specific actions that must be present.
Reduces the amount of subjectivity in the judgment (dichotomous decision)
Can be problematic when aspects of the performance are valued but not represented in the criteria. (e.g., esthetically pleasing, interesting)
Examples [pg 282]
Common Rating Errors
Personal Bias – Generosity error – too easy, grade inflation Severity error – too hard, no perfect papers Central tendency – rating everyone about average Halo Effect – general impression of individual
(positive or negative) influences an individual rating Logical error – rating alike or different based on the
belief that factors are related (e.g., studious and able) [see 9-44, pg. 277]
Effective Rating Review
Focus on educationally significant outcomes
Characteristics should be directly observable
Clearly define key points on scale Select most appropriate type of
instrument Use an appropriate scale (# of points)
Effective Rating Review
Rate all performances on one task before going on to next.
When possible rate performances without knowing the raters name
If the assessment has significant impact, several ratings should be used.
Example practice [7-22,7-23]