Sound%20 design%20%28ch%204 7%29

Sound Assessment DesignChapters 4-7

Sunday, October 13, 13

Chapter 4: Sound DesignLearning Targets

Select appropriate method(s) to assess specific learning targets.

Follow steps in the Assessment Development Cycle.

Create an assessment blueprint.

Use an assessment blueprint with students as an assessment for learning


Key 3: Sound Design

Do assessment methods match learning targets?

Does the sample represent learning appropriately?

Are items, tasks, and scoring rubrics of high quality?

Does the assessment control for bias?


Assessment MethodsSelected response

Multiple choice

True/False

Matching

Fill-in-the-blank questions

Written response

Short answer items

Extended written response items

Performance assessment

Performance task

Performance criteria

Personal communication

Questions during instruction

Interviews and conferences

Participation

Oral exams

Student journals and logs


Target-Method MatchStrong - The method works for all learning targets of this type

Good - The method works for many learning topics of this type

Partial - The method works in some instances for learning targets of this type

Poor - The method never works for learning targets of this type

See chart p94


Target-Method MatchSelected

ResponseWritten

ResponsePerformance Assessment

Personal Communication

Knowledge

Reasoning

Skill

Product

Good Strong Partial Strong

Good Strong Partial Strong

Partial Poor Strong Partial

Poor Poor Strong Poor


Activity 4.2 Target-Method Match Template

Learning TargetLearning Target

Target TypeTarget TypeTarget TypeTarget Type Assessment MethodAssessment MethodAssessment MethodAssessment Method

Learning TargetLearning Target K R S P SR WR PA PC

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.


Assessment Development CyclePlanning Stage

Determine who will use the assessment results and how they will use them.

Identify the learning targets to be assessed.

Select the appropriate assessment method or methods.

Determine the sample size.

Development Stage

Develop or select items, exercises, tasks, and scoring procedures.

Review and critique the overall assessment for quality before use.

Use Stage

Conduct and score the assessment.

Revise as needed for future use.


Test Blueprint

List/classify the learning targets

Write learning targets into appropriate spaces depending on blueprint format

If using multiple methods, identify appropriate method for each target

Weight importance of each target


Test Blueprint (Multiple-Method)

Unit:Unit:Unit:Unit:

Learning Targets Target Type Assessment Method Percent Importance


Test Blueprint (Selected Response)

Learning Targets Problem #s Total Points


Learning Unit Activities

Use Target-Method Match template to determine proper assessment strategies for learning targets

Audit existing assessment(s) for clear purpose

Audit existing assessment(s) for clear learning targets


Assessment Methods

Selected response

Multiple choice

True/False

Matching

Fill-in-the-blank questions


When do we need Selected Response?Use selected response assessments when...

Broad topic requiring wide-range coverage

Diagnose student misconceptions and flaws in reasoning

Students can read/understand language well enough to know what is being asked


Choosing Selected Response Types

Item Used when... Advantage Limitations

Multiple-Choice

True/False

Matching

Fill-in-the-Blank

1 correct/several plausible

Variety; Easy score; Coverage; Can diagnose Guessing

Large Content/Many items Time; Easy score Trivial/misleading;

Guessing

Related thoughts/facts; Association

Coverage; Several MC in one (others are

distractors)Elimination

Clear/short answer; Answer known rather than picked from list

Response production; Reduced guessing; Coverage

Scoring time


Writing Quality ItemsKISS - Keep wording simple and focused. Aim for the lowest possible reading level.

Ask a full question in the stem.

Eliminate clues to the correct answer either within the question or across questions within a test.

Do not make the correct answer obvious to students who have not studied the material.

Highlight critical, easily overlooked words.

Have a qualified colleague read your items to ensure their appropriateness.

Double-check scoring key for accuracy before scoring.


Multiple-ChoiceAsk a complete question to the the item started, if you can.

Don’t repeat the same words within each response option; rather, reword the item stem to remove the repetitive material from below.

Be sure there is only one correct or best answer.

Choose distractors carefully.

Word response options as briefly as possible and be sure they are grammatically parallel.

Make all response options the same length.

Don’t use “all of the above” or “none of the above” merely to fill space.

Use “always” or “never” in your answer choices with caution.

It’s okay to vary the number of response options presented as appropriate to pose the problem you want your students to solve


True/False

Make the entire item entirely true or entirely false!!


MatchingProvide clear directions for making the match.

Keep the list of things to be matched short.

Keep the list of things to be matched homogenous.

Keep the list of response options brief in their wording and parallel in construction.

Include more response options than stems and permit students to use response options more than once when appropriate.


Fill-in-the-BlankAsk respondents a question and provide space for an answer.

Try to stick to one blank per item.

Don’t let the length of the line to be filled in be a clue as to the length or nature of the correct response.

Put the blank toward the end of the sentence.


Review for QualityMatch to blueprint

List/number targets to be covered

On test...mark each item with learning target number and points worth

Tally the points and write on target list

Compare with blueprint


Review for Quality

Does the item test what you intended?

Is the item well-written?


Activity 5.2 Selected Response Test Quality Checklist 1

Classroom Assessment for Student Learning: Doing It Right—Using It Well, 2e

Learning Teams Facilitation Guide and Activities & Resources Copyright © 2012 by Pearson Education, Inc., All rights reserved.


Problem # Learning Target # Confident Unsure Right Wrong Simple Mistake Don’t Get It

Reviewing/Analyzing Results


Activities for Selected Response Assessment

Develop an Assessment for Learning Activity

Prepare a Quiz or Test for Formative Use


Assessment Methods

Written response

Short answer items

Extended written response items


Written ResponseUse when...

Students are capable of writing in English (or desired language)

Less time-efficient methods are not plausible

Scoring guides are of high quality (consistency)

Learning targets are Knowledge or Reasoning


Sample Size

Prioritization should parallel the amount of time and emphasis give the various targets or topics in teaching.

If standards referenced then point distributed evenly.


Written Assessment Blueprint

Learning Target Points


Written Assessment Blueprint

Content

Pattern of ReasoningPattern of ReasoningPattern of Reasoning

Know Compare Evaluate Total

Target 1

Target 2

Target 3

TOTAL


Devising Extended Written Response ItemsKnowledge Mastery: Assess factual and conceptual knowledge

Knowledge Mastery AND Reasoning: Use factual and conceptual knowledge to form well-reasoned logical argument

Interpretive: Assess mastery of specific reasoning patterns disentangled from prerequisite knowledge


Devising Extended Written Response Items

Knowledge Mastery

Set the Context

Tell what to describe or explain

Point the way to an appropriate response



Knowledge Mastery AND Reasoning

Set the Context

Describe the reasoning task




Interpretive

Set the Context

Describe the reasoning task



Scoring Guides

Scoring List - Simply lists required tasks and the point allotment for each task

Scoring Rubric - Rates responses according to predetermined hierarchy of quality

Task-specific - Rates each task

General - Rates the whole response


Creating Task-Specific RubricBegin with correct statement of conceptual understanding

Identify characteristics of partial understanding

Identify characteristics of misunderstanding

Determine levels of rubric


Creating General Rubric

Same as task-specific rubric, but in place of content-specific statements, use general descriptions


Quality Control

Can you write an outline of a high-quality response?

Have a qualified colleague write a response and discuss the aspects of a high-quality response.


Activity 6.4 Quality Guidelines for Written 1

Response Assessments




Activities for Written Response Assessment

Create a Short Answer Item and Scoring Guide

Create an Extended Written Response Item and Scoring Guide


Assessment Methods

Performance assessment

Performance task

Performance criteria


Performance AssessmentUse it when...

Assessing skills, products, and some forms of reasoning

Working with young/primary students, or students who cannot read/write in English (or desired language

You cannot get the information you need through written assessments


Determining Sample SizeComplexity of the Target: Complex = Larger; Simple = Smaller

Decision the Evidence will Inform: How do you intend to use the results?

Consistency of Performance: Can they do it or did they just get lucky?

Proximity to Cutoff Mark: Regression to the mean


Characteristics of a Good TaskContent

Target Alignment (does the task hit the mark)

Authenticity (realistic context)

Choice (all options are equivalent)

Scaffolding (points to success without overhelping)

Interference (does not depend on unrelated or cultural background)

Availability of Resources (do they have what they need)


Characteristics of a Good TaskStructure

Knowledge students are to use (What should they know?)

What students are to accomplish (What should they do with what they know?)

Performance or product students are to create (What does the finished product look like?)

Materials to be used (What materials should they use?)

Timeline for completion (How long will they have?)

Conditions (What will the conditions of the performance be?)

Help allowed (Who can help and how?)

Criteria (What will be the focus of the assessment?)


Characteristics of a Good Task

Sampling

Use of information (How many tasks assigned? Does the task sample adequately?)

Coverage of the Target (Does the task match the target in terms of breadth?)


Characteristics of a Good Task

Activity 7.5 Rubric for Rubrics 1



SEE P241


Tasks to Elicit Good WritingRAFTS

Role: What is my role?

Audience: Who is my audience?

Format: What is the appropriate format?

Topic: What is the topic?

Strong verb: Strong verbs set the tone of a piece (see p220)


Tasks to Elicit Good WritingStrong verbs

Role: What is my role?

Audience: Who is my audience?

Format: What is the appropriate format?

Topic: What is the topic?

Strong verb: Strong verbs set the tone of a piece


Quality ControlActivity 7.4 Rubric for Tasks 1



FULL CHART P222


Structure of a RubricCriteria

Content

Organization

Delivery

Language Use

Indicators (for each Criterion)

Content

Organization

Delivery

Language Use


Structure of a Rubric

Content

Clear main topic

All information important to understanding

Facts, details, anecdotes, and/or examples make topic come alive


Structure of a RubricOrganization

Opening introduces topic and catches attention

Sequence of ideas supports meaning and is easy to follow

Transition words guide audience

Conclusion wraps up topic and leaves audience satisfied


Structure of a RubricDelivery

Eye contact

Voice

Articulation

Pace

No fillers

Gestures/movements (where appropriate)

Notes as reminders

Visual aids


http://http://youtu.be/6lrhTHRQfSU/

http://http://youtu.be/6lrhTHRQfSU/

Structure of a RubricOrganization

Opening introduces topic and catches attention

Sequence of ideas supports meaning and is easy to follow

Transition words guide audience

Conclusion wraps up topic and leaves audience satisfied


Process for Developing a RubricEstablish a knowledge base

Gather samples of student performances or products

Sort student work by level of quality

Cluster descriptors into traits

Identify samples that illustrate each level

Test the rubric and revise as needed


7 Strategies for Using Rubrics as Instructional Tools in the ClassroomWhere am I going?

1. Provide students with a clear and understandable vision of the learning target

2. Use examples and models of strong and weak work

Where am I now?

3. Offer regular descriptive feedback

4. Teach students to self-assess and set goals

How can I close the gap?

5. Design lessons to focus on one learning target or aspect of quality at a time

6. Teach students focused revision

7. Engage students in self-reflection and let them keep track of and share their learning


Activities for Performance Task

Create a Rubric

Evaluate a Rubric for Quality

Create a Student-friendly Version of a Rubric


Assessment MethodsPersonal communication

Questions during instruction

Interviews and conferences

Participation

Oral exams

Student journals and logs
