Assessment Literacy Series

1

-Orientation Module-

What is Assessment Literacy?

2

A process of designing, building, and reviewing performance

measures for use in evaluating student achievement.

1. Develop high-quality performance measures for use within the greater educator effectiveness system.

2. Increase assessment literacy of participants.

3. Gain a deeper understanding of academic content standards.

3

Series Objectives

Assessment Life Cycle

4

The Assessment Literacy training contains six (6) modularized components within three (3) action strands:◦ DESIGN

1. M1-Design & Purpose Statement2. M2-Test Specifications & Blueprints

◦ BUILD3. M3-Item Specifications4. M4-Scoring Keys & Rubrics5. M5-Operational Forms & Administrative

Guidelines◦ REVIEW

6. M6-Quality Assurance & Form Reviews

5

Overview of the Development Process

Purpose Statement•Explain why measures are made, what they will measure, and how the results will be used.

Targeted Standards•Select content standards associated with the performance measure.

6

Development Process

Test Specifications and Blueprints•Outline the types of items being used, point values, and depth of knowledge (DoK) distribution. •Guide item development by focusing items/tasks on discrete aspects of the content standards. Item/Tasks Specifications•Articulate the type and characteristics of items/tasks that align to the test specifications and blueprints.

7

Development Process (cont.)

Scoring Keys & Rubrics•Provide answers for multiple choice items and detailed scoring rubrics for constructed response tasks.

Operational Forms•Organize and sequence items/tasks within an outlined format that includes item tags (to uniquely identify each item/task).

8

Development Process (cont.)

Administrative Guidelines•Provide detailed instructions for how to prepare, administer, and collect results.

Quality Assurance & Form Reviews•Review all operational forms, scoring keys/rubrics, and guidelines to ensure pre-established quality expectations have been met.•Determine the overall rigor and alignment at the item/task-level and for the overall operational form.

9

Development Process (cont.)

Guides•Handout #1 – Purpose Statement Examples•Handout #2 – Targeted Standards Example•Handout #3 – Test Specifications & Blueprint Example•Handout #4 – Depth of Knowledge (DoK) Chart•Handout #5 – Item Examples•Handout #6 – Scoring Key Example•Handout #7 – Rubric Examples•Handout #8 – Operational Form Example•Handout #9 – Administrative Guidelines Example•Handout #10 – Quality Assurance Checklist

10

Helpful Materials

Templates • Template #1-Purpose Statement•Template #2-Targeted Standards•Template #3-Test Specifications & Blueprint•Template #4-Scoring Key•Template #5-Operational Form•Template #6-Administrative Guidelines

11

Helpful Materials (cont.)

Stuff

“Smart Book”◦Provides techniques and technical criteria for developing measures.

Performance Rigor Checklist◦A quick screening tool used to examine item/task quality.

12

Helpful Material (cont.)

Principles of Well-developed Measures

Measures must:• Be built to achieve the designed purpose• Produce results that are used for the intended

purpose• Align to targeted content standards• Contain a balance between depth and breadth

of targeted content• Be standardized, rigorous, and fair• Be sensitive to testing time and objectivity• Be valid and reliable

13

Evaluating Measures

Quality Assurance RubricsThree (3) versions-◦Teacher◦District◦State or Vendor• Comprehensive tools that evaluate the

technical process used in developing performance measures.• Organized into strands and provides an

overall rating.

14

Decision-Making Approaches

Build Consensus Approach• All opinions expressed• Ideas of all participants gathered and

synthesized• Group agreement reached

Team Leader Approach• Guiding questions • Group discussion about the pros and cons• Create a “best fit” course of action

15

Think-Pair-Share

5 minutes to think about the statement below and write down your thoughts

5 minutes to share your ideas with a partner10 minutes to share with your group

“The Assessment Literacy process will help me as an educator by _______________.”

16

SummaryOrientation ModuleIntroduced the Assessment Literacy process used to create high quality measures of student achievement.

Next StepsModule 1: Design & Purpose StatementDesigning performance measures, including creating purpose statements and selecting academic content standards.

17

Summary & Next Steps