STAAR Systems: Developing Quality Local Assessments

Dr. Stephanie Zelenak

Norms (we do have to spend a lot of time together after all )

1. Breaks should be taken as a group. It is critical that the whole group be around the table working together to the extent possible.

2. Limit side conversations and use of electronics for non-assessment development purposes

3. Staying on pace/task- the process is key AND we are not in a race.

4. More heads are better than one…in this case we want to avoid the concept of divide and conquer! There is no “I” in TEAM.

Norms (continued)

6. Think big, start small. What are our priorities and objectives?!?!

7. Quality over Quantity- we have this year to EARN buy in.

Materials List Every team working on a CBA will need the following:

Binder and 8 dividers per team per Assessment Assessment Blueprint template (hard copy and electronic version) Validation and Discussion Worksheet template (hard copy and

electronic version) Assessment Checklist template (hard copy and electronic version) lead4ward data heat maps comparing STAAR 2012 and 2013 results

for each grade/course the team will be developing. Data heat maps should be by standard type in math and science and by reporting category in ELAR and SS.

Curriculum documents for each grade/course the team will be developing.

Access to Lead4ward snapshots, IQ templates and scaffold documents

Access to district item banks and/or printed test items aligned to SEs that will be included in designated assessment for each grade/course the team will be developing.

Let’s Get Organized! Constructing our team binders:

CBA #1 (on colored paper): Label your dividers by CBA- 1. Data Heat Maps 2. Curriculum Documents (YAGs, TVMs, IFDs) 3. Lead4ward Snapshots and Scaffold Documents 4. Blueprints & Step by Step 5. Validation & Discussion Worksheet & CBA

Checklist 6. Hard copies of test items and Lead4ward IQs 7. Notes/Miscellaneous (i.e. Responsibility Sheet, Test

Question Framework, Team Lists) 8. Resources & References (i.e. DOK, workshop packet,

Ppts, etc.)

Developing an Assessment Blueprint:

1. Determine Units of Study to be Covered – Unit(s) of study (YAG)

2. Determine # of Standards: – Count Standards and Determine Breakdown (R/S/P)

3. Prioritize Standards – Readiness / Supporting – Hard to reach / hard to teach (data heat maps) – Frequency/Timing in Curriculum (YAG) – Complexity of concept/content (data and perception) – Instructional focus during the timeframe being assessed (IFD)

PG. 28

Developing an Assessment Blueprint (cont.):

4. Fill in SE’s, Unit, # of Questions per SE 5. Notate Verbs

– Look for multiple verbs

6. Determine level of Cognitive Complexity of each SE (DOK)

PG. 28

Developing a Quality Assessment: Choose/Develop Items for each SE

– Focus on one SE at a time – Deconstruct the SE, understand the levels of specificity

(content, verb, level, R/S, prioritization in unit) – Gather several assessment items – Review blueprint for the number of items needed – Choose the best items available and look for novelty – Review for accuracy, validity with the Validation &

Discussion worksheet

Developing a Quality Assessment:

Be Mindful of STAAR (& other important considerations)

– Match of the SE: At the level of the standard or above – Novelty – Rigor: A good mix- balance / don’t go crazy… use DOK level

of SE as your guide – Ratio of Readiness & Supporting standards – Learned vs. Taught/Applied: Mix of direct taught /

application – Dual coded/Griddables/Short Answer/Etc. – Unit specificity of SEs

Validating Existing Assessments:

Which assessment is in the center of the table? Are the curriculum documents present? Have the teachers actually taken the assessment?

PG. 37

Assessment Validation Items for Consideration: • Cognitive complexity of the item (rigor of the

question itself) • Academic Vocabulary • Context/Novelty • Weighting of Reporting Category • Standard Type (Readiness, Supporting, Process-

Dual Coded) • DOK Level • Taught versus Learned • Other considerations…What do you see?

As you are selecting items, be sure to be asking yourself:

• What is the Readiness/Supporting ratio of the items selected?

• Do the selected items align to the Content and Cognitive Specificity of the SEs during the designated instructional timeframe?

• Do the selected items align to the Instructional Focus during the designated instructional timeframe?

• Have we included enough Dual Coded, Griddable, Short Answer, Etc. Items?

• Have we selected items that utilize the Academic Vocabulary of the SEs?

As you are selecting items, be sure to be asking yourself:

• Are we using what we know about our Hot Spots and Hard to Teach / Hard to Reach SEs when selecting the number of items per SE?

• Based on the data, are there any additional items that should be selected/included?

• Have we considered the Frequency/Timing in Annual Curriculum when selecting the number of items per SE?

• Have we considered the Complexity of concept/ content when selecting the number of items per SE?

• Have we selected novel items that allow students to practice with learned AND applied learning items?