Assessment “We assess students not merely to evaluate them, but to improve the entire process of...

Assessment

“We assess students not merely to evaluate them, but to improve the entire process of teaching and

learning.”- Douglas B. Reeves, Making Standards Work

Designing STAAR-Like Quality Assessments

Learning Objective

Participants will be able to create more rigorous assessments by analyzing the content and cognitive level of state released test questions.

Why Assessment Training?

Teacher and campuses are overwhelmed with the number of assessments they have to give

How do we target High Priority TEKS? How is a good unit assessment

constructed? Individual teacher assessments should

follow the guidelines of quality assessment

Agenda

Six Steps to Creating Quality Assessments

1. Data2. Content3. Processes4. Review Examples5. Item & Test Leveling6. Item Design

Look at the Data

Review the District Learning Profile.

Which SEs are the lowest?

Which SEs had the most questions?

Analyzing the content , concepts, processes, and skills that will be

assessed(Cognitive and Procedural Knowledge)

TEKS and Assessment: Things to Remember

The wording of the standard tells us: WHAT CONTENT will be assessed on STAAR

and AT WHAT LEVEL the standard will be assessed

on STAAR

Cognitive and Content Expectations Content

The content items for which students must demonstrate understanding at the appropriate cognitive level in order to adequately meet the standard.

Cognitive The level at which students are expected to perform in

order to adequately meet the standard. Determined by the verbs used in BOTH the Knowledge

and Skills statements and the Student Expectations

What Should Students be able to DO ?

Look at the cognitive level of the verb

• If the cognitive level of the SE is UNDERSTAND, what does that mean students have to be able to do?

Understanding:  Constructing meaning from different types of

functions be they written or graphic messages activities like

interpreting, exemplifying, classifying, summarizing, inferring,

comparing, and explaining. 

ELAR – Grade 8

8.4  Comprehension of Literary Text/Poetry. Students understand, make inferences and draw conclusions about the structure and elements of poetry and provide evidence from the text to support their understanding. The student is expected to:– (A) compare and contrast the relationship

between the purpose and characteristics of different poetic forms (e.g., epic poetry, lyric poetry). Supporting Standard

ELAR – Grade 8

Figure 19 Reading/Comprehension Skills.Students use a flexible range of metacognitive reading skills in both assigned and independent reading to understand an author’s message. Students will continue to apply earlier standards with greater depth in increasingly more complex texts as they become self-directed, critical readers. The student is expected to: – (D) make complex inferences about text

and use textual evidence to support understanding

Bundling and Dual Coding

Brain Research and Dual Coding

Application of Knowledge and Building Schema The brain learns new knowledge (content) by

attaching that knowledge to existing schema The brain builds schema by applying (do)

conceptual and content knowledge in a variety of novel ways

You can most effectively test conceptual knowledge through application questions

STAAR and Dual Coding

Math = 75% of items Science = 40% of items Social Studies = 30% English = 60%???

Figure 19

As we review the following examples, think about the implications of test and item design

ELAR: Dual Coding with Figure 19

ELAR: Dual Coding with Figure 19

Your Turn: ACTIVITY

Look at the STAAR released items in your handout.

Identify what content is being assessed. Identify the skill being assessed. Identify how the student has to apply

content with the selected skill. Discuss with your group.

STAAR Overview: Generalizations

Verbiage in the questions is complex and reflective of the TEKS

Supporting Standards REALLY are being tested! More questions at a higher level Inference from stimulus; no direct answers (no clues or

cues) Complex distracters - 4 viable answer choices Dual coding is prevalent Greater depth of content knowledge required to answer

each question

Creating an Assessment Blueprint

Step 5:Determining Cognitive and Procedural Difficulty Levels

Aligning Item and Test Difficulty to STAAR

Determining Difficulty Level

Cognitive Difficulty (verbs) EASY

Remember Understand

MEDIUM Apply Analyze

HARD Evaluate Create

Determining Difficulty Level

Procedural DifficultyThis is basically: How many mental

processing steps does the student have to go through to answer the question?

The greater the number of processing steps the higher the difficulty level

Determining Difficulty Level

Procedural Difficulty EASY

The item includes only the stem and the answer choices MEDIUM

The item includes a graphic, short reading selection, map etc. (Stimulus piece) The student only has to interpret the stimulus or pull information from it to select the correct answer.

HARD The item includes a graphic, short reading selection, map

etc. (Stimulus piece) The student has to infer, analyze, summarize, etc. and apply that to the stem or answer choices to select the correct answer.

Your Turn: Determining Item Difficulty

1. Look at the following items

2. Identify the cognitive difficulty of the item

3. Identify the procedural difficulty of the item

Determining Item Difficulty

Determining Item Difficulty

Item Writing Checklist

Demonstration of Learnig

After having determined the cognitive and procedural difficulty of a test item, learners will rewrite the question so that it is either up a level or down a level.

Creating an Assessment Blueprint

Identify the process/skills SE that you are pairing with the content SE

Write these on the Blueprint Use a variety of processes and skills on the

test and throughout the year.

Design Your Test So….

How many items per SE? Cognitive Level- Make 70% of the test easy-

medium and 30% difficult. Procedural Level- Make 60% easy

procedural and 40% difficult. Use same vocabulary throughout.

Next Steps: What are the implications of today’s learning for your campus?

What are things you will change on your assessments?

What are things you are still pondering?

What is something you want to know more about?

Bonus Slides: Examples of Higher Level Stems Which is an example of…

Who would most likely have written (asked, said…) followed by a

quote

Analogies Fill in the missing part of this graphic organizer

Which of the following does not belong?

What is the best category for the following?

Give a series of clues to the name of a person, place, etc. (riddle format)

Cloze passage

Hypothetical situation – analysis

Sequencing by Roman numeral

Who would have been helped by (law, invention, organization –

hypothetical)

Use a variety of stimuli

Textbox of primary source material Graphs and Charts Pictures or illustrations Spoke Diagrams Multiple visuals: comparison maps, charts Political Cartoons Time lines Flow Charts Graphic designs Headlines Speaker Questions

Qualities of a Good Multiple-Choice Item Effectively written stems Focused on one idea Clear and concise Phrased as a direct question Phrased positively Has only one best and correct answer Contains options that are plausible, but incorrect Has four choices that are homogeneous, parallel

in structure, and logical

Qualities of a BAD Multiple-Choice Item Tests trivial information Contains unnecessary/confusing information in the

stem or options Is tricky or cute Gives cues or clues to the correct answer Does not have plausible answers Poses a question for which many defensible

answers are possible Contains a bias toward or against a group of

individuals

Develop plausible distractors

Plausible distracters include… common misconceptions statements that sound logical but that are not based on the

material given in the stimulus and question common or familiar phrases statements that are true but that do not answer the

question

Be sure there is not more than one plausible answer.

Organization of Answer Choices Alphabetical Ascending or descending order Shortest to longest Same order as presented in stimulus