Summer 2012 DataCamp

June/August, 2012

Everyone is using standards…

What Do We Ask Ourselves Before Building an

Assessment Instrument?

• What do we want to measure?• What information do we want to

gather?• How long do we want the assessment to

take?• How will we determine which standards

we want to assess?• How will we use the results?

Guiding Principles for Building Assessment

Instruments• A minimum of three items for every

standard you want to measure.• Spread items measuring an individual

standard across the assessment.• Range of overall assessment difficulty and

for each standard measured.• Begin assessment with easier items, build

to more difficult and conclude with easier items.

Guiding Principles for Building Assessment

Instruments

• Spread items with graphics across test.• Readability levels for reading passages; genres

needed.• A minimum of five items per passage.• Even spread of answer choices across

assessment when possible; spread them out across the form.

• Check items for appropriateness to use online, paper, or both.

Use a blueprint to build validitySource: Ernie Bauer and Ed Roeber, “Technical Standards for

Locally-Developed Assessments”, April 8, 2010

• A grid is used to summarizes the content and format of the test.– The rows are the learning objectives– The columns are the level of cognitive complexity

• The cells list the types and numbers of items.• You can place a margin that can be used to

sum the total points.

Test Blueprint Depth of Knowledge Recall Skills/Concepts Strategic Extended Target Totals

# Points # Points # Points # Points # Points

Learnin

g Targets

DOK Totals

Excellent Examples

Webb (1997) Depth of Knowledge • Recall • Skills and Concepts• Strategic Thinking• Extended Thinking

Anderson, & Krathwohl (2001)

Revised Bloom’s Taxonomy

Depth of Knowledge

• Recall includes facts, definitions, terms, or a simple procedure,– “identify”, “recall” ,“recognize”

• Skills and Concepts includes some processing beyond a habitual response – “classify”, “organize”, “compare”

• Strategic requires reasoning, planning, or using evidence– “justify” ,“draw conclusion”, “explain”

• Extended Thinking requires complex reasoning, planning, developing, and thinking. – “evaluate”, “design”, “generalize”

Reverse DesignSource: Ernie Bauer, and Jim Gullen, “Locally Developed Assessment:

What Do the Results Mean?”, April 2010

• Start with your items• Group the items by content (GLCE/HSCE)– This will give you the rows for the blueprint

• Split the items in the content piles by DOK– This will give you the columns for the blueprint

• Create a blueprint document to check if this provides acceptable evidence for you.

Time Elements of an Assessment CalendarSource: White, S. H. (2005). “Beyond the Numbers: Making Data Work for Teachers and School

Leaders”. Lead and Learn Press: Englewood, CO

• When will we administer the assessment?• When will we collect the data?• When will we disaggregate the data?• When will we analyze the data?• When will we reflect upon the data?• When will we make recommendations?• When will we make the decisions about the

recommendations?• When will we provide written documentation about the

decisions?• When will we share the data with other stakeholders?

“You Be George” A Formative Assessment Activity

• Imagine that you have just received some feedback on a recent science assessment.

• Using the information from your teacher,– Identify which learning targets you got

right– Identify which learning targets you

need to review– Identify which learning targets you

need to keep practicing

Questions?Stan MastersCoordinator, Instructional Data ServicesLenawee Intermediate School District4107 N. Adrian HighwayAdrian, Michigan 49921

517-265-1606 (phone)517-265-2953 (fax)stan.masters@lisd.us

Mike HusbandData Entry SpecialistLenawee Intermediate School District4107 N. Adrian HighwayAdrian, Michigan 49921

517-266-6981 (phone)517-265-7405 (fax)mike.husband@lisd.us

Data Warehouse webpage:www.lisd.us/links/data