Participant Handouts
Assessment Literacy
(Winter 2014)
Berks Intermediate Unit
Carbon Lehigh Intermediate Unit
Knowledge of Assessment
Factors That Impact the Effectiveness of an Assessment
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
________________________________________________________________ _________
1
4
19
Name of Assessment
Whats the Purpose of this assessment?
Who Is Being Assessed?
How Is The Data Being Used?
Actual Type of Assessment
Is it being used as it should be? Are you using the right assessment? (Yes or No)
EXAMPLE:
End of course exam
Measure mastery of course standards
All 7th graders
Determining final grades and a screener for 8th grade courses
Types of Assessments
Principles of Well-Developed Measures
Multiple Choice Guidelines:
Items consist of a stem and answer options
Grades K 1 three options
Grades 2 12 four options
Stems
State as a direct question when possible
Include word(s) that might be repeated in the answer choices
Answer options
Answer options contain only one correct answer
Other answer options (distractors) are the same structure and length as the correct answer
Distractors should be plausible
Correct answer placement should be balanced (approximately the same number of As, Bs, etc)
Avoid answer options that provide clues to the answer
Answer options should be in ascending or descending order when possible
Avoid all of the above and None of the above
Directions are clear and precise; include point values
Can be written at DOK levels 1, 2, & 3
Question written as multiple choice will often lower DOK level; craft question and answer options carefully
Constructed Response Guidelines:
Two components: Item/task and scoring rubric
Short: usually 2 points
Extended: usually 4 points
Item/Task
Student expectations are clear-
Explain vs. Discuss
Describe vs. Comment
State the extent of the expected answer
Give three reasons vs. give some reasons
Directions state what to do, where and how to respond, and point values
Rubric
Begin with a generic rubric and modify language using specific criteria expected in the response to award the maximum number of points
Determine how much the answer can deviate from the highest points award for the next lower; continue for all scores
Create an example of a response for each level in the rubric
Verify alignment between item/task, scoring rubric, and sample responses
POP QUIZ
Reliability and Assessment
1. A parent called you to ask about the reliability coefficient on a recent standardized test. The coefficient reported was .89 and the parent thinks that must be a very low number. How would you explain that .89 is an acceptable coefficient?
2. Your school district is looking for an assessment instrument to measure reading reliability. They have narrowed the selection to two possibilitiesTest A provides data indicating that it has high validity, but there is no information about its reliability. Test B provides data indicating that is has high reliability, but there is no information about its validity. Which Test would you recommend? Why?
Hess Cognitive Rigor Matrix & Curricular Examples: Applying Webbs Depth-of-Knowledge Levels to Blooms Cognitive Process Dimensions Math/Science
Revised Blooms Taxonomy
Webbs DOK Level 1
Recall & Reproduction
Webbs DOK Level 2
Skills & Concepts
Webbs DOK Level 3
Strategic Thinking/ Reasoning
Webbs DOK Level 4
Extended Thinking
Remember
Retrieve knowledge from long-term memory, recognize, recall, locate, identify
Recall, observe, & recognize facts, principles, properties
Recall/ identify conversions among representations or numbers (e.g., customary and metric measures)
Understand
Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, infer a logical conclusion (such as from examples given), predict,
compare/contrast, match like ideas, explain, construct models
Evaluate an expression
Locate points on a grid or number on number line
Solve a one-step problem
Represent math relationships in words, pictures, or symbols
Read, write, compare decimals in scientific notation
Specify and explain relationships (e.g., non-examples/examples; cause-effect)
Make and record observations
Explain steps followed
Summarize results or concepts
Make basic inferences or logical predictions from data/observations
Use models /diagrams to represent or explain mathematical concepts
Make and explain estimates
Use concepts to solve non-routine problems
Explain, generalize, or connect ideas using supporting evidence
Make and justify conjectures
Explain thinking when more than one response is possible
Explain phenomena in terms of concepts
Relate mathematical or scientific concepts to other content areas, other domains, or other concepts
Develop generalizations of the results obtained and the strategies used (from investigation or readings) and apply them to new problem situations
Apply
Carry out or use a procedure in a given situation; carry out (apply to a familiar task), or use (apply) to an unfamiliar task
Follow simple procedures (recipe-type directions)
Calculate, measure, apply a rule (e.g., rounding)
Apply algorithm or formula (e.g., area, perimeter)
Solve linear equations
Make conversions among representations or numbers, or within and between customary and metric measures
Select a procedure according to criteria and perform it
Solve routine problem applying multiple concepts or decision points
Retrieve information from a table, graph, or figure and use it solve a problem requiring multiple steps
Translate between tables, graphs, words, and symbolic notations (e.g., graph data from a table)
Construct models given criteria
Design investigation for a specific purpose or research question
Conduct a designed investigation
Use concepts to solve non-routine problems
Use & show reasoning, planning, and evidence
Translate between problem & symbolic notation when not a direct translation
Select or devise approach among many alternatives to solve a problem
Conduct a project that specifies a problem, identifies solution paths, solves the problem, and reports results
Analyze
Break into constituent parts, determine how parts relate, differentiate between relevant-irrelevant, distinguish, focus, select, organize, outline, find coherence, deconstruct
Retrieve information from a table or graph to answer a question
Identify whether specific information is contained in graphic representations (e.g., table, graph, T-chart, diagram)
Identify a pattern/trend
Categorize, classify materials, data, figures based on characteristics
Organize or order data
Compare/ contrast figures or data
Select appropriate graph and organize & display data
Interpret data from a simple graph
Extend a pattern
Compare information within or across data sets or texts
Analyze and draw conclusions from data, citing evidence
Generalize a pattern
Interpret data from complex graph
Analyze similarities/differences between procedures or solutions
Analyze multiple sources of evidence
analyze complex/abstract themes
Gather, analyze, and evaluate information
Evaluate
Make judgments based on criteria, check, detect inconsistencies or fallacies, judge, critique
Cite evidence and develop a logical argument for concepts or
