Teacher Evaluation Institute Assessment Literacy for Student Achievement Goal Setting 1 Division of Teacher Education and Licensure September 2013

1

Teacher Evaluation Institute

Assessment Literacy for Student Achievement Goal Setting

Division of Teacher Education and Licensure

September 2013

2

Standard 7: The work of the teacher results in acceptable, measurable, and appropriate student academic progress.

Virginia’s Uniform Performance Standards for Teachers

3

7.1 Sets acceptable, measurable and appropriate achievement goals for student academic progress based on baseline data.

7.2 Documents the progress of each student throughout the year.

7.3 Provides evidence that achievement goals have been met, including the state provided growth measure when available as well as other multiple measures of student growth.

7.4 Uses available performance outcome data to continually document and communicate student academic progress and develop interim learning goals.

Standard 7: Performance Indicators

4

Assessment Data Sources

• Student Growth Percentiles (SGPs) based on Standards of Learning (SOL) assessments

• Other standardized assessments

• Common assessments (developed for the division or school level)

• Teacher-developed assessments

5

Guiding Questions

1. What role can teacher-developed assessments play in demonstrating student learning?

2. How can the validity and reliability of teacher-developed assessments be strengthened?

3. How can teachers analyze results from students’ performance on teacher-developed assessments and demonstrate student progress?

4. How can a teacher and/or principal determine what currently available assessments may be appropriate for establishing baseline data for student achievement goal setting?

6

Guiding Question 1

What role can teacher-developed assessments play in demonstrating student learning?

7

Standard 4: The teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year.


8

CommonUnit Test

Key Skill Performance Assessment

Demonstrated Student

Progress

Standard 7Standard 4

Other Teacher-Developed

Assessments

9

Guiding Question 2

How can the validity and reliability of these assessments be strengthened?

10

Assessment Literacy

The ability to create and use valid and reliable assessments as a classroom teacher to facilitate and communicate student learning.

11

Desired Characteristics of Teacher-Developed Assessments

Reliable

Practical to Employ

Instructionally Useful

Valid

12

Overview of the Process for Creating a Valid and Reliable Teacher-Developed Assessment

1. Identify objectives for a unit of instruction.

2. Unpack the objectives.

3. Map the objectives onto a “table of specifications.”

4. Use the table of specifications to guide the construction of the assessment.

5. Use the table of specifications to analyze student learning.

13

Definition: Curriculum

A set of objectives for student learning.

14

4.16 The student willa) recognize and demonstrate the meaning of equality in an equation; andb) investigate and describe the associative property for addition and multiplication.

UNDERSTANDING THE STANDARD(Background Information for Instructor

Use Only)ESSENTIAL UNDERSTANDINGS ESSENTIAL KNOWLEDGE AND SKILLS

Investigating arithmetic operations with whole numbers helps students learn about several different properties of arithmetic relationships. These relationships remain true regardless of the numbers.

The commutative property for addition states that changing the order of the addends does not affect the sum (e.g., 4 + 3 = 3 + 4). Similarly, the commutative property for multiplication states that changing the order of the factors does not affect the product (e.g., 2 3 = 3 2).

The associative property for addition states that the sum stays the same when the grouping of addends is changed [e.g., 15 + (35 + 16) = (15 + 35) + 16]. The associative property for multiplication states that the product stays the same when the grouping of factors is changed [e.g., 6 (3 5) = (6 3) 5].

All students should

Understand that mathematical relationships can be expressed using equations.

Understand that quantities on both sides of an equation must be equal.

Understand that the associative property for addition means you can change the groupings of three or more addends without changing the sum.

Understand that the associative property for multiplication means you can change the groupings of three or more factors without changing the product.

The student will use problem solving, mathematical communication, mathematical reasoning, connections, and representations to

Recognize and demonstrate that the equals sign (=) relates equivalent quantities in an equation.

Write an equation to represent equivalent mathematical relationships (e.g., 4 3 = 2 6).

Recognize and demonstrate appropriate use of the equals sign in an equation.

Investigate and describe the associative property for addition as (6 + 2) + 3= 6 + (2 + 3).

Investigate and describe the associative property for multiplication as (3 x 2) x 4 = 3 x (2 x 4).

15

Sources of Objectives for Instructional Units

Objectives for an instructional unit

VA SOL

Local curriculum (including text series)

Teacher (as subject-area expert)

Wha

t get

s ta

ught

16

Sample Objectives for a Unit on “Equivalency and the Associative Property” (SOL 4.16)

The student will:

a. Recognize and demonstrate the meaning of equality in an equation (i.e., an equals sign [=] relates

equivalent quantities in an equation).

Write equations to represent equivalent mathematical relationships (e.g., 4 × 3 = 2 × 6).

b. Investigate and describe the associative property for addition, such as (6 + 2) + 3 = 6 + (2 + 3).

Use mathematical communication to demonstrate understanding that the associative property for

addition means you can change the groupings of three or more addends without changing the

sum.

c. Investigate and describe the associative property for multiplication, such as (3 x 2) x 4 = 3 x (2 x 4).

Use mathematical communication to demonstrate understanding that the associative property for

multiplication means you can change the groupings of three or more factors without changing the

product.

d. Infer how the associative property does not operate for subtraction and division by exploring and

proposing explanations of non-examples of the associative property.

Sour

ce: V

A SO

LSo

urce

: The

teac

her

17

Unpacking Objectives

ContentCognitiv

eLevel

Objective

18


The student will describe the

planets and their relative

positions from the sun.

19

Bloom’s Taxonomy of Cognitive Behaviors

Evaluation

Synthesis

Analysis

Application

Comprehension

Knowledge

20

Bloom’s Taxonomy VerbsTaxonomy Level Associated Verbs

Knowledge count, define, describe, draw, find, identify, label, list, locate, name, state, recall, recite, record, tell

Comprehension clarify, classify, compare, contrast, describe, exemplify, explain, extrapolate, give examples, identify, illustrate, infer, interpret, outline, paraphrase, report, represent, restate, summarize, tell, translate

Application apply, carry out, compute , demonstrate, dramatize, draw , execute, illustrate , implement, prepare, select, show, solve, transfer, use

Analysis analyze, break down, characterize, deconstruct, deduce, differentiate, discriminate, distinguish, examine, infer, investigate, relate, separate

Synthesis adapt, create, design, develop, formulate, integrate, invent , imagine, make, modify, perform, plan, predict, produce, propose

Evaluation argue, assess, choose, conclude, critique, decide, evaluate, judge, justify, predict, prioritize, prove, rank, rate, select

21


The student will describe the

planets and their relative

positions from the sun.

comprehension

22

Unpacking Objectives for Content and Cognitive Level

The student will:

a. Recognize and demonstrate the meaning of equality in an equation (i.e.,

an equals sign [=] relates equivalent quantities in an equation).

b. Investigate and describe the associative property for addition, such as (6

+ 2) + 3= 6 + (2 + 3).

c. Investigate and describe the associative property for multiplication, such

as (3 x 2) x 4 = 3 x (2 x 4).

d. Infer how the associative property does not operate for subtraction and

division by exploring and proposing explanations of non-examples of the

associative property.

CO

CO

CO

AP

AN

AN

AN

AN

SY

CO: comprehension AN: analysisAP: application SY: synthesis

23

Unpacking Objectives• The student will identify the narrator of a short story.

• The student will find the surface area of a rectangular prism.

• The student will list the qualifications necessary to vote in Virginia and describe the process for registering to vote in Virginia.

• The student will create a diagram that demonstrates the steps in the cell cycle, including the phases of mitosis.

24

Unpacking Objectives• The student will identify the narrator of a short story.

• The student will find the surface area of a rectangular prism.

• The student will list the qualifications necessary to vote in Virginia and describe the process for registering to vote in Virginia.

• The student will create a diagram that demonstrates the steps in the cell cycle, including the phases of mitosis.

CO

AP

KN

CO

CO

KN: knowledge CO: comprehension AP: application

25


• The student will write a persuasive essay on a school-wide issue.

• The student will verify the properties of circles.

• By reviewing slogans in post-World War II America, the student will describe changes in economic opportunities for women.

26


• The student will write a persuasive essay on a school-wide issue.

• The student will verify the properties of circles.

• By reviewing slogans in post-World War II America, the student will describe changes in economic opportunities for women.

EV

AP

AN

AN

AP: application AN: analysis EV: evaluation

27

Cautions About Unpacking

1. Bloom’s taxonomy may be incorrectly viewed as hierarchical in nature.

2. Not all learning objectives fit neatly into one category of the taxonomy.

3. Most words associated with the taxonomy have different meanings in different contexts.

4. There are other taxonomies of cognitive behaviors besides Bloom’s.

5. Bloom’s taxonomy addresses the cognitive domain only, not the psychomotor or affective domains.

6. Subject-matter expertise is critical in unpacking objectives.

28

Table of Specifications

A blueprint for what should be included (and should not be included) on an assessment.

29

30

ContentCognitive Level

Knowledge Compre-hension

Application Analysis Synthesis Evaluation

31

Steps to Create aTable of Specifications

Step #1 – Unpack objectives for…

• Content

• Cognitive level.

Step #2 – Plot the intersection between the content and level of cognitive demand for each learning objective on a matrix.

Step #3 – If helpful, indicate the relative emphasis of each intersection on the chart.

32


Know-ledge

Compre-hension

Appli-cation

Analysis Syn-thesis

Evalu-ation

the meaning of equality in an equation

the associative property for addition

the associative property for multiplication

how the associative property does not operate for subtraction and division (teacher-developed objective)

recognize

demonstrate

describe

investigate

describe

investigate

explore

infer

propose explana-

tions

Math 4.16

33

Using a Table of Specifications

1. To create an assessment.

2. To critique and improve an existing assessment.

3. To create a unit assessment plan.

34

Sixth-Grade Unit: “Resource Use and Conservation” (Science SOL 6.9)

Science SOL 6.9 Overview

“The strand focuses on student understanding of the role of resources in the natural world and how people can utilize those resources in a sustainable way. An important idea represented in this strand is the importance of managing resources. This begins with basic ideas of conservation and proceeds to more abstract consideration of costs and benefits. The topics developed include conservation of materials, soil and plants as resources, energy use, water, Virginia’s resources, and how public policy impacts the environment.”

35

Objectives for the “Resource Use and Conservation” Unit (Science 6.9)

The student will:

a. Differentiate between renewable and nonrenewable resources.

b. Describe the role of local and state conservation professionals in managing natural

resources. These include wildlife protection; forestry and waste management; and

air, water, and soil conservation.

c. Analyze reports, media articles, and other narrative materials related to waste

management and resource use to determine various perspectives concerning the

costs/benefits in real-life situations.

d. Analyze how renewable and nonrenewable resources are used and managed within

the home, school, and community.

e. Analyze resource-use options in everyday activities and determine how personal

choices have costs and benefits related to the generation of waste.

f. Evaluate the impact of resource use, waste management, and pollution prevention in

the school and home environment.

36


Know-ledge

Compre- hension

Applica-tion

Analy-sis

Synthe-sis

Evalua-tion

Renewable and nonrenewable resources

Differentiate

between

Role of local and state conservation professionals in managing natural resources

Describe

Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations

Determine

Analyze

Resource use and management in the home, school, and community

Analyze

Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation

Analyze

Deter-mine

personal choices

Impact of resource use, waste management, and pollution prevention in school and at home

Evaluate

What would you expect to see on a test that assesses these objectives?

37

Unit TestQuestion #1

Which of these is a nonrenewable resource?

A. fossil fuelsB. sunlightC. treesD. wind

What content does this item assess

and at what cognitive level?

38


Know-ledge

Compre- hension

Applica-tion

Analy-sis

Synthe-sis

Evalua-tion

Renewable & nonrenewable resources

Differentiate

between


Describe


Determine

Analyze


Analyze


Analyze

Deter-mine

personal choices


Evaluate

1

39


What content does

this item assess

and at what

cognitive level?

40


Know-ledge

Compre- hension

Applica-tion

Analy-sis

Synthe-sis

Evalua-tion


Differentiate

between


Describe


Determine

Analyze


Analyze


Analyze

Deter-mine

personal choices


Evaluate

1

9

41


What content does

this item assess

and at what

cognitive level?

42


Know-ledge

Compre- hension

Applica-tion

Analy-sis

Synthe-sis

Evalua-tion


Differentiate

betweenRole of local and state conservation professionals in managing natural resources

Describe


Determine

Analyze


Analyze


Analyze

Deter-mine

personal choices


Evaluate

1

14

9

43


Know-ledge

Compre- hension

Applica-tion

Analy-sis Synthe-sis

Evalua-tion


1, 2, 16 Differentiate

Role of local and state conservation professionals in managing natural resources 11

Describe

205, 10


Determine

97, 12,

15

Analyze4, 8

Resource use and management in the home, school, and community 14 6

Analyze


17

Analyze

Deter-mine

personal choices


3, 13, 18, 19

Evaluate

How VALID is this test?

44


Know-ledge

Compre- hension

Applica-tion

Analysis Syn-thesis

Evalua-tion

Renewable & nonrenewable resources 1, 2

Differentiate5, 10, 16

Role of local and state conservation professionals in managing natural resources 11

Describe

7, 20, 24Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations

Determine

9, 12

Analyze

4, 8, 21Resource use and management in the home, school, and community

14 6

Analyze18, 19, 25


17

Analyze15, 22, 23

Deter-mine

personal choices


3, 13

Evaluate

Is this a more valid test?


KN CO AP AN SY EV


Differentiate


Describe


Determine

Analyze


Analyze


Analyze

Determine personal choices


Evaluate

Using a TOS as a “unit assessment plan”

MC Test

MC Test MC Test

MC Test

MC Test MC Test

MC Test MC TestMC Test

MC Test MC Test

MC Test

Personal Resource Use Project

46

Three Practical Uses of a Table of Specifications




47

Strengthening the Validity and Reliability of Assessments by Using a “Table of Specifications”

• Does the assessment align to the objectives not only in content but also in the targeted cognitive levels?

• Does the assessment adequately and proportionately cover the objectives?

• Does the assessment account for chance, error, and bias by providing more than one opportunity for the student to demonstrate learning on key objectives?

Construct Validity

Sampling Validity

Repeated Trials Reliability

48

Guiding Question 3

How can teachers analyze results from students’ performance on teacher-developed assessments and demonstrate student progress?

49

Four Examples

Performance Task Paper-Pencil Unit Test with Subsample

Pre-assessment Paper-Pencil Unit Test Cumulative Test

50

Performance Task for Fourth-Grade Art

51

Fourth-Grade Art Unit: Perspective Drawing

The student will:

• Use perspective drawing techniques to create a work of art that depicts a three-dimensional object on a two-dimensional surface. (SOL 4.9)

• Use a variety of lines in the one point perspective drawing. (SOL 4.6)

• Use characteristics of color, including hue, tint, shade, and intensity in the one point perspective drawing. (SOL 4.4)

AP SY

AP

AP

AP: application SY: synthesis

52


Know-ledge

Compre- hension

Applica-tion

Analysis Syn-thesis

Evalua-tion

Perspective drawing techniques to create a work of art that depicts a three-dimensional object on a two-dimensional surface

Use

Create

Use a variety of lines in the one point perspective drawing

Use

Characteristics of color in the one point perspective drawing

Use

Fourth-Grade Art Unit: Perspective Drawing

53

Three Elements of a Performance Task

1. Prompt

2. Response Format

3. Grading Criteria

54

Fourth-Grade Art Pre-Assessment Prompt and Response

Format• Teacher Directions: Have students look around the

classroom from their perspectives and describe what they see. Ask them to pick a point of focus (e.g., the whiteboard, the classroom door) and describe what they see.

• Student Directions: Picking a point of focus, draw a picture showing your perspective on what you see in the classroom. Be sure to include as much detail as possible. Use appropriate lines, points, and color to provide a realistic perspective. Use colored pencils and the 11” X 14” paper provided for your drawing.

55

Grading CriteriaAdvanced (3) Proficient (2) Developing (1) Unacceptable (0)

Element #1: One Point Perspective

Drawing is in complete one-point perspective indicating an excellent level of craftsmanship in drawing and shading the room.

Drawing includes most objects in the room drawn in accurate one point perspective, indicating a high level of craftsmanship in drawing and shading the room.

The majority of the objects in the room are not drawn in one point perspective, indicating a low level of craftsmanship in drawing and shading the room.

Unable to discern objects in room, indicating a low level of craftsmanship.

Element #2: Design Principles

Drawing indicates a complete understanding of line and how it is used to draw objects in one point perspective.

Drawing indicates a mostly accurate understanding of line and how it is used to draw objects in one point perspective.

Drawing indicates an unclear understanding of line and how it is used to draw objects in one point perspective.

Drawing indicates no concept of line and how it is used to draw objects in one point perspective.

Element #3: Use of color

Excellent use of color, used multiple colors and layering to achieve extreme depth.

Good use of color, mixing and layering achieves some depth.

Basic use of color. Layers are thin, used few colors, little depth.

Poor use of color. Did not layer or mix multiple colors, flat.

56


Know-ledge

Compre- hension

Applica-tion

Analysis Synthesis Evalua-tion

Perspective drawing techniques to create a work of art that depicts a three-dimensional object on a two-dimensional surface

Use

Element #1

Create

Element #1Use a variety of lines in the one point perspective drawing

Use

Element #2Characteristics of color in the one point perspective drawing

Use

Element #3

Art Unit: Perspective Drawing

57

Fourth-Grade Art Post-Assessment Prompt and Response

Format

• Teacher Directions: Take students into the library instructing students to sit down at tables. Using the digital cameras provided, have students take a picture of the library from their seats.

• Student Directions: Draw a picture showing your perspective on what you see in the library. Be sure to include as much detail as possible. Use appropriate lines, points, and color to provide a realistic perspective. Use the colored pencils and 11” X 14” paper provided for your drawing.

58

Grading CriteriaAdvanced (3) Proficient (2) Developing (1) Unacceptable (0)

Element #1: One Point Perspective

Drawing is in complete one-point perspective indicating an excellent level of craftsmanship in drawing and shading the room.

Drawing includes most objects in the room drawn in accurate one point perspective, indicating a high level of craftsmanship in drawing and shading the room.

The majority of the objects in the room are not drawn in one point perspective, indicating a low level of craftsmanship in drawing and shading the room.

Unable to discern objects in room, indicating a low level of craftsmanship.

Element #2: Design Principles

Drawing indicates a complete understanding of line and how it is used to draw objects in one point perspective.

Drawing indicates a mostly accurate understanding of line and how it is used to draw objects in one point perspective.

Drawing indicates an unclear understanding of line and how it is used to draw objects in one point perspective.

Drawing indicates no concept of line and how it is used to draw objects in one point perspective.

Element #3: Use of color

Excellent use of color, used multiple colors and layering to achieve extreme depth.

Good use of color, mixing and layering achieves some depth.

Basic use of color. Layers are thin, used few colors, little depth.

Poor use of color. Did not layer or mix multiple colors, flat.

59

Teacher P

Goal Statement

During the course of this school year, all students will make measurable progress in U.S. History II as determined by the multiple choice and essay common assessment. Students will improve as follows:• Students scoring between 0 and 30

percent on the pre-assessment will improve by 55 percentage points on the post-assessment.

• Students scoring 55 percent and higher on the pre-assessment will improve their scores by at least 30 percentage points on the post-assessment.

Three Approaches to Pre- and Post-Assessment

1) A 10-question unit pre-assessment and 40-question unit post-assessment.

2) A 40-question unit pre-assessment and 40-question unit post-assessment.

3) A 40-question cumulative pre-assessment and 40-question cumulative post-assessment.

60

Goal Setting Sample for a Unit Assessment:

10-Question Pre-Assessment 40-Question Post-Assessment

61

U.S. History II.8b-e

The student will demonstrate knowledge of the economic, social, and political transformation of the United States and the world between the end of World War II and the present by:

b) describing the conversion from a wartime to a peacetime economy;

c) identifying the role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges;

d) describing the changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities;

e) describing how international trade and globalization have impacted American life.

62

U.S. History II.1

• Make connections between past and present (USII.1b)

• Analyze and interpret primary and secondary source documents (USII.1a)

• Interpret slogans and documents (USII.1h)

• Identify the costs and benefits of specific choices made in trade and globalization (USII.1i)

• Interpret charts and graphs (school division addition)


Know-ledge

Compre- hension

Appli-cation

Analy-sis

Syn-thesis

Evaluation Percent of Questions Correct

Conversion from a wartime to a peacetime (USII.8b)

Describe

1

Make

connections between past and present

(USII.1b)

2

27.5

Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c)

Identify

8, 9

Analyze and

interpret primary and secondary

source documents

(USII.1a)

10

23.3

Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d)

Describe

3

Interpret

slogans and documents

(USII.1h)

4

20

The impact of international trade and globalization on American life (USII.8e)

Describe

5

Interpret

charts and graphs

6

Identify the costs and

benefits of specific choices made in

trade and globalization (USII.1i)

7

8

Table of Specifications for

Pre-Assessm

ent

With Asse

ssment R

esults

Teacher P: U.S. History II.8b-e Pre-Assessment Data

USII.8b USII.8c USII.8d US118.eTOTAL

CORRECT Percent CorrectNumber of Questions 2 3 2 3 10 100%

Student 1 0 0 0 0 0 0.0%

Student 2 0 0 0 0 0 0.0%

Student 3 0 0 0 0 0 0.0%

Student 4 0 0 1 0 1 10.0%

Student 5 1 0 0 0 1 10.0%

Student 6 1 0 0 0 1 10.0%

Student 7 0 1 0 0 1 10.0%

Student 8 0 1 0 0 1 10.0%

Student 9 1 1 0 0 2 20.0%

Student 10 1 1 0 0 2 20.0%

Student 11 0 1 1 0 2 20.0%

Student 12 1 0 1 0 2 20.0%

Student 13 0 1 1 0 2 20.0%

Student 14 1 0 0 1 2 20.0%

Student 15 1 1 0 0 2 20.0%

Student 16 1 1 0 0 2 20.0%

Student 17 0 1 1 1 3 30.0%

Student 18 1 1 1 0 3 30.0%

Student 19 1 2 1 1 5 50.0%

Student 20 1 2 1 2 6 60.0%

AVERAGE .55 .7 .4 .25 19.9%

64


KN CO AP AN SY EV Percent of Questions

Correct


1, 28 Describe

3, 4, 13, 38

Make connections between past and present (USII.1b)

33, 34, 35

27.5(Pre)

82(Post)

+54.5Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c)

2, 27 Identify

5, 6, 9, 36, 39

Analyze and interpret

primary and secondary source documents

(USII.1a)

10, 11, 12

23.3(Pre)

84(Post)

+60.7


14, 29 Describe

7, 15, 16, 37, 40

Interpret slogans and documents (USII.1h)

8, 24, 25

20(Pre)

80(Post)

+60The impact of international trade and globalization on American life (USII.8e)

17, 18 Describe

19, 20, 26

Interpret charts and

graphs

30, 31, 32

Identify the costs and benefits of specific choices

made in trade and globalization

(USII.1i)

21, 22, 23

8(Pre)

67(Post)

+59

Using the Table of

Specifications to Connect

Student Results to the

Curriculum

65

Teacher P: U.S. History II.8b-e Post-Assessment Data


CORRECTPercentCorrect

Number of Questions 9 10 10 11 40 100%

Student 1 8 6 3 5 22 55.0%

Student 2 6 7 6 8 27 67.5%

Student 3 8 8 5 7 28 70.0%

Student 4 8 7 8 5 28 70.0%

Student 5 8 7 7 6 28 70.0%

Student 6 6 10 7 6 28 70.0%

Student 7 7 8 10 7 32 80.0%

Student 8 7 9 7 6 29 72.5%

Student 9 5 6 6 5 22 55.0%

Student 10 8 9 9 9 35 87.5%

Student 11 6 9 9 8 32 80.0%

Student 12 8 10 10 7 35 87.5%

Student 13 7 8 9 8 32 80.0%

Student 14 8 9 8 8 33 82.5%

Student 15 7 9 7 6 29 72.5%

Student 16 9 10 10 10 39 97.5%

Student 17 8 9 10 7 34 85.0%

Student 18 8 9 9 8 34 85.0%

Student 19 8 8 10 8 34 85.0%

Student 20 9 10 10 11 40 100.0%

AVERAGE 7.45 8.40 8.0 7.25 31.05 77.6%

67

Teacher P: Pre-Assessment and Post-Assessment Data

PRE-ASSESSMENT DATAPercent Correct

POST-ASSESSMENT DATAPercent Correct

TOTAL 100% 100%Student 1 0.0% 55.0%

Student 2 0.0% 67.5%

Student 3 0.0% 70.0%

Student 4 10.0% 70.0%

Student 5 10.0% 70.0%

Student 6 10.0% 70.0%

Student 7 10.0% 80.0%

Student 8 10.0% 72.5%

Student 9 20.0% 55.0%

Student 10 20.0% 87.5%

Student 11 20.0% 80.0%

Student 12 20.0% 87.5%

Student 13 20.0% 80.0%

Student 14 20.0% 82.5%

Student 15 20.0% 72.5%

Student 16 20.0% 97.5%

Student 17 30.0% 85.0%

Student 18 30.0% 85.0%

Student 19 50.0% 85.0%

Student 20 60.0% 100.0%

68

Goal Setting Sample for a Unit Assessment:

40-Question Pre-Assessment 40-Question Post-Assessment

69


Know-ledge

Compre- hension

Appli-cation

Analysis Syn-thesis

Evaluation


Describe



Identify

Analyze and


source documents (USII.1a)


Describe

Interpret slogans and documents

(USII.1h)


Describe

Interpret charts

and graphs


benefits of specific choices made in trade

and globalization (USII.1i)

70


Know-ledge

Compre-hension

Appli-cation

Analysis Syn-thesis

Evalu-ation


1, 28 Describe

3, 4, 13, 38


33, 34, 35


2, 27 Identify

5, 6, 9, 36, 39



(USII.1a)

10, 11, 12


14, 29 Describe

7, 15, 16, 37, 40


8, 24, 25


17, 18 Describe

19, 20, 26

Interpret charts

and graphs30, 31, 32


benefits of specific choices made in trade and

globalization (USII.1i)

21, 22, 23


Pre-Assessment



Correct


1, 28 Describe

3, 4, 13, 38

Make connections between past and present (USII.1b)33, 34, 35

28


2, 27 Identify

5, 6, 9, 36, 39

Analyze and


source documents (USII.1a)

10, 11, 12

18


14, 29 Describe

7, 15, 16, 37, 40

Interpret slogans and documents

(USII.1h)8, 24, 25

22


17, 18 Describe

19, 20, 26


graphs30, 31, 32


benefits of specific choices made in

trade and globalization

(USII.1i)21, 22,

23

27

Using the Table of



Curriculum

Teacher P: U.S. History II.8b-e Pre-Assessment Data


CORRECT Percent CorrectNumber of Questions 9 10 10 11 40 100%

Student 1 0 0 1 1 2 5.0%

Student 2 0 0 2 1 3 7.5%

Student 3 1 0 1 2 4 10.0%

Student 4 2 0 1 2 5 12.5%

Student 5 2 1 1 2 6 15.0%

Student 6 2 1 1 2 6 15.0%

Student 7 2 0 1 3 6 15.0%

Student 8 3 0 1 2 6 15.0%

Student 9 2 2 2 2 8 20.0%

Student 10 2 1 2 5 10 25.0%

Student 11 3 1 3 3 10 25.0%

Student 12 4 2 2 2 10 25.0%

Student 13 3 2 2 3 10 25.0%

Student 14 3 2 2 4 11 27.5%

Student 15 2 3 2 4 11 27.5%

Student 16 3 2 3 3 11 27.5%

Student 17 3 2 3 4 12 30.0%

Student 18 4 1 3 4 12 30.0%

Student 19 4 8 4 6 22 55.0%

Student 20 6 8 7 5 26 65.0%

AVERAGE 2.55 1.8 2.2 3.00 9.55 23.9%

72

73


Know-ledge

Compre-hension

Appli-cation

Analysis Syn-thesis

Evalu-ation


1, 28 Describe

3, 4, 13, 38


33, 34, 35


2, 27 Identify

5, 6, 9, 36, 39



(USII.1a)

10, 11, 12


14, 29 Describe

7, 15, 16, 37, 40


8, 24, 25


17, 18 Describe

19, 20, 26


graphs

30, 31, 32


benefits of specific choices made in trade and

globalization (USII.1i)

21, 22, 23


Post-Assessment(Different Questions)



Correct


1, 28 Describe

3, 4, 13, 38


33, 34, 35

28(Pre)

82(Post)

+54


2, 27 Identify

5, 6, 9, 36, 39



(USII.1a)

10, 11, 12

18(Pre)

84(Post)

+66


14, 29 Describe

7, 15, 16, 37, 40


8, 24, 25

22(Pre)

80(Post)

+62


17, 18 Describe

19, 20, 26


graphs

30, 31, 32


benefits of specific choices made in trade

and globalization

(USII.1i)

21, 22, 23

27(Pre)

67(Post)

+40

Using the Table of



Curriculum

75

Teacher P: U.S. History II.8b-e Post-Assessment Data


CORRECTPercentCorrect

Number of Questions 9 10 10 11 40 100%

Student 1 8 6 3 5 22 55.0%

Student 2 6 7 6 8 27 67.5%

Student 3 8 8 5 7 28 70.0%

Student 4 8 7 8 5 28 70.0%

Student 5 8 7 7 6 28 70.0%

Student 6 6 10 7 6 28 70.0%

Student 7 7 8 10 7 32 80.0%

Student 8 7 9 7 6 29 72.5%

Student 9 5 6 6 5 22 55.0%

Student 10 8 9 9 9 35 87.5%

Student 11 6 9 9 8 32 80.0%

Student 12 8 10 10 7 35 87.5%

Student 13 7 8 9 8 32 80.0%

Student 14 8 9 8 8 33 82.5%

Student 15 7 9 7 6 29 72.5%

Student 16 9 10 10 10 39 97.5%

Student 17 8 9 10 7 34 85.0%

Student 18 8 9 9 8 34 85.0%

Student 19 8 8 10 8 34 85.0%

Student 20 9 10 10 11 40 100.0%

AVERAGE 7.45 8.40 8.0 7.25 31.05 77.6%

76

Teacher P: Pre-Assessment and Post-Assessment Data



TOTAL 100% 100%Student 1 5.0% 55.0%

Student 2 7.5% 67.5%

Student 3 10.0% 70.0%

Student 4 12.5% 70.0%

Student 5 15.0% 70.0%

Student 6 15.0% 70.0%

Student 7 15.0% 80.0%

Student 8 15.0% 72.5%

Student 9 20.0% 55.0%

Student 10 25.0% 87.5%

Student 11 25.0% 80.0%

Student 12 25.0% 87.5%

Student 13 25.0% 80.0%

Student 14 27.5% 82.5%

Student 15 27.5% 72.5%

Student 16 27.5% 97.5%

Student 17 30.0% 85.0%Student 18 30.0% 85.0%

Student 19 55.0% 85.0%

Student 20 65.0% 100.0%

77

Goal Setting Sample for a Cumulative Assessment:

40-Question Cumulative Pre-Assessment

40-Question Cumulative Post-Assessment

78

U.S. History II Test Blueprint

79


KN CO AP AN SY EV

Reconstruction to Modern America [7] 1, 2

10, 11, 19

Historical Analysis Skill Questions (U.S. II.1)

20, 21Turmoil and Change [10] 15

3, 4, 7, 23, 29

Historical Analysis Skill

Questions (U.S. II.1)

5, 6, 22

United States through World War II [9] 35, 36

14, 30, 31, 37, 38



33, 34

Geography [7] 9 8, 24, 25,

32



12, 13

Civics and Economics [7] 26, 27 16, 28, 40



17, 18, 39


Pre-Assessment

80



Correct


10, 11, 19


20, 21

35.0

Turmoil and Change [10] 15 3, 4, 7, 23,

29



5, 6, 22

13.5


14, 30, 31, 37, 38



33, 34

19.4

Geography [7] 9 8, 24, 25,

32



12, 13

40.0




17, 18, 39

17.1

Using the Table of



Curriculum

Teacher P: U.S. History IIPre-Assessment Data

Reconstruction to Modern America

Turmoil and Change

United States Since WW II Geography

Civics and Economics

TOTALCORRECT

Percent Correct

Number of Questions 7 10 9 7 7 40 100%Student 1 0 0 1 0 1 2 5.0%

Student 2 0 0 2 1 0 3 7.5%

Student 3 1 0 1 2 0 4 10.0%

Student 4 2 0 1 1 1 5 12.5%

Student 5 2 1 1 2 0 6 15.0%

Student 6 2 1 1 2 0 6 15.0%

Student 7 2 0 1 2 1 6 15.0%

Student 8 3 0 1 2 0 6 15.0%

Student 9 2 2 0 2 2 8 20.0%

Student 10 2 1 2 4 1 10 25.0%

Student 11 3 1 1 3 2 10 25.0%

Student 12 4 2 2 2 0 10 25.0%

Student 13 3 0 2 3 2 10 25.0%

Student 14 3 2 1 4 1 11 27.5%

Student 15 2 1 2 4 2 11 27.5%

Student 16 3 2 2 3 1 11 27.5%

Student 17 3 2 2 4 1 12 30.0%

Student 18 4 1 1 4 2 12 30.0%

Student 19 4 5 4 6 3 22 55.0%

Student 20 4 6 7 5 4 26 65.0%

AVERAGE 2.45 1.35 1.75 2.8 1.2 9.55 23.9%

PERCENTAGE OF QUESTIONS ANSWERED

CORRECTLY IN REPORTING CATEGORY 35.0% 13.5% 19.4% 40.0% 17.1% 23.9%

82


KN CO AP AN SY EV


10, 11, 19


20, 21Turmoil and Change [10] 15

3, 4, 7, 23, 29



5, 6, 22


14, 30, 31, 37, 38



33, 34

Geography [7] 9 8, 24, 25,

32



12, 13




17, 18, 39


Post-Assessment(Different Questions)



Correct

Reconstruction to Modern America 7

1, 2 10, 11, 19



20, 21

35 (Pre)

82.9 (Post)

+47.9

Turmoil and Change 10 15 3, 4, 7, 23,

29



5, 6, 22

13.5 (Pre)

71.5 (Post)

+58

United States through World War II 9

35, 36 14, 30, 31,

37, 38



33, 34

19.4 (Pre)

73.3 (Post)

+53.9

Geography 7 9 8, 24, 25,

32



12, 13

40 (Pre)

85 (Post)

+45

Civics and Economics 7 26, 27 16, 28, 40



17, 18, 39

17.1 (Pre)

79.3 (Post)

+62.2

Using the Table of



Curriculum

Teacher P: U.S. History IIPost-Assessment Data

Reconstruction to Modern America

Turmoil and Change

United States Since WWII Geography

Civics and Economics

TOTAL CORRECT

PercentCorrect

Number of Questions 7 10 9 7 7 40 100%Student 1 5 3 3 5 6 22 55.0%Student 2 6 4 6 5 6 27 67.5%Student 3 6 8 5 4 5 28 70.0%Student 4 5 7 6 5 5 28 70.0%Student 5 5 7 4 6 6 28 70.0%Student 6 6 7 4 6 5 28 70.0%Student 7 6 6 8 6 6 32 80.0%Student 8 5 7 6 5 6 29 72.5%Student 9 4 5 5 4 4 22 55.0%

Student 10 6 8 8 7 6 35 87.5%Student 11 4 7 7 7 7 32 80.0%Student 12 7 8 8 7 5 35 87.5%Student 13 6 7 8 7 4 32 80.0%Student 14 6 8 6 7 6 33 82.5%Student 15 6 8 6 5 4 29 72.5%Student 16 6 10 9 7 7 39 97.5%Student 17 7 8 8 6 5 34 85.0%Student 18 6 8 8 6 6 34 85.0%Student 19 7 7 8 7 5 34 85.0%Student 20 7 10 9 7 7 40 100.0%AVERAGE 5.8 7.15 6.6 5.95 5.55 31.05 77.6%

PERCENTAGE OF QUESTIONS ANSWERED CORRECTLY IN

REPORTING CATEGORY 82.9% 71.5% 73.3% 85.0% 79.3% 77.6%

85

Teacher P: U.S. History II Pre-Assessment and Post-Assessment Data



TOTAL 100% 100%Student 1 5.0% 55.0%

Student 2 7.5% 67.5%

Student 3 10.0% 70.0%

Student 4 12.5% 70.0%

Student 5 15.0% 70.0%

Student 6 15.0% 70.0%

Student 7 15.0% 80.0%

Student 8 15.0% 72.5%

Student 9 20.0% 55.0%

Student 10 25.0% 87.5%

Student 11 25.0% 80.0%

Student 12 25.0% 87.5%

Student 13 25.0% 80.0%

Student 14 27.5% 82.5%

Student 15 27.5% 72.5%

Student 16 27.5% 97.5%

Student 17 30.0% 85.0%

Student 18 30.0% 85.0%

Student 19 55.0% 85.0%

Student 20 65.0% 100.0%

86

Using a Table of Specifications




4. To analyze student results.

87

Guiding Question 4

How can a teacher and/or principal determine what currently available assessments may be appropriate for establishing baseline data for student achievement goal setting?

88

Desired Characteristics of Teacher-Developed Assessments Used to Demonstrate

Student Growth

Reliable

Practical to Employ

Instructionally Useful

Valid

89

Uses of Tables of Specifications To…

• Create teacher-developed assessments, especially “common assessments”

• Critique and improve existing assessments, including commercially produced assessments and assessments from test generators

Construct Validity

Sampling Validity

Repeated Trials

Reliability

90

Choose Assessment Formats Appropriate to the Content and Skills of the Objectives

SkillsPerformance Task Performance Task Performance Task

Content and Skills

Oral Q & AComputer-based

TestPerformance Task

Paper-Pencil Test

Computer-based Test

Performance Task

Paper-Pencil Test

Computer-based Test

Performance Task

ContentOral Q & A

Computer-based Test

Paper-Pencil Test

Computer-based Test

Paper-Pencil Test

Computer-based Test

Primary Upper Elementary Secondary

Curr

icul

um C

onsi

dera

tions

Student Developmental Considerations

Considerations for Selecting a Type of Pre-Assessment

Type of Pre-Assessment• Sub-Sample or Key Concept

Pre-Assessment

• Cumulative or Comprehensive Pre-Assessment

Consideration• Student motivation• Time away from instruction• Instructional

appropriateness• Construct validity• Sampling validity• Repeated trials reliability• Test-Retest effect

91

92
