Using empirical feedback to develop a learning progression in

science

Karen DraneyUniversity of California, Berkeley

2

The components of the BEAR Assessment System

Measurement model provides empirical feedback at every step

• Theoretical structure of learning progression• Quality of items• Match between item scoring process and

learning progression• Degree to which assessment is an accurate

reflection of student proficiency

Carbon Cycle as example -- two of the original constructs

Wright map

(Note: “X’ = 1 respondent.)

Original scoring with 5 levels

6

Quite difficult to reach second level

Relatively easy to reach second level

Latest structure:Learning progression for Carbon

Item thresholds

CHARACTERISTICS OF RESPONSES

CRD: “A carbon atom from Grandma Johnson's remains sink into the ground and mixes with the soil. Then when the soil is mixed and churned, it rises to the top of the ground. When the coyote kills something upon that dirt, he may consume it and have some of the soil come with it, which produces him with Grandma Johnson's carbon atom.”

MCK: “The carbon in grandma’s body is decomposed into the ground. The plants then use the fertile soil to use her carbon atoms. As the soil passes it to the plant, the plant is eventually eaten by the coyote. The carbon atom then travels to its leg.”

DES: “Grandma Johnson's remains decay and decomposers use respiration and turn it to carbon dioxide. The plants absorb the carbon dioxide. Rodents eat the plants and

then the coyote eats the rodent.”

Written Response ItemsGrandma Johnson: Describe the path of a carbon atom from Grandma Johnson’s remains, to inside the leg muscle of a coyote. NOTE: The coyote does not dig up and consume any part of Grandma Johnson’s remains.

Tracing Matter: Material Kinds, Properties of Matter

5Correctly identifies reactants and products of individual chemical processes: Substances/molecules and the elements/atoms of which they are composed.

4Correctly identifies some reactants and products of single chemical process, and solids, liquids, and gases involved.

3Correctly identifies some reactants and products of single chemical process, and solids, liquids, but not gases involved.

2Attention to hidden mechanism, but cannot identify any material kinds

1Identify changes by common sense of natural phenomena, but not as changes in materials and cannot identify material kinds.

Other item types4. When a person loses weight, where does the matter of the person’s fat go? Pleasechoose ONE answer that you think is best.

a. The fat is broken down and leaves the person’s body as water and gas. (Level 4)b. The fat is converted into energy. (Level 2)c. The fat is burned up providing energy for the person’s body functions. (Level 2)d. The fat is broken down and leaves the person’s body as feces and urine. (Level

1)

Item fit - how well are items working

QuickTime™ and aGraphics decompressor

are needed to see this picture.

More random noise than expected in this item

xxx

xxxxxx

xxxxxxxxxxx

xxxxxxxxxxx

xxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxx

xxxxxx

x

4

3

2

1

4

3

2

1

4

3

21

Item difficulty estimatesStudent performance estimates

4

3

21

Q1 Q2 Q3 Q4

x = one student

2.0

1.0

0

-1.0

-2.0

one scale

Match between scoring and learning progression

Person fit - how well does the assessment work for individuals