Knowing What Students Know:The Science and Design of Educational Assessment

Committee on the Foundations of Assessment

Board on Testing and Assessment, Center for Education

National Research Council

Implications of New Foundations for Assessment Design

Recursive process Begin with model of cognition and learning in the domain

Examples: development of number sense (Case & Griffin); subtraction bugs (Brown & Burton); ACT -R theory (Anderson et al.); Facets in physics (Minstrell & Hunt); Middle School Math through Applications (Greeno et al.); CRESST models; Australia’s progress maps (ACER)

Design of Observations

Tasks (and scoring) guided by cognition and interpretation models Examples:

Number Knowledge Test (Case & Griffin) CAETI simulation & expert scoring system

Sometimes sets of tasks useful for diagnosis Example:

Diagnoser (Minstrell)

Validation

Should include evidence that tasks actually tap the intended knowledge and cognitive processes methods: e.g., think alouds, analyses of errors Examples:

cognitive complexity of science performance tasks (Baxter & Glaser);

QUASAR (Lane & Silver) Standards for Educational and Psychological Tests.

(1999)

Reporting

Assessments must be designed from the start to ensure that reporting of desired types of information will be possible

Results should be reported relative to model of learning Example: Australia’s progress maps (ACER;

Queensland)

A Way of Thinking About Fairness

Methods of conditional inference (ways of taking into account info about students’ instructional histories)

1. Control instruction and/or task to ensure task-examinee match up

2. Obtain background info about student’s instruction

3. Allow student choice of tasks in light of what they know about themselves

4. Minimize irrelevant variance

Implications of New Foundations for Assessment Practice

In the classroom: assessment should be integral part of instruction students should get information about particular

qualities of their work and what they can do to improve

students understand learning goals and landmark performances along the way

cognitive science findings need to be made teacher-friendly

Assessment Practice

In large-scale testing contexts: proposed approach holds promise for drawing more

valid and fair inferences about student achievement but to fully capitalize on advances, need to relax

constraints that drive current practices feasibility & scalability in this policy context should

be demonstrated some alternatives: population sampling (e.g., Maryland),

curriculum-embedded (e.g., AP Art)

Assessment Practice Committee envisions systems of assessments

that are: comprehensive coherent continuous

Shift emphasis toward classroom where learning occurs and where information support is most needed