Standards, Assessments, and Accountability National Academy of Education White Paper

University of Colorado at Boulder / School of Education

Standards, Assessments, and Accountability

National Academy of Education White Paper

Lorrie A. ShepardUniversity of Colorado at Boulder

Performance Assessment BriefingWashington, DC

April 13, 2010

University of Colorado at Boulder / School of EducationUniversity of Colorado at Boulder/ School of Education

Standards, Assessments, and Accountability Shepard, Hannaway, & Baker (2009), http://www.naeducation.org/

National Academy of Education Working Groupon Standards, Assessments, and Accountability:

Eva Baker (Co-Chair) P. David Pearson

Jane Hannaway (Co-Chair) Diane Ravitch

Patricia Gandara William Schmidt

Drew Gitomer Alan Schoenfeld

Margaret Goertz David Stern

Helen Ladd William Trent

Robert Linn Mark Wilson

University of Colorado at Boulder / School of EducationUniversity of Colorado at Boulder / School of Education

Main Points

History of Standards-Based Reform and Its Theory of Action

Research on Teaching-the-Test Effects

Importance of Curricular Coherence

Significance of Performance Assessments for Recommendations 1 and 4

Mention of Recommendations 2 and 3


RECOMMENDATION 2: The federal government should support research on accountability system indicators……reflect both status and growth…set performance standards that are ambitious but realistic…communicate to the public …and students what is to be learned…report assessment results along the entire achievement continuum (not just at the cutpoint)

RECOMMENDATION 3: The federal government should support the redesign and ongoing evaluation of accountability systems……so that they contribute to school improvement…use test results to trigger closer investigations…verify results before sanctions or rewards…consider audit assessments, expert inspectorates, etc.


Testing, Teaching, & Learning Elmore & Rothman (1999), NRC

1994 Goals 2000: vision of coherent capacity building1994 ESEA set forth an incentives theory of change – assumed that with sufficient motivation, teachers and other school personnel would find the means to improve instruction.

Unfortunately, implementation studies then and now show that many schools do not understand the changes that were needed and lack capacity to make them happen.


20 years of standards have produced gains, especially in mathematics, but gains on NAEP are modest compared to state tests and have not accelerated under intense NCLB pressure.

Negative Effects of Teaching the TestCurriculum distortion

Test-score inflation Many policymakers believe that curriculum narrowing is a necessary trade off because reading and math skills are so essential.

But, a close look at research on test-score inflation shows that students cannot apply skills that they appear to know on the test, and

Excessive drill on worksheets resembling the test denies students opportunity to understand context and purpose that would enhance skill development.

University of Colorado at Boulder / School of EducationUniversity of Colorado at Boulder/CRESST

1. A series circuit consists of three identical light bulbs connected to a battery as shown here. When the switch S is closed, do the following increase, decrease, or stay the same?

(a) The intensities of bulbs A and B (b) The intensity of bulb C (c) The current drawn from the battery (d) The voltage drop across each bulb (e) The power dissipated in the circuit

2. For the circuit shown, calculate (a) the current in the 2-Ω resistor and (b) the potential difference between points P and Q.

Mazur (1997)


The Importance of CurriculumThe NAEd Standards paper argues for the concurrent, coherent design of curriculum, assessments (both large-scale and classroom level), and teacher professional development.

Top performing countries in TIMSS have leaner, more hierarchically sequenced curricula leading to progressively more advanced topics and deeper understanding. (Schmidt et al., 2005)

Recent analysis of content standards in 14 states found they did not

focus on big ideas or build from grade to grade. (Porter et al., 2009). Common Standards are a help but are still only a skeletal framework.

National control is not required for coherence (Schmidt & Prawat,

2006). Rather, coherence leads to effective outcomes if it is achieved at whatever level of governance has authority over policy instruments.


RECOMMENDATION 1: The federal government should encourage the redesign and clear connection of content and performance standards – and the curricula, teacher training, and high-quality assessments to go with them – with the goal of clearly articulating expected progressions of learning.

Instead of turning Standards over to test makers for implementation, the NRC report on state science assessments calls for:

Horizontal coherence (linking curriculum, instruction, and assessment)

Vertical coherence (shared vision at classroom, school, district, and state levels)

Developmental coherence (taking account of how understanding develops over time)

To achieve coherence, state consortia might consider concurrent development of standards and assessments, or at least prototypes of assessment tasks, anchored to learning progressions.


Learning Progressions Knowing What Students Know (2001)

Learning progressions or Progress Maps provide an underlying model of learning to coherently link classroom and large-scale assessments.

“Progress maps describe skills, understandings, and knowledge in the sequence in which they typically develop: a picture of what it means to ‘improve’ in an area of learning.” (Masters & Forster, 1996).

Learning progressions provide a substantive rather than a quantitative picture of developing competence.

A criterion-referenced growth model.


Learning progressions are an effective tool for teacher professional development.

• How learning typically unfolds helps teachers know “what next” and how to “back-up” (though we must also be aware of natural variations and departures from the typical pattern).

• When embedded in units of instruction LPs can target typical misconceptions.

Substantive and statistical growth trajectories are not the same, but with care they could be brought together.

While much research is still needed, it is possible to start with judgmentally set sequences and then improve empirically.

ooo

oooooo

oooooooo

ooooooooooooo


++

++

-

--

- -

+

-

+- +

An example of a typical misconception and challenge question:

What happens if you cut the nail in half?

Otero, Jalovec, & HerManyHorses. (2008).


RECOMMENDATION 4: The federal government should support an intensive program of research and development to create the next generation of performance assessments explicitly linked to well-designed content standards and curricula.

Rather than alignment, embodiment might be a better term to characterize the more complete and substantive alignment that occurs when the tasks, problems, and projects in which students are engaged represent the range and depth of what we want students to understand and be able to do.

Assessment tasks convey what’s important to learn as well as providing an opportunity to check on students’ understanding and evaluate achievement.

New technologies make it possible to tap complex and dynamic aspects of cognition and performance.

When conversations about student learning are engaged around rich tasks, there is greater opportunity for instructional insights.


To truly transform learning opportunities in classrooms, in ways

consistent with what we know is possible from research,

we will need to remove policy structures especially low-level tests

that misdirect effort, provide coherent curricula consistent with ambitious

reforms, and take seriously the need for

capacity building at every level of the educational system.

Documents

Standards, Assessments, and Accountability National Academy of Education White Paper