State Reform Efforts: Multiple Measures of Accountability & Improvement

Institute for Governors’ Education Advisors

NGA Association for Best PracticesAmelia Island, Florida November 3-5, 2001

Mari Pearlman

2

The current landscape: dangers and opportunities

Dangers: Single focus on annual test scores Lack of useful data to interpret these scores Little connection between testing and instruction Search for the guilty rather than search for

success Opportunities

Systemic improvement for all parts of the education system

Useful data collection and interpretation to make improvement an empirical rather than rhetorical and emotional issue

3

Protection against the dangers: first steps

Align the entire accountability system, with particularly careful connections between what is taught and what is tested

Establish direct feedback loops between test score information and teacher instructional planning

Use multiple different measures of student achievement, including but not limited to standardized tests

Employ sensible data collection and analysis plans

4

Aligning the accountability system - 1

Criteria or standards for learning A clear definition of what students will be

able to do to demonstrate that learning—the specifications for assessments, both ongoing (classroom-based) and summative (school, district, state-based)

A coherent continuum for applying those criteria by developmental level of students

5

Aligning the accountability system - 2

A curriculum that supports the criteria for learning and includes demonstrations of learning that are aligned with the assessments

Instruction that implements the curriculum, with continuing teacher professional development

A plan for collecting data that defines what questions the data should answer and then specifies the data to be collected

A plan for using those data to inform curriculum, instruction, teacher professional development, and assessment designs

6

Scope and sequence: Content, performance & assessment

The sequence of alignment matters: standards for what students must know (and

teachers must teach)—content standards Standards for how much of the content

students must know – performance standards A definition of what learning looks like—what

should students—at each level--be able to do if they meet the performance standards—assessment specifications

7

Scope and sequence—Curriculum & Instruction

A careful examination and articulation of what should be taught, when it should be taught, and how the knowledge and skills articulated are connected to the final expectations for student learning—the curriculum

Classroom instruction that really focuses on the curriculum—helping teachers make good choices about what, of all possible things, should be taught—teacher mentoring and evaluation and professional development

8

Scope & Sequence:Data collection and interpretation

What questions do you want to be able to answer about student learning and the effectiveness of the education system?

To answer those questions, what data would you need to collect?

How can you use these collected data to close the loop and improve the criteria for student learning, the curriculum, the instruction, and teacher professional development?

9

Realistic Expectations Think realistically about milestones

for student learning and their connections with annual testing What is the gain you expect to see from

year to year? Given a particular range of

performance in year 1 (say 1st quartile on the test, or lowest 25%), what is reasonable to expect as a growth goal in the next year?

10

Check those expectations against

What the curriculum says teachers should teach

How much actual instructional time is devoted to these areas of learning at each level

What kind of assessment data you actually have that would be evidence of learning against these expectations

11

And against Whether assessment data are ever

available in time for teachers to change the way they teach particular students

Whether teachers are given any structured guidance about the use of assessment data, changes in instruction to serve student learning, or aligning their practice with ultimate goals for student learning

12

Using multiple measures of educational effectiveness: outputs

Output measures Multiple formal assessments: this is not impossible.

All teachers in a school or district can come up with easy-to-use performance measures that could supplement norm-referenced standardized results.

Informal assessments: teacher standardized assessment at the beginning and end of each year—create a standard form and keep those data.

Grades and teacher evaluations of student performance

Student performance itself—the science fair, the music program, the art exhibit, the auto shop, the contests and awards from disciplinary organizations

13

Multiple measures 2: inputs Teacher profiles-

Number with full certification Number with content majors and advanced degrees Years of experience

School leadership profiles— Teaching experience Training and experience as a school leader

Demographic variables that matter % of students who come and go within a teaching

year % of drop outs Absenteeism

14

Realistic Expectations Recognize that this is an ongoing, continuous

process—this is about human growth from birth to age 21

Student learning—in particular, not enough of it—is not like an infection that can be cured with a massive dose of antibiotics

Everyone must be accountable—in particular, it is not the fault of the students who are struggling. Improving student achievement is not just the job of teachers. Parents, principals, superintendents, legislators, the public all have a role and responsibility