6
Changing the National Conversation on Assessment Author(s): Phyllis Tashlik Source: The Phi Delta Kappan, Vol. 91, No. 6 (Mar., 2010), pp. 55-59 Published by: Phi Delta Kappa International Stable URL: http://www.jstor.org/stable/27755670 . Accessed: 25/06/2014 02:58 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. . Phi Delta Kappa International is collaborating with JSTOR to digitize, preserve and extend access to The Phi Delta Kappan. http://www.jstor.org This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AM All use subject to JSTOR Terms and Conditions

Changing the National Conversation on Assessment

Embed Size (px)

Citation preview

Page 1: Changing the National Conversation on Assessment

Changing the National Conversation on AssessmentAuthor(s): Phyllis TashlikSource: The Phi Delta Kappan, Vol. 91, No. 6 (Mar., 2010), pp. 55-59Published by: Phi Delta Kappa InternationalStable URL: http://www.jstor.org/stable/27755670 .

Accessed: 25/06/2014 02:58

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Phi Delta Kappa International is collaborating with JSTOR to digitize, preserve and extend access to The PhiDelta Kappan.

http://www.jstor.org

This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions

Page 2: Changing the National Conversation on Assessment

Changing the National

Conversation on Assessment

A consortium of New York public high schools serves as a model for a multi-dimensional system that is

performance-based.

By Phyllis Tashlik

"The sharp separation often seen in the literature between qualitative and quantitative methods is a spurious one."

? Bent Flyvbjerg

"The conventions of standardized testing have become so widely accepted that many evaluators cannot think of assessment based on project-based individualized educa tion."

? Robert E. Stake, director, Center for Instructional Research and

Curriculum Evaluation, University of Illinois, Urbana-Champaign

Given the narrowed focus of the current conversation about

assessment, it's hard

to conceive how it could possibly be moved to change. From Secretary of

Education Arne Duncan to state com

missioners, chancellors, mayors, and the press, the language of quantitative

measures has dominated the "conversa

tion." "Assessment" has come to mean

really one thing only: the numerical re sults of standardized tests.

Social scientists have long cited the critical shortcomings of quantitative in dicators. As Donald T. Campbell wrote, in what has now become a truism for those involved with public policy, "The

more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor" (1976).

Despite Campbell's warning and the varieties of corrupting influences that have been documented across the states, more and more school systems rely on an excessive series of standardized exams to determine an ever-widening range of education decisions: promotion and

graduation requirements for students, bonuses for principals, tenure and salaries for teachers, ratings for schools, and school closings. We run the risk of

"valuing what we measure [rather than] measuring what we value" (Biesta

PHYLLIS TASHLIK is director of the Center for Inquiry in Teaching and Learning, New York Per

formance Standards Consortium.

ixikmtLrMxi V91 N6 Kappan 55

This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions

Page 3: Changing the National Conversation on Assessment

2009). The possibilities for other options ?

performance-based or

qualitative or

project-based, whatever they may be called ? have become severely restricted if not totally eliminated.

We run the risk of

"valuing what we

measure [rather than]

measuring what we

value,"

But one group of New York public high schools has managed to defy the odds. The staffs of these schools have sur

vived the era of "one size fits all" by cre

ating and sustaining an entire perform ance-based assessment system. Perhaps the conversation can still be changed.

PERFORMANCE-ASSESSMENT SCHOOLS

A group of 30 public schools in New

York, working together as the New York Performance Standards Consortium since 1998, have been successfully grad uating students using a performance based assessment option (in addition to the New York State Regents exam in

English Language Arts). They have been at the forefront of changing the conversation about assessment and re

defining "data" and how best to use it to

engage both students and teachers in the

complex tasks of teaching and learning. Researchers have shown that con

sortium graduates succeed in college, achieving a higher GPA than the na

tional norm, placing into credit-bear

ing courses (thus avoiding the expen sive but uncredited remedial classes), and outpacing national rates for return

ing to college as second-year students. This achievement is all the more im

pressive when considering that they en

tered high school with lower scores on

English and math state exams, a higher percentage of students in special educa

tion, and higher rates of poverty when

compared to the overall New York City school population (Foote 2007).

Consortium schools include urban

public high schools in New York City, Rochester, and Ithaca. Their assessment

system meets the New York State Standards for Learning. Instead of exit

exams, student assessments are based

on specific performance-based assess ment tasks (PBATs) that grow out of the schools' curricula. With students

expected to demonstrate college-level skills, the curriculum must be challeng ing as well as engaging. Too often, in

systems governed solely by standard ized exit exams, the curriculum becomes

overwhelmingly test-prep, a series of interim assessments and assignments that mirror the standardized exam.

The consortium's PBATs include an

analytic, comparative literary essay on two works of literature; an intensive so

cial studies research paper; a hands-on,

original science experiment that is well documented in science reports; and ap plications of higher-order mathematics to problem solving. Each school then

stipulates additional graduation tasks. For example, schools have added per formance tasks in the creative arts, arts

criticism, foreign language, and intern

ships. All tasks are the summative indi cators of years spent in classrooms where standards for teaching inquiry, higher-order thinking skills, analysis, and synthesis are daily goals. Classroom teachers and external assessors evaluate

student papers using the consortium

rubrics, which consortium teachers re

vise every two years or as needed.

In addition to the above tasks, con sortium schools require an oral compo nent ?

such as a presentation, exhibi

tion, debate, or defense ? in each of the

disciplines. Like the written work, these oral components also involve external assessors. A number of different options are available for the oral presentations: Some take place as panels, some are one-on-one interviews with external as

sessors, some are exhibitions to a group of assessors. Schools have developed a

pool of subject-area experts, writers,

journalists, and lawyers who volunteer to participate; parents are also involved in the process. Participants receive the

Student papers in advance to prepare for the presentation. (The consortium is

producing a DVD that will focus on the oral component; it will be part of the Teacher to Teacher series available from Teachers College Press.)

The system itself is reviewed by the Performance Assessment Review Board, an affiliation of nationally respected ed ucators who visit the schools and pre pare reports based on what they've ob served and the documents they've read.

DEMONSTRATE RELIABILITY

As a further check on the objectivity and reliability of the system, consor

tium schools participate in an ongoing moderation study,

a process for evalu

ating the consistency of the assessment

system. Hundreds of teachers across

the consortium schools are involved in

reading and assessing each other's stu

dent work and conferencing on the standards that teachers expect students to meet, regardless of their skill level

when they enter the schools. What is most impressive in this system is the maintenance of high standards for all

students, including students in "trans fer schools" (that is, second-chance

schools), students for whom English is their second language, and students in

special education. The four phases of the moderation

study look like this:

Phase 1:150 teacher representatives read submitted student work and assess

it using consortium rubrics. Phase II: Teachers at each school

read and evaluate a selection from the same collection of papers. Schools are

later informed of the consensus

reached during the two phases and whether the school's own assessment

matches the assessment reached by the

larger group of consortium teachers. Phase III: Teachers focus on out

standing student work and deepen their

understanding about the curriculum and assignments most likely to produce such work.

Phase IV: Educators external to the

system go through the same process as

consortium teachers to provide a check on their work.

56 Kappan March 2010

This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions

Page 4: Changing the National Conversation on Assessment

Each fall, teachers participate in fol

low-up sessions, both for assessment

purposes and as a productive and stim

ulating professional development expe rience.

Following the first two years of moderation work, the consortium pub lished booklets of exemplar student pa pers in literature, social studies, math, and science. The exemplars

now pro

vide an added level of consistency and

reliability to the system. The data gathered by teachers

through the students' written and oral

presentations constitute just a part of the important information used to as sess students and their accomplish

ments. As a fully realized performance assessment system, other indicators are

also taken into consideration when ad

justing instruction, developing curricu

lum, and reaching decisions about pro motion and graduation. Instead of rely ing solely on the results of a standard ized exit exam ? an incomplete and not

always reliable quantitative representa tion of what students know and can do

? the performance assessment system

requires and is capable of producing more and varied comprehensive data, which lead to a better understanding of students and their accomplishments, as

well as their needs.

A MORE COMPLEX UNDERSTANDING OF DATA

In New York and elsewhere, the small school movement encouraged building strong communities within small, inti

mate environments. Small public schools were founded on lofty principles, such as educating the whole child, designing student-centered innovative curricula,

building professional communities, and

providing individualized interventions for students. As the creed of quantita tive data monopolized local, state, and national conversations, many small

schools found themselves struggling to

maintain their identity as communities with a mission, but the performance as sessment schools have been steadfast in their founding principles. When com

pared with the restrictions of schools

geared toward standardized testing, they are the institutions most in touch

with meeting the academic, social, and

psychological needs of their students, the burgeoning requirements for achieve

ment in the 21st century, and the "skills. . . crucial to students' success as

college freshmen, professionals, and

participants in a democracy" (Schmoker 2008-09).

John W. Creswell, co-director of the Office of Qualitative and Mixed

Methods Research at the University of

Nebraska-Lincoln, describes qualitative inquiry as "a complex, holistic picture," one that requires us to consider the "mul

tiple dimensions" of the issues and an

understanding of all its complexity (1998). The scores of standardized tests

do not meet that requirement; nor do

they provide an adequate measure for

helping students succeed in the long term. In fact, too often, quantitative

meas

ures present only a negative picture of our students so that they, and others, become convinced that they can't suc

ceed, a phenomena all too common in the urban public schools that poor and

minority students attend. As one prin cipal recently remarked, "About the

only thing we learn from all the data go ing into the new reporting system is at tendance patterns

? which day of the week kids seem to miss school most.

Otherwise, teachers already know the information and have informed parents in a far more detailed and contextual ized way than the data offer."

The success of schools engaged in

performance assessment has shown that it is the tests that have failed our stu dents and not vice versa. To demon

strate the complexity of the issue and the "data" needed for making substan tive decisions about students, at right and on the next page are lists of the in dicators and obstacles ? both academic and nonacademic

? that consortium

teachers consider when assessing their students and developing their schools'

program. Consideration of these phe nomena has translated into a more nu

anced approach to curriculum develop ment, classroom instruction, making de

cisions about students' futures, interim and summative assessments, and coun

seling. The result is quite different from the narrow understanding of "data" and

ACADEMIC INDICATORS

Classroom artifacts that help teachers learn about their students

Student work collected over time: evidence of growth over

time, not limited to a two-hour

exam;

Examples of thoughtful revisions of research and writing;

Performance on teacher

constructed exams and

homework, which grow out of classroom instruction and

course content;

Performance as evaluated by teacher-developed and -revised

rubrics;

Completion of course

requirements, which involve

reading complex texts and

writing extensively;

Classroom discussions and level of participation;

Questions asked and comments made during class;

Student participation in out-of

classroom interviews and field

trips, and interactions with

experts invited to make

presentations;

Student self-assessments and

their analyses of their

assessments; and

Student self-awareness as a

learner, reader, and writer.

V91N6 Kappan 57

This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions

Page 5: Changing the National Conversation on Assessment

assessment prevalent in rigid test-bound

systems.

Through nonacademic indicators, teachers learn about personality traits,

NONACADEMIC INDICATORS

Qualitative data that help teachers better understand and teach their students.

Student participation in tutorials;

Student participation during advisories;

Teacher-written narratives on

student participation and behavior in classes and

relationships with other students and teachers;

Information obtained through case studies;

Information gained through in

depth staff and committee

meetings with colleagues;

Student responses ?

positive or

negative ? to school's set of

cultural expectations;

Comfort within the social mix of the school setting;

Information on the physical space and time allotted for work and study in the home setting;

Accessibility of computers and other useful technology outside the school setting;

Degree of parental or extended

family support;

Availability of reading materials in the home;

Extent of supportive peer network (including schools'

graduates); and

Feedback from supervisors at

community service and

internship sites.

motivation, strengths, and weaknesses

that can make the difference between success and failure and that test scores

do not and cannot reveal ? regardless of the type of test used.

In addition to the academic and nonacademic indicators, teachers also consider layers of nonacademic obsta cles when they work with students,

NONACADEMIC OBSTACLES AFFECTING ACADEMIC PERFORMANCE

Contrary to much of the current

rhetoric, nonacademic obstacies can

have a profound effect on academic

performance.

History of illnesses (for example, asthma, diabetes);

Health and physical condition of

family members or caregivers;

Pregnancies;

Responsibilities at home as

caregivers;

Responsibilities of after-school and weekend jobs to sustain

families;

Counseling availability outside of the school setting;

Homelessness;

Dependence on food stamps;

History of foster care;

Language interruption;

Lack of first-language literacy;

Continuous state of family crises;

Anger management issues;

History of incarceration,

probation history;

History of parental incarceration; and

History of suicide attempts.

many of whom come from greatly dis

advantaged neighborhoods. Both stu

dents and teachers want strategies that will help students overcome daunting obstacles.

The consortium schools succeed because they acknowledge that stu

dents face obstacles and because they have developed structures, methods, and assessments responsive to students'

needs. Students and teachers have es

tablished a different set of relationships and guiding principles than those based

wholly on a collection of quantitative data.

THE PERFORMANCE-ASSESSMENT CULTURE

To thrive, an effective performance assessment system requires the com

plete interplay of several components:

Teaching and learning must remain the constant focus of the school Staff energy is concentrated on developing interesting and challenging courses and instruc tion that engage students, many of

whom have been disengaged in the

learning process in their previous schools. Standards are high, with the conviction that students can succeed over time. Inquiry techniques and dis cussion-based classes motivate students

and keep them attending, even in

schools that are considered transfer or

"second-chance" schools.

Teams of teachers collaborate on all as

pects of instruction and assessment. They create professional communities, work

ing together to read student work, share

insights, evaluate teacher assignments, assess student performance, evaluate

assessment methods, and visit and ob serve each other's classes. Collaboration occurs within each school and across

schools, so that consortium teachers have opportunities for innovation and

leadership. New York state has sup

ported this local initiative and enabled it to flourish without imposing a top down, formulaic approach, which would diminish the high level of student work and teacher engagement.

Staff, department, and grade meetings are regular features of the school week.

Faculty members regularly share infor mation that helps them understand stu

58 Kappan March 2010

This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions

Page 6: Changing the National Conversation on Assessment

dents as learners in a far more effective

way than excessive dependence on data

spreadsheets can. In addition to aca

demic performance, staff members also consider what motivates a student and what personal, familial, or educational

challenges confront each student. In consortium schools, because each

child is known well by groups of teach

ers, staff members can easily share their

insights and knowledge regarding:

Strategies that work effectively with particular students;

Challenges (for example, math,

writing, anger management) facing each student;

Types of reading a student might avoid or welcome;

Work habits;

Unique talents and interests; and Attitudes about learning and school.

Scheduling includes blocks of time for teachers to mentor and supervise student

progress on the performance assessment

tasks. It is time-consuming work, but it

yields far more valuable information than most conventional approaches to

data gathering. Typically, each school schedules time for:

Participation in moderation studies by assembling staffs from

performance assessment schools

to evaluate students' interim assessments and graduation-level

tasks; Critical analysis of the

relationship between assignments and the student work they produce; Participation in professional development workshops led by experienced practitioners of

performance assessment;

Participation in student oral

presentations of work (panels, exhibitions, roundtables, discussions, defenses) in addition to reading and commenting closely on student written work,

conferencing with students on

written work, documenting the

drafting process;

Team teaching and mentor/mentee relationships, both of which require time built into the schedule for planning conferences;

Participation in advisories, tutorials, and family conferences; and

Writing reports and narratives to

keep parents informed of students' progress toward

graduation.

Ongoing reflection on curriculum, and courses focuses on such questions as:

How is inquiry reinforced over time in the courses offered and the tasks assigned?

Are tasks focusing on higher order thinking skills? Are all students challenged with

college preparatory work? Do faculty expect and require independent and long-term projects and research? What is the feedback from graduates?

What is the feedback of visitors, external assessors, the Performance

Assessment Review Board? How have parents and other informed adults continued to be involved in the performance assessment system?

Continual development of new courses

and units of study to better engage students in their learning and meet their academic needs. Teachers share curricula, lead

workshops for each other on pedagogic methods, and design new course ideas and lessons together, either within a de

partment or as interdisciplinary proj ects.

FUTURE GOALS

A number of states have adopted new approaches to assessment, looking beyond the limitations of standardized tests and quantitative data. They are in

corporating assessments that value a

qualitative approach, "a complex, holis tic picture" of each student.

In New York, the performance as sessment system developed by the con

sortium schools has succeeded in meet

ing student needs, preparing them for success in postsecondary work, and

providing the support they need for se

curing their futures. Just as important, it has engaged its teachers in a profes sional community of educators com

mitted to students and intent on devel

oping challenging curricula, meaning mi instruction, and many and diverse

opportunities to learn. If we're serious about preparing students for this cen

tury, and not the last, the consortium assessment system has important les

sons to teach. The layers of authentic,

qualitative information gathered by teachers and staff members contribute to the schools' successes and deserve serious consideration as a worthwhile assessment system for other schools that choose to accept the challenge. IC

REFERENCES

Biesta, Gert J. "Good Education in An Age of

Measurement: On the Need to Reconnect with

the Question of Purpose in Education."

Educational Assessment, Evaluation, and

Accountability 21, no. 1 (2009): 33-46.

Campbell, Donald T. "Assessing the Impact of

Planned Social Change." Occasional Paper

Series, No. 8. Hanover, N.H.: Public Affairs

Center, Dartmouth College, 1976.

Creswell, John W. Oualitative Inquiry and

Research Design. Thousand Oaks, Calif.: Sage

Publications, 1998.

Foote, Martha. "Keeping Accountability

Systems Accountable." Phi Delta Kappan 88,

no. 5 (January 2007): 359-363.

Schm?ker, Mike. "Measuring What Matters."

Educational Leadership 66, no. 4 (December

2008/January 2009): 70-74.

Learn more about the New York

Performance Standards Consortium,

including examples of student work.

http://performanceassessment.org

V91N6 Kappan 59

This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions