Upload
phyllis-tashlik
View
214
Download
0
Embed Size (px)
Citation preview
Changing the National Conversation on AssessmentAuthor(s): Phyllis TashlikSource: The Phi Delta Kappan, Vol. 91, No. 6 (Mar., 2010), pp. 55-59Published by: Phi Delta Kappa InternationalStable URL: http://www.jstor.org/stable/27755670 .
Accessed: 25/06/2014 02:58
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp
.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].
.
Phi Delta Kappa International is collaborating with JSTOR to digitize, preserve and extend access to The PhiDelta Kappan.
http://www.jstor.org
This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions
Changing the National
Conversation on Assessment
A consortium of New York public high schools serves as a model for a multi-dimensional system that is
performance-based.
By Phyllis Tashlik
"The sharp separation often seen in the literature between qualitative and quantitative methods is a spurious one."
? Bent Flyvbjerg
"The conventions of standardized testing have become so widely accepted that many evaluators cannot think of assessment based on project-based individualized educa tion."
? Robert E. Stake, director, Center for Instructional Research and
Curriculum Evaluation, University of Illinois, Urbana-Champaign
Given the narrowed focus of the current conversation about
assessment, it's hard
to conceive how it could possibly be moved to change. From Secretary of
Education Arne Duncan to state com
missioners, chancellors, mayors, and the press, the language of quantitative
measures has dominated the "conversa
tion." "Assessment" has come to mean
really one thing only: the numerical re sults of standardized tests.
Social scientists have long cited the critical shortcomings of quantitative in dicators. As Donald T. Campbell wrote, in what has now become a truism for those involved with public policy, "The
more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor" (1976).
Despite Campbell's warning and the varieties of corrupting influences that have been documented across the states, more and more school systems rely on an excessive series of standardized exams to determine an ever-widening range of education decisions: promotion and
graduation requirements for students, bonuses for principals, tenure and salaries for teachers, ratings for schools, and school closings. We run the risk of
"valuing what we measure [rather than] measuring what we value" (Biesta
PHYLLIS TASHLIK is director of the Center for Inquiry in Teaching and Learning, New York Per
formance Standards Consortium.
ixikmtLrMxi V91 N6 Kappan 55
This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions
2009). The possibilities for other options ?
performance-based or
qualitative or
project-based, whatever they may be called ? have become severely restricted if not totally eliminated.
We run the risk of
"valuing what we
measure [rather than]
measuring what we
value,"
But one group of New York public high schools has managed to defy the odds. The staffs of these schools have sur
vived the era of "one size fits all" by cre
ating and sustaining an entire perform ance-based assessment system. Perhaps the conversation can still be changed.
PERFORMANCE-ASSESSMENT SCHOOLS
A group of 30 public schools in New
York, working together as the New York Performance Standards Consortium since 1998, have been successfully grad uating students using a performance based assessment option (in addition to the New York State Regents exam in
English Language Arts). They have been at the forefront of changing the conversation about assessment and re
defining "data" and how best to use it to
engage both students and teachers in the
complex tasks of teaching and learning. Researchers have shown that con
sortium graduates succeed in college, achieving a higher GPA than the na
tional norm, placing into credit-bear
ing courses (thus avoiding the expen sive but uncredited remedial classes), and outpacing national rates for return
ing to college as second-year students. This achievement is all the more im
pressive when considering that they en
tered high school with lower scores on
English and math state exams, a higher percentage of students in special educa
tion, and higher rates of poverty when
compared to the overall New York City school population (Foote 2007).
Consortium schools include urban
public high schools in New York City, Rochester, and Ithaca. Their assessment
system meets the New York State Standards for Learning. Instead of exit
exams, student assessments are based
on specific performance-based assess ment tasks (PBATs) that grow out of the schools' curricula. With students
expected to demonstrate college-level skills, the curriculum must be challeng ing as well as engaging. Too often, in
systems governed solely by standard ized exit exams, the curriculum becomes
overwhelmingly test-prep, a series of interim assessments and assignments that mirror the standardized exam.
The consortium's PBATs include an
analytic, comparative literary essay on two works of literature; an intensive so
cial studies research paper; a hands-on,
original science experiment that is well documented in science reports; and ap plications of higher-order mathematics to problem solving. Each school then
stipulates additional graduation tasks. For example, schools have added per formance tasks in the creative arts, arts
criticism, foreign language, and intern
ships. All tasks are the summative indi cators of years spent in classrooms where standards for teaching inquiry, higher-order thinking skills, analysis, and synthesis are daily goals. Classroom teachers and external assessors evaluate
student papers using the consortium
rubrics, which consortium teachers re
vise every two years or as needed.
In addition to the above tasks, con sortium schools require an oral compo nent ?
such as a presentation, exhibi
tion, debate, or defense ? in each of the
disciplines. Like the written work, these oral components also involve external assessors. A number of different options are available for the oral presentations: Some take place as panels, some are one-on-one interviews with external as
sessors, some are exhibitions to a group of assessors. Schools have developed a
pool of subject-area experts, writers,
journalists, and lawyers who volunteer to participate; parents are also involved in the process. Participants receive the
Student papers in advance to prepare for the presentation. (The consortium is
producing a DVD that will focus on the oral component; it will be part of the Teacher to Teacher series available from Teachers College Press.)
The system itself is reviewed by the Performance Assessment Review Board, an affiliation of nationally respected ed ucators who visit the schools and pre pare reports based on what they've ob served and the documents they've read.
DEMONSTRATE RELIABILITY
As a further check on the objectivity and reliability of the system, consor
tium schools participate in an ongoing moderation study,
a process for evalu
ating the consistency of the assessment
system. Hundreds of teachers across
the consortium schools are involved in
reading and assessing each other's stu
dent work and conferencing on the standards that teachers expect students to meet, regardless of their skill level
when they enter the schools. What is most impressive in this system is the maintenance of high standards for all
students, including students in "trans fer schools" (that is, second-chance
schools), students for whom English is their second language, and students in
special education. The four phases of the moderation
study look like this:
Phase 1:150 teacher representatives read submitted student work and assess
it using consortium rubrics. Phase II: Teachers at each school
read and evaluate a selection from the same collection of papers. Schools are
later informed of the consensus
reached during the two phases and whether the school's own assessment
matches the assessment reached by the
larger group of consortium teachers. Phase III: Teachers focus on out
standing student work and deepen their
understanding about the curriculum and assignments most likely to produce such work.
Phase IV: Educators external to the
system go through the same process as
consortium teachers to provide a check on their work.
56 Kappan March 2010
This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions
Each fall, teachers participate in fol
low-up sessions, both for assessment
purposes and as a productive and stim
ulating professional development expe rience.
Following the first two years of moderation work, the consortium pub lished booklets of exemplar student pa pers in literature, social studies, math, and science. The exemplars
now pro
vide an added level of consistency and
reliability to the system. The data gathered by teachers
through the students' written and oral
presentations constitute just a part of the important information used to as sess students and their accomplish
ments. As a fully realized performance assessment system, other indicators are
also taken into consideration when ad
justing instruction, developing curricu
lum, and reaching decisions about pro motion and graduation. Instead of rely ing solely on the results of a standard ized exit exam ? an incomplete and not
always reliable quantitative representa tion of what students know and can do
? the performance assessment system
requires and is capable of producing more and varied comprehensive data, which lead to a better understanding of students and their accomplishments, as
well as their needs.
A MORE COMPLEX UNDERSTANDING OF DATA
In New York and elsewhere, the small school movement encouraged building strong communities within small, inti
mate environments. Small public schools were founded on lofty principles, such as educating the whole child, designing student-centered innovative curricula,
building professional communities, and
providing individualized interventions for students. As the creed of quantita tive data monopolized local, state, and national conversations, many small
schools found themselves struggling to
maintain their identity as communities with a mission, but the performance as sessment schools have been steadfast in their founding principles. When com
pared with the restrictions of schools
geared toward standardized testing, they are the institutions most in touch
with meeting the academic, social, and
psychological needs of their students, the burgeoning requirements for achieve
ment in the 21st century, and the "skills. . . crucial to students' success as
college freshmen, professionals, and
participants in a democracy" (Schmoker 2008-09).
John W. Creswell, co-director of the Office of Qualitative and Mixed
Methods Research at the University of
Nebraska-Lincoln, describes qualitative inquiry as "a complex, holistic picture," one that requires us to consider the "mul
tiple dimensions" of the issues and an
understanding of all its complexity (1998). The scores of standardized tests
do not meet that requirement; nor do
they provide an adequate measure for
helping students succeed in the long term. In fact, too often, quantitative
meas
ures present only a negative picture of our students so that they, and others, become convinced that they can't suc
ceed, a phenomena all too common in the urban public schools that poor and
minority students attend. As one prin cipal recently remarked, "About the
only thing we learn from all the data go ing into the new reporting system is at tendance patterns
? which day of the week kids seem to miss school most.
Otherwise, teachers already know the information and have informed parents in a far more detailed and contextual ized way than the data offer."
The success of schools engaged in
performance assessment has shown that it is the tests that have failed our stu dents and not vice versa. To demon
strate the complexity of the issue and the "data" needed for making substan tive decisions about students, at right and on the next page are lists of the in dicators and obstacles ? both academic and nonacademic
? that consortium
teachers consider when assessing their students and developing their schools'
program. Consideration of these phe nomena has translated into a more nu
anced approach to curriculum develop ment, classroom instruction, making de
cisions about students' futures, interim and summative assessments, and coun
seling. The result is quite different from the narrow understanding of "data" and
ACADEMIC INDICATORS
Classroom artifacts that help teachers learn about their students
Student work collected over time: evidence of growth over
time, not limited to a two-hour
exam;
Examples of thoughtful revisions of research and writing;
Performance on teacher
constructed exams and
homework, which grow out of classroom instruction and
course content;
Performance as evaluated by teacher-developed and -revised
rubrics;
Completion of course
requirements, which involve
reading complex texts and
writing extensively;
Classroom discussions and level of participation;
Questions asked and comments made during class;
Student participation in out-of
classroom interviews and field
trips, and interactions with
experts invited to make
presentations;
Student self-assessments and
their analyses of their
assessments; and
Student self-awareness as a
learner, reader, and writer.
V91N6 Kappan 57
This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions
assessment prevalent in rigid test-bound
systems.
Through nonacademic indicators, teachers learn about personality traits,
NONACADEMIC INDICATORS
Qualitative data that help teachers better understand and teach their students.
Student participation in tutorials;
Student participation during advisories;
Teacher-written narratives on
student participation and behavior in classes and
relationships with other students and teachers;
Information obtained through case studies;
Information gained through in
depth staff and committee
meetings with colleagues;
Student responses ?
positive or
negative ? to school's set of
cultural expectations;
Comfort within the social mix of the school setting;
Information on the physical space and time allotted for work and study in the home setting;
Accessibility of computers and other useful technology outside the school setting;
Degree of parental or extended
family support;
Availability of reading materials in the home;
Extent of supportive peer network (including schools'
graduates); and
Feedback from supervisors at
community service and
internship sites.
motivation, strengths, and weaknesses
that can make the difference between success and failure and that test scores
do not and cannot reveal ? regardless of the type of test used.
In addition to the academic and nonacademic indicators, teachers also consider layers of nonacademic obsta cles when they work with students,
NONACADEMIC OBSTACLES AFFECTING ACADEMIC PERFORMANCE
Contrary to much of the current
rhetoric, nonacademic obstacies can
have a profound effect on academic
performance.
History of illnesses (for example, asthma, diabetes);
Health and physical condition of
family members or caregivers;
Pregnancies;
Responsibilities at home as
caregivers;
Responsibilities of after-school and weekend jobs to sustain
families;
Counseling availability outside of the school setting;
Homelessness;
Dependence on food stamps;
History of foster care;
Language interruption;
Lack of first-language literacy;
Continuous state of family crises;
Anger management issues;
History of incarceration,
probation history;
History of parental incarceration; and
History of suicide attempts.
many of whom come from greatly dis
advantaged neighborhoods. Both stu
dents and teachers want strategies that will help students overcome daunting obstacles.
The consortium schools succeed because they acknowledge that stu
dents face obstacles and because they have developed structures, methods, and assessments responsive to students'
needs. Students and teachers have es
tablished a different set of relationships and guiding principles than those based
wholly on a collection of quantitative data.
THE PERFORMANCE-ASSESSMENT CULTURE
To thrive, an effective performance assessment system requires the com
plete interplay of several components:
Teaching and learning must remain the constant focus of the school Staff energy is concentrated on developing interesting and challenging courses and instruc tion that engage students, many of
whom have been disengaged in the
learning process in their previous schools. Standards are high, with the conviction that students can succeed over time. Inquiry techniques and dis cussion-based classes motivate students
and keep them attending, even in
schools that are considered transfer or
"second-chance" schools.
Teams of teachers collaborate on all as
pects of instruction and assessment. They create professional communities, work
ing together to read student work, share
insights, evaluate teacher assignments, assess student performance, evaluate
assessment methods, and visit and ob serve each other's classes. Collaboration occurs within each school and across
schools, so that consortium teachers have opportunities for innovation and
leadership. New York state has sup
ported this local initiative and enabled it to flourish without imposing a top down, formulaic approach, which would diminish the high level of student work and teacher engagement.
Staff, department, and grade meetings are regular features of the school week.
Faculty members regularly share infor mation that helps them understand stu
58 Kappan March 2010
This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions
dents as learners in a far more effective
way than excessive dependence on data
spreadsheets can. In addition to aca
demic performance, staff members also consider what motivates a student and what personal, familial, or educational
challenges confront each student. In consortium schools, because each
child is known well by groups of teach
ers, staff members can easily share their
insights and knowledge regarding:
Strategies that work effectively with particular students;
Challenges (for example, math,
writing, anger management) facing each student;
Types of reading a student might avoid or welcome;
Work habits;
Unique talents and interests; and Attitudes about learning and school.
Scheduling includes blocks of time for teachers to mentor and supervise student
progress on the performance assessment
tasks. It is time-consuming work, but it
yields far more valuable information than most conventional approaches to
data gathering. Typically, each school schedules time for:
Participation in moderation studies by assembling staffs from
performance assessment schools
to evaluate students' interim assessments and graduation-level
tasks; Critical analysis of the
relationship between assignments and the student work they produce; Participation in professional development workshops led by experienced practitioners of
performance assessment;
Participation in student oral
presentations of work (panels, exhibitions, roundtables, discussions, defenses) in addition to reading and commenting closely on student written work,
conferencing with students on
written work, documenting the
drafting process;
Team teaching and mentor/mentee relationships, both of which require time built into the schedule for planning conferences;
Participation in advisories, tutorials, and family conferences; and
Writing reports and narratives to
keep parents informed of students' progress toward
graduation.
Ongoing reflection on curriculum, and courses focuses on such questions as:
How is inquiry reinforced over time in the courses offered and the tasks assigned?
Are tasks focusing on higher order thinking skills? Are all students challenged with
college preparatory work? Do faculty expect and require independent and long-term projects and research? What is the feedback from graduates?
What is the feedback of visitors, external assessors, the Performance
Assessment Review Board? How have parents and other informed adults continued to be involved in the performance assessment system?
Continual development of new courses
and units of study to better engage students in their learning and meet their academic needs. Teachers share curricula, lead
workshops for each other on pedagogic methods, and design new course ideas and lessons together, either within a de
partment or as interdisciplinary proj ects.
FUTURE GOALS
A number of states have adopted new approaches to assessment, looking beyond the limitations of standardized tests and quantitative data. They are in
corporating assessments that value a
qualitative approach, "a complex, holis tic picture" of each student.
In New York, the performance as sessment system developed by the con
sortium schools has succeeded in meet
ing student needs, preparing them for success in postsecondary work, and
providing the support they need for se
curing their futures. Just as important, it has engaged its teachers in a profes sional community of educators com
mitted to students and intent on devel
oping challenging curricula, meaning mi instruction, and many and diverse
opportunities to learn. If we're serious about preparing students for this cen
tury, and not the last, the consortium assessment system has important les
sons to teach. The layers of authentic,
qualitative information gathered by teachers and staff members contribute to the schools' successes and deserve serious consideration as a worthwhile assessment system for other schools that choose to accept the challenge. IC
REFERENCES
Biesta, Gert J. "Good Education in An Age of
Measurement: On the Need to Reconnect with
the Question of Purpose in Education."
Educational Assessment, Evaluation, and
Accountability 21, no. 1 (2009): 33-46.
Campbell, Donald T. "Assessing the Impact of
Planned Social Change." Occasional Paper
Series, No. 8. Hanover, N.H.: Public Affairs
Center, Dartmouth College, 1976.
Creswell, John W. Oualitative Inquiry and
Research Design. Thousand Oaks, Calif.: Sage
Publications, 1998.
Foote, Martha. "Keeping Accountability
Systems Accountable." Phi Delta Kappan 88,
no. 5 (January 2007): 359-363.
Schm?ker, Mike. "Measuring What Matters."
Educational Leadership 66, no. 4 (December
2008/January 2009): 70-74.
Learn more about the New York
Performance Standards Consortium,
including examples of student work.
http://performanceassessment.org
V91N6 Kappan 59
This content downloaded from 62.122.77.83 on Wed, 25 Jun 2014 02:58:16 AMAll use subject to JSTOR Terms and Conditions