Reproductions supplied by EDRS are the best that can be ... · Susan Lytle, Bernard McMullan, Ruth Curran Neild, Claire Passantino, Rhonda Phillips, Amy Rhodes, Elaine Simon, Mary

DOCUMENT RESUME

ED 476 940 UD 035 685

AUTHOR Corcoran, Tom; Christman, Jolley Bruce

TITLE The Limits and Contradictions of Systemic Reform: ThePhiladelphia Story.

INSTITUTION Consortium for Policy Research in Education, Philadelphia,PA.; Research for Action, Inc., Philadelphia, PA.

SPONS AGENCY Pew Charitable Trusts, Philadelphia, PA.; AnnenbergFoundation, St. Davids, PA.

PUB DATE 2002-11-00

NOTE 47p.; Additional support from Greater Philadelphia First. Fora related document, see UD 035 364.

AVAILABLE FROM Consortium for Policy Research in Education, Graduate Schoolof Education, University of Pennsylvania, 3440 Market Street,Suite 560, Philadelphia, PA 19104-3325. Tel: 215-573-0700;Fax: 215-573-7914; Web site: http://www.cpre.org.

PUB TYPE Reports Evaluative (142)EDRS PRICE EDRS Price MF01/PCO2 Plus Postage.DESCRIPTORS Academic Standards; Accountability; Community Involvement;

Decentralization; *Educational Change; Elementary SecondaryEducation; Instructional Leadership; Parent Participation;Public Schools ; Resource Allocation; *Urban Schools

IDENTIFIERS *Philadelphia School District PA; Reform Efforts; *SystemicChange

ABSTRACT

This report examines Philadelphia, Pennsylvania's AnnenbergChallenge, Children Achieving, a districtwide systemic reform initiativedesigned and led by a small core group of district officials and externalpartners. It was organized around eight core strategies (fair funding,standards, accountability, decentralization, leadership and support forteachers and principals, better coordination of resources, civic and parentengagement, and the all at once philosophy). A combination of factorscontributed to Children Achieving's failure to produce sustained improvementsand deep changes in practice and included flaws in the theory of actionitself, flaws in implementation, lack of capacity, and behavior inconsistentwith underlying beliefs and values. Reformers encouraged stakeholders tobelieve that standards, accountability, and decentralization would result inimproved student performance,' but this did not occur. Lessons learnedinclude: content standards and accountability measures offer a foundation forinstructional change, but produce results only if teachers are given the timeand support needed to develop high-quality curricula and alter theirclassroom methods; curriculum-based, sustained professional development isessential to instructional change; and too much reform can overload schoolsand teachers. Two appendices present additional reading on Children Achievingand research methodology. (Contains 28 endnotes(SM)

Reproductions supplied by EDRS are the best that can be madefrom the original document.

.

-

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL HAS

BEEN GRANTED BY

L.Ple_Paelcultd _

_ I et5taaitTO THE EDUCATIO AL R S URCES

INFORMATION CENTER (ERIC)

1

a1

U S DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it

Minor changes have been made toimprove reproduction quality

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI positior or policy

ela

9

I

S

P A

71111111110%.

BEST COPY AVAILABLE

Biographies of the AuthorsTom Corcoran is co-director of the Consortium for Policy Research inEducation (CPRE). Prior to joining CPRE, Corcoran served as the policyadvisor for education for Governor Florio of New Jersey, director of schoolimprovement for Research for Better Schools, and director of evaluation andlater chief-of-staff of the New jersey Department of Education. His majorresearch interests are the use of research to inform policy and practice inpublic education, the efficacy of different approaches to professionaldevelopment, the effectiveness of whole-school reforms, the impact ofchanges in work environments on the productivity of teachers and students,and the factors affecting the effectiveness of scaling-up strategies.

Jolley Bruce Christman is Principal of Research for Action and has authorednumerous evaluation reports and journal articles including "The Five SchoolStudy Restructuring Philadelphia's Comprehensive High Schools" and"Taking Stock/Making Change: Stories of Collaboration in Local SchoolReform" (with Fred Erickson). Her research interests include school reform,gender and education, and participatory evaluation. She received the Councilon Anthropology and Education's (CAE) award for Excellence inEthnographic Evaluation and currently serves on CAE's hoard. Christman alsoteaches at the University of Pennsylvania,

ST COPY AVAILABLE

The Limitsand Contradictions of

Systemic Reform:The Philadelphia Story

By

Tom CorcoranJolley Bruce Christman

November 2002Copyright 2002 by the Consortium for Policy Research in Education

4

Consortium for Policy Research in EducationResearch for Action

Table of ContentsAcknowledgments iv

Authors' Note iv

About this ReportTimeline vi

Background 1

Arrival of Superintendent David Hornbeck 1

The Results of Reform 3

Elementary Schools Show Real Gains 4

Middle Schools Offer a Mixed Story 4

High Schools Struggle 5

Strategies of Reform within Children Achieving 7

Strategy One: Fair Funding 9

Strategy Two: Standards 11

Strategy Three: Accountability 13

Strategy Four: Decentralization 17

Central Office 17

Cluster Offices 18

Local School Councils 20

Small Learning Communities 20

Strategy Five: Leadership and Support for Teachers and Principals 23

Strategy Six: Better Coordination of Resources 25

Strategy Seven: Civic and Parent Engagement 27

Strategy Eight: "All at Once" 31

Lessons Learned 33

Remedial Lessons about Teaching and Learning 33

New Lessons about Systemic Reform 34

In Conclusion 37

Endnotes 39

Appendix A - Additional Reading on Children Achieving 41

Appendix B Children Achieving Evaluation Research Methods 43

List of FiguresFigure 1. Percent of Students Scoring At or Above Basic in Math

by School Level, SAT-9, 1996-2000 4

Figure 2. Percent of Students Scoring At or Above Basic in Readingby School Level, SAT-9, 1996-2000 5

Figure 3. Percent of Students Scoring At or Above Basic in Scienceby School Level, SAT-9, 1996-2000 6

Figure 4. Performance Levels on the SAT-9 15

Figure 5. Diagram of Core Beliefs and Values andtheir Effects on Centralization and Decentralizationin the School District of Philadelphia 18

The Limits and Contradictions of Systemic Reform:The Philadelphia Story

Acknowledgments

This summary report is based on several years ofwork by a large team of people from theConsortium for Policy Research in Education andResearch for Action. The authors thank all whohave contributed including: Shirley Brown, EllenFoley, Eva Gold, Nancy Lawrence, Theresa Luhm,Susan Lytle, Bernard McMullan, Ruth CurranNeild, Claire Passantino, Rhonda Phillips, AmyRhodes, Elaine Simon, Mary Helen Spiri, ErinTighe, Diane Waff, Aubrey Wang, Susan Watson,and Hitomi Yoshida. We are also grateful toCPRE's communications staff who supported ourwork including: Deborah Schwartz Alexander, GailDanckert, Mar lies Dunson, Christine Gazzara,Robb Sewell, Kelly Stanton and consultants JaneCentury and Joan Stevens. We are indebted aswell to the late Peg Hoppe whose contributionsto this project were immeasurable.

We wish to thank the many staff members of thePhiladelphia School District in the schools, clusteroffices, and central office who generously gave ustheir time and shared their work. We particularlywant to thank David Hornbeck, Katherine Conner,and Mitchell Chester for their openness and fortaking the time to review countless drafts ofreports. We also want to thank the staff of theOffice of Accountability and Assessment at theSchool District of Philadelphia, especially Tom

Clark, Michael Schlesinger, and Robert Offenbergfor their help in supplying important data.

We thank Jack Steinberg and Rosalind Jones-Johnson of the Philadelphia Federation ofTeachers (PFT) and the building representatives ofthe PFT for their help in the design and adminis-tration of two district-wide teacher surveys.

And we thank Terry Orr and colleagues at theNational Center for Restructuring Education,Schools, and Teaching, and Betsy Useem and hercolleagues at the Philadelphia Education Fund forsharing the results of their research inPhiladelphia and for being willing to review ours.Thanks also go to Vicki Phillips and SuzanneBecker, the two executive directors of theChildren Achieving Challenge at GreaterPhiladelphia First, for their help in reviewingreports, and to Warren Simmons and NancyMcGinley, the two executive directors with whomwe worked at the Philadelphia Education Fund.

We acknowledge the support and patience of theAnnenberg Foundation, the Pew CharitableTrusts, and Greater Philadelphia First. Finally, ourthanks to the three members of our advisorygroup, Karen Seashore Lewis, Ruben Carriedo,and Dick Clark, for their wise counsel over thecourse of this study.

Authors' Note

The research reported herein was conducted bythe Consortium for Policy Research in Educationand Research for Action. Funding for this workwas provided by Greater Philadelphia First andthe Pew Charitable Trust. Opinions expressed in

iv

this report are those of the authors, and do notnecessarily reflect the views of GreaterPhiladelphia First, the Pew Charitable Trusts, orthe institutional partners of CPRE.

6


About this Report

In Philadelphia, the Annenberg Challenge wasknown as Children Achieving and was adistrictwide systemic reform initiative designedand led by a small core group of District officialsand external partners. This report examines theChildren Achieving Challenge and the strategiesthe designers employed to improve teaching andlearning in the public schools. Among the condi-tions associated with the Annenberg Challengewere requirements that two matching dollars beraised for each one received from the AnnenbergFoundation and that an independent manage-ment structure be created to provide program,fiscal, and evaluation oversight of the grant. InPhiladelphia, a business organization, GreaterPhiladelphia First, assumed these responsibilities,and with them, the challenge of working with theSchool District to build and sustain civic supportfor the improvement of the public schools.

The story of Children Achieving provides anopportunity to examine the dynamics and effectsof systemic reform in a large urban district. Thetheory of systemic reform seems straightforward.It holds that if districts and states set academicstandards for student performance; aligncurriculum, instruction, and assessment with thesestandards; measure students' progress; and offerrewards or sanctions to educators based on

performance, then school staffs will make thechanges in their practice necessary to ensure thatstudents achieve at high levels. From 1996 to2002, the Consortium for Policy Research inEducation (CPRE) and Research for Action (RFA)served as the evaluators of the Challenge inPhiladelphia and observed this theory being putinto practice. CPRE and RFA staff collected datafrom hundreds of Philadelphia schools, observedclassrooms, and interviewed teachers, principals,and other school officials. The District's testscores and other indicators of performance wereanalyzed, as were the results of two districtwidesurveys of teachers. From the analysis of thisdata, we have drawn some lessons that shouldinform educational reform in our large cities.

This report summarizes those lessons and drawson findings published in previous reports fromCPRE and RFA to provide an overall assessmentof the reforms undertaken in Philadelphia in the1990s. (See Appendix A for a list of otherreports). Information on research methods andsources of data are in Appendix B. The report isorganized around the eight core strategies of thereform and the rationale, design, implementation,and effectiveness of each strategy are discussed.The report concludes with some lessons drawnfrom the Philadelphia experience.


Children Achieving Challenge Timeline

1994

August David Hornbeckappointed Superintendent ofPhiladelphia schools.

October Philadelphia Inquirerpublishes "District in Distress," its

first report card on the state ofPhiladelphia public schools,including information on povertyrates, dropout rates, teachers'salaries, and student performanceon the SAT-9.

November As part of a desegre-gation case against the district,Judge Doris Smith orders theSchool District to submit a plan forreducing racial disparities in studentachievement.

1995

February Philadelphia receives a$50 million Annenberg Challengegrant, which is matched by $100million from Philadelphia corpora-tions, foundations, and federalgrants.

May Five months after takingoffice, Governor Ridge proposes astatewide voucher plan. This is thefirst of Ridge's three attempts toestablish a statewide voucherprogram, none of which were ableto garner needed legislativesupport.

December Standards WritingTeams, composed of parents,teachers, and community members,are convened; writing of newacademic standards begins.

1996

April - May SAT-9 administereddistrictwide in grades 2, 4, 6, 8, and11.

September All 22 clusters estab-lished. Standards CurriculumResource Guides distributed toteachers.

December Standards inreading/English/language arts,science, mathematics, and the artsadopted by School Board.

1997

February Mayor Rendell, CityCouncil President Street, Super-intendent Hornbeck, School Boardmembers, and community leadersissue Realities Converge: ThisYear is Different. The authorspromised a zero-growth SchoolDistrict budget but "drew a line in

the sand" and refused to cut anymore school-based programs.

District, city, and community leadersfile lawsuit against the statecontending that Pennsylvania doesnot provide a "thorough andefficient" education.

July Standards in final threesubjects adopted by School Board.

September Professional Respon-sibility Index scores made public forthe first time. This marks the firstreporting period of the first two-year accountability cycle (measuringprogress from 1996-1998).

1998

January Curriculum frameworksfor English/language arts, mathe-matics, science, and social studiesdistributed to all schools; graduationand promotion requirements identi-fied.

February Hornbeck threatens toadopt an unbalanced budget if thestate does not provide needed funds,which could lead to the schools

vi

closing before the end of the schoolyear.

March District, city, and otherofficials and interest groups file afederal civil-rights suit against thestate, contending that the state'sfunding practices discriminateagainst school districts with largenumbers of non-White students.

April State legislature respondsby passing Act 46, a state takeoverlaw aimed specifically atPhiladelphia.

May City Council adopts SchoolDistrict budget, relying on letters ofcredits from banks to avert an earlyschool closure in the 1998-1999school year.

BEST COPY AVAILABLE


1999

January David Hornbeck'scontract extended for two moreyears.

March School District presents

budget to City Council with projected

$94 million deficit for 1999-2000school year; refuses to make further

cuts.

November John Street iselected Mayor. Philadelphians pass

a referendum to change the CityCharter to allow the Mayor toappoint all members of the SchoolBoard with terms concurrent to hisown.

2000

January Street selects a new

School Board, the first time this hashappened in the history ofPhiladelphia and also appoints thefirst Secretary of Education for the city,Debra Kahn, who is charged withleading the district's team in negotia-tion of a new teachers' contract.

May Pennsylvania Legislaturepasses and Governor Ridge signsthe Education Empowerment Act,state reform and "takeover" billtargeted at 11 school districts,including Philadelphia, with highstudent failure rates.

June Threat of state takeover crisisin the district during the RepublicanConvention in Philadelphia is avertedby a financial settlement between theSchool District and Governor Ridge.Still facing a deficit, the School Boardcuts the budget and Hornbeckresigns in protest.

August Teacher contract expires,beginning two months of tense

a negotiating between the union andthe Board of Education. Thecontract is settled before school isdisrupted with the intervention ofMayor Street and pressure from

Governor Ridge, who threatens astate takeover.

Board of Education announcesdecision to adopt a corporate styleof management. Deidre Farmbry, aveteran Philadelphia educator,appointed Chief Academic Officer.

October Philip Goldsmith, alawyer and journalist, appointedinterim Chief Executive Officer.

2001

May - District's 22 clusters arereplaced by eight academic officesin a cost cutting effort. SchoolBoard adopts a budget with $216million deficit creating a new fiscalcrisis with state takeover of theDistrict possible.

July Mayor Street and GovernorRidge sign a Memorandum ofUnderstanding giving the Governor

the right to commission an analysisof the District's financial and educa-tional situation. Ridge hires EdisonSchools to do the analysis.

October Governor Ridge isappointed to lead U.S. Office ofHomeland Security. Lt. GovernorMark Schweiker replaces him.

December Philip Goldsmithresigns as CEO of the District. State

executes a "friendly takeover". Theschool board is replaced by a five-member School ReformCommission, with three membersappointed by the governor and twoby the mayor. Governor Schweikerappoints James Nevels as theinterim chairman. Other fourmembers appointed in January.

vii


Background

The Philadelphia School District has the dauntingresponsibility of providing an effective educationto 215,000 students, most of whom live inpoverty and must cope with the stress andproblems associated with severe economic disad-vantage. Eighty percent are children of color whoare also affected by the racial discriminationdirected at their families and communities. Thedistrict faces challenges common to other urbanschool districts low achievement and highdropout rates, inadequate funding, decayingbuildings, and high teacher and principalturnover. This story is not new and these andrelated problems have persisted for decades. ThePhiladelphia Inquirer, in 1994, published a reportabout the Philadelphia schools called "A Districtin Distress" that outlined the grim realities in theschools at that time:

Less than half of Philadelphia studentsentering high school in 1989 graduated fouryears later.

Students in only 15 out of 171 elementaryschools scored at or above average onnationally normed reading tests.

Test scores were highly correlated withpoverty levels.

Philadelphia per-pupil funding was the lowestin the area.

In 1995, some hope arrived. Philadelphia was oneof 16 locations nationwide designated to receivelarge grants through the Annenberg Challengeprogram. Walter Annenberg, benefactor andformer Ambassador to Great Britain, announcedan unprecedented gift of $500 million for publiceducation in America and challenged citiesaround the country to "speed the movement" ofeducational reform initiatives. One condition ofthe Annenberg Challenge was that each of thegrants, varying in amounts from $10 to $50million, had to be matched two to one withadditional dollars raised by each city.Philadelphia's $50 million grant was successfully

matched with $100 million raised by Philadelphiacorporations and foundations, and federal grants.

Arrival of SuperintendentDavid Hornbeck

Immediately prior to the launch of the AnnenbergChallenge in Philadelphia, a new superintendent,David Hornbeck, had been recruited by a coali-tion of Philadelphia business, civic, andgovernment leaders. Hornbeck was an advocateof systemic reform and had been one of thearchitects of Kentucky's Education Reform Act. Hebelieved that content and performancestandards, decentralization, and accountabilitymechanisms should be used in concert to trans-form entire school systems rather than seekingimprovement school-by-school. His theory ofaction, as articulated to CPRE and RFAresearchers, summarizes Children Achieving'sreform agenda:

Given high academic standards and strong incen-tives to focus their efforts and resources; morecontrol over school resource allocations, organiza-tion, policies, and programs; adequate fundingand resources; more hands-on leadership andhigh-quality support; better coordination ofresources and programs; schools restructured tosupport good teaching and encourage improve-ment of practice; rich professional developmentof their own choosing and increased publicunderstanding and support; the teachers andadministrators of the Philadelphia schools willdevelop, adopt, or adapt instructional technolo-gies and patterns of behavior that will help allchildren reach the district's high standards.

Philadelphia's leaders welcomed Hornbeck'sapproach. They saw high standards and strongaccountability as a tonic for Philadelphia's poorlyperforming schools. New leadership with apersuasive strategy combined with the award ofthe Annenberg Challenge grant and the receiptof the matching funds brought hope thatsomething would finally be done to turn the city'sschools around.

101


To put his theory into practice in Philadelphia,Hornbeck developed a 10-point plan thatoutlined the steps the district would take to makedramatic improvements in the Philadelphiaschools. Hornbeck believed that these strategies

2

would bring about dramatic improvement inPhiladelphia. He set an ambitious goal of havingall students achieve at or above the standards in12 years.These points were:

1. Set high expectations for all students so that everychild gets the "basics" and a lot more.

2. Set standards to measure the results of reforms anduse these measures to hold educators accountable.

3. Shrink the centralized bureaucracy and let schools makemore decisions.

4. Provide intensive and sustained training to staff sothey can meet the tough challenges ahead.

5. Make sure all students are healthy and ready to learn.

6. Provide students with the community support andservices they need to succeed in school.

7. Provide up-to-date technology: one computer for every sixstudents, books, and clean and safe schools.

8. Engage the public in understanding, supporting, and participatingin school reform.

9. Ensure adequate financial and other resources, anduse them effectively.

10. Be prepared to address all these priorities together,all at once, and for the long term starting now.

BEST COPY AVAILABLE

11


wher

The Results of Reform

-consistent irnpf%tvernents in instruc io ticesand sustained gains in student achievement, there was strong

school leadership focused on instruction, a sense of professionalcommunity; curriculum-based professional development, and effective

use of data including review of student work"

We will never know if Children Achieving wouldhave produced the performance gains thatHornbeck envisioned. Political and financialproblems brought Hornbeck's tenure to an end inthe summer of 2000, and the Children Achievinginitiative essentially ended with his departure.What we do know is that modest gains weremade in the first five years (See Figures 1, 2, and3 for SAT-9 data). Based on the magnitude ofthose gains and the trends appearing in the data,it seems unlikely that the district would haveachieved the ambitious goal of having thestudent population on average reach thestandards by 2007 or 2008. Nevertheless, gainswere made. Here we describe the progress madeat the elementary, middle, and high school levels.

The Children Achieving reformers did not initiallyarticulate distinct strategies for elementary,middle, and high schools. The design was to alarge extent a one-size-fits-all approach. However,there was some customizing of the design toaddress issues unique to different levels ofschooling. At each level, the challenges ofmotivating students and staff varied and differentsupports were needed to make the changes. Thedistrict's focus on literacy in the early elementarygrades reaped significant benefits becauseelementary teachers were provided with guidanceabout how to implement research-based instruc-tional practices in their classrooms and with thematerials and tools they needed to do the job.But this kind of districtwide focus on instructionwas missing in the middle and high schools andmany of these schools floundered attempting todesign their own instructional interventions.

Clearly, the issues faced by the high schools werethe most daunting. Faculty were asked to makemajor changes in curriculum and instruction with

little guidance or support. They were asked totake ill-prepared students to higher levels ofperformance without being offered plausiblestrategies for helping their students catch up.They were asked to create small learning commu-nities and introduce school-to-work programs,service requirements, and senior projects at thesame time. All of this had to be done in largeschools in which many faculty members wereisolated and demoralized, and many werestrongly resistant to the reforms. Attention tothese issues might have helped high schools toimprove student achievement. The experience inPhiladelphia suggests that reform strategies mustbe customized to address the issues at each levelof schooling to have a chance of success andthere must be strong guidance and supportabout curriculum and instruction to produceresults.

Nevertheless, some schools made significantprogress and they shared some common attrib-utes regardless of grade level. Qualitativeresearch in 41 schools indicated that where therewere consistent improvements in instructionalpractices and sustained gains in student achieve-ment, there was strong school leadership focusedon instruction, a sense of professional community,curriculum-based professional development, andeffective use of data including review of studentwork. In many of these schools, parents wereaccepted as partners in improving schoolperformance. While these factors were importantat all grade levels, these conditions were seenmore frequently in elementary schools than inmiddle schools or high schools.

12

BEST COPY AVAILABLE

3


- " :A $

mom

70.0

60.0

50.05L. 40.0

30.0

20.0

10.0 _

.01995-1996 1996-1997 1997-1998 1998-1999 1999-2000

32.2 41.9 45.3 47.8 49.1ES

40.4 48.5 53.7 54.5 NAK-8

15.7 18.5 25.2 24.4 23.2MS

HS 11.6 13.8 15.9 15.2 16.4

Elementary Schools ShowReal Gains

A critical mass of elementary schools made signif-icant gains in student performance in reading asmeasured by test scores, but there were notcomparable gains in mathematics. The strongcurricular focus on early literacy had a majorimpact on student achievement as did the estab-lishment of full-day kindergarten. In theelementary schools, professional developmentfocused primarily on the literacy curriculum andresearch-based teaching practices. And smalllearning communities were built on the founda-tion of the working relationships teachers hadestablished in grade-level groups. This helpedteachers to learn how to use the new materialsand practices associated with the balancedliteracy initiative as they were able to share theirideas, struggles, and questions with their smalllearning community colleagues. The literacyinitiative gave teachers in the small learningcommunities a shared focus and purpose that inturn strengthened professional community. Theresult was continuous progress in improving

4

School Year

reading achievement. There was no comparabledistrict initiative in mathematics in the elementaryschools. Since elementary teachers often lackadequate preparation to teach mathematics inthe first place, they could not be expected toimprove mathematics instruction on their own.The lack of a coherent district strategy produceda predictable result in most schools; there waslittle improvement in mathematics achievement.In mathematics at least, standards, incentives,and decentralization were not sufficient togenerate performance gains.

Middle Schools Offera Mixed Story

Middle school students made modest gains onthe SAT-9. While eighth-graders made someprogress during the first three years, their scoresflattened in the fourth year, and their math andscience scores declined in year five. Mathematicsscores were especially low in the middle schoolsas few teachers had strong mathematicsbackgrounds and the mathematics curriculum wasnot well aligned with the SAT-9.

13BEST COPY AVAILABLE


FIGURE 2. PERCENT OF STUDENTS SCORING AT OR ABOVE BASICIN READING BY SCHOOL LEVEL, SAT-9, 1996-2000

Gum

70.0

60.0 _

LI 50.0LI1., 40.0

30.0

20.0

10.0

.01995-1996 1996-1997 1997-1998 1998-1999 1999-2000

40.4 48.7 54.4 55.1 57.7ES

59.9 65.1 70.4 71.3 NAK-8

43.3 50.5 55.0 58.5 55.8........m... MS

HS 25.6 34.0 33.6 37.0 40.7

Nevertheless, most middle school teachers feltthe Children Achieving reforms had positiveimpacts on their schools. In fact, they were morepositive about the reforms than their counterpartsat the elementary and secondary levels. Facedwith an onslaught of demands from the centraloffice, middle school staff often chose to work onthose pieces of the reform with which they weremost comfortable. So they focused on thecreation of small learning communities (SLCs), thedevelopment and implementation of thematiccurriculum, and test preparation. While wellintended, the thematic curricula developed bymiddle school staff often lacked rigor anddistracted them from developing more rigorousstandards-based curricula. As has been discov-ered by others, it proved difficult to build stronginterdisciplinary programs on a foundation ofweak disciplinary curriculum.

Student discipline was a serious problem inmiddle school classrooms, undermined attemptsto improve instruction, and contributed to highteacher turnover. This combination of poorstudent discipline and high teacher turnoverproduced chaos in some schools. In the highest

School Year

poverty middle schools, 46 percent of theteachers in the 1999-2000 school years were newto their schools in the last two years. In the lowestpoverty middle schools the figure was 34percent.'

High Schools Struggle

High school students were performing poorly ondistrict tests at the beginning of ChildrenAchieving in 1996 and were still doing poorly fiveyears later. Test scores were flat during the firstfew years of the reform and then edged upslightly in the last three years. High schoolteachers, facing huge gaps between the enteringachievement of their students and the district'sperformance expectations, also showed the leastsupport for the reform. The architects of thereform lost a rich opportunity to increase teacherbuy-in and improve implementation when theydecided to ignore the teacher networks that hadbeen nurtured during the previous reform era.Even highly committed high school teachers feltexcluded from the reform. Many high schoolslacked effective principals which only furtherimpeded the possibility of success. Academic

14 BEST COPY AVAILABLE5


FIGURE 3. PERCENT OF STUDENTS SCORING AT OR ABOVE BASICIN SCIENCE BY SCHOOL LEVEL, SAT-9, 1996-2000

OA ftOVA/

70.0

60.0

1; 50.0;' 40.0

30.0

-------±20.0

10.0

.01995-1996 1996-1997 1997-1998 1998-1999 1999-2000

37.1 46.4 36.6 52.2 61.5ES

42.2 51.9 43.2 59.3 NAK-8

18.1 23.7 18.8 30.2 27.4MS

' HS 4.9 8.0 5.2 12.0 19.1

School Year

departments were weak to begin with and wereeliminated in many high schools as school leadersfocused their resources on supporting SLCs. ButSLC coordinators were overwhelmed with admin-istration and disciplinary responsibilities, andcould not fill the leadership vacuum. Thus, therewas inadequate instructional leadership andteachers were left to cope as best they could.

The improvements in achievement in Philadelphiacan be explained largely by intensive test prepa-ration and increasing familiarity with the contentand format of the test. In fact, the pattern of earlygains in performance in the elementary andmiddle schools follows a pattern seen in otherlocalities when new tests are introduced: scoresare suppressed initially and then rise for two orthree years before leveling off. This pattern raisesquestions about the amount of change inteaching and learning that actually occurred inPhiladelphia.

6

Although the gains in student achievement weremodest, there was enormous celebration in thedistrict when the gains were announced each yearand a huge investment in telling this "goodnews." To show that progress was being made,the district created categories of proficient, basic,and below basic to differentiate levels of studentperformance. However, because the initial level ofstudent performance was so low, leaders dividedthe below-basic category into three sub-categories to make student progress visible. Itwas difficult for many civic leaders in Philadelphiato feel that substantial progress was being made,since even after five years, while some studentsprogressed, many remained in the below-basiccategory. In addition, civic leaders' expectationshad been raised and they expected progress tocome quickly and steadily. The reality of poorperformance belied the exuberant claims ofprogress coming from the central office andincreased skepticism about Children Achieving'spotential for success.



Strategies of Reform withinChildren Achieving

Eight strategies served as prime movers within Children Achieving's complicatedtheory of action. Examining, these strategies in retrospect, one can better under-stand the problems that arose and the disconnects with the realities in the schoolsthat surfaced as Children Achieving was being implemented. It also becomes easierto understand why its impact was limited. These eight key strategies were:

1. Fair FundingSeek increased state funding and a new state funding formula that was fair toPhiladelphia's schools.

2. StandardsSet high standards to outline the knowledge and skills that Philadelphiastudents should acquire.

3. AccountabilityCreate an accountability system to assess how schools perform annually andreward progress or sanction decline every two years.

4. DecentralizationMove decision-making back to schools and neighborhoods by organizingclusters of schools around high school feeder patterns, developing smalllearning communities within schools, and creating local school councils.

5. Leadership and SupportPrepare teachers and administrators to understand, support, and effectivelyimplement the reforms.

6. Better Coordination of ResourcesProvide students with the social services and supports they need to succeed inschool.

7. Civic and Parent EngagementBuild support from citizens for the Children Achieving reforms and involveparents as leaders and active participants in their children's schools.

8. Doing It All At OnceAddress all of the elements of Children Achieving at once in order to movethe system forward.

BEST COPY AVAILABLE 167


8 17


Strategy One: Fair Funding

entiir t re crm effort .2n i adei hia can be v. as a icukirisk by Superintendent j-iorn,...ck that he could secure additional funds

for the district in time o sustain the reforms before theAnnenberg Challenge grant ra ,0 out."

One tenet of the 10-point plan of ChildrenAchieving was to "ensure adequate financial andother resources, and use them effectively." Whilethis seemed like an obvious point, over thecourse of the reform it became a source of aggra-vation to some and a disappointment to many.The struggle between state and city leaders overthe costs of adequately financing the schools andthe state's responsibility for funding became aprimary point of contention and debate withinthe city and overwhelmed discussions about theprogress of instructional reforms.

When Children Achieving was launched in 1995,Philadelphia's population was shrinking, busines-ses were leaving the city, and the number of jobsin the city was declining. Consequently, the taxbase supporting city services, including theschools, was shrinking. And local taxes were high.Philadelphia residents were paying higherproperty tax rates than residents in surroundingcounties and they were paying a city wage tax ofnearly four percent. The wage tax was a disincen-tive to businesses that might consider moving intothe city and an obstacle to economic growth. Inthis economic environment, it was hard to increaselocal financial support for the public schools.

State support for the Philadelphia schools alsowas declining. The state had frozen its fundingformula in 1993 and from that time onward, stateaid to Philadelphia did not increase in responseto increases in enrollment. On a per-pupil basisadjusted for inflation, the value of state fundscoming to Philadelphia actually decreased by 5.9percent between 1993-1998.

As a consequence, Philadelphia's per-pupilexpenditures were significantly lower than thosein surrounding Pennsylvania school districts. Inthe five-county region around the city, somedistricts were spending twice as much per pupil

as Philadelphia. Average starting salaries forteachers in the suburbs were more than $3,500higher than those in Philadelphia and maximumsalaries were nearly $10,000 higher. As a conse-quence, there was a steady stream of teachersleaving the city for the suburbs.

So Philadelphia faced the task of educating thestate's neediest children with resources that mostsuburbanites would have regarded as whollyinadequate for the provision of good publicschools. The only solution to the fiscal problemsof the district was increased state aid, butpersuading the legislature to alter the schoolfunding formula and increasing state taxes was adifficult assignment in a large and conservativestate in which rural and suburban interests oftenclashed with those of the cities.

The entire reform effort in Philadelphia can beviewed as a calculated risk by SuperintendentHornbeck that he could secure additional fundsfor the district in time to sustain the reformsbefore the Annenberg Challenge grant ran out.He believed he could demonstrate the efficacy ofadditional funding by raising test scores andpersuading either the courts or the legislature thatfiscal reforms were needed to provide equalopportunity for Philadelphia's children and thatthey would result in improved performance.Children Achieving's leaders pursued multiple,overlapping strategies to generate the politicalwill to obtain increased city and state funding forPhiladelphia public schools. One strategy was togain the endorsement of the city's civic andbusiness leaders for the reform agenda throughtheir involvement in such groups as theAnnenberg Challenge coordinating and oversightcommittees that developed the initial work plans.A second was to show evidence of improvedstudent performance to prove that Children

18 BEST COPY AVAILABLE

9


Achieving was a reform worthy of an increasedinvestment of state and local dollars. A third wasto make a convincing case that the district wasbeing well managed by cutting costs anddocumenting what would be needed to maintainstudent progress. A fourth strategy was to renego-tiate the teachers' contract to give managementmore flexibility to address student needs andincrease labor productivity. A fifth strategy was tocatalyze and nurture grassroots organizing efforts,and develop a strong group of parents willing tofight for increased funding in the state capital. Thesixth strategy was to go to court.

Each year, Superintendent Hornbeck delivered abudget statement to Philadelphia's City Councilthat outlined the expenditures for the comingyear. With the budget documents for the 1997-1998 school year, district leaders distributed awhite paper titled, "Realities Converge: This Yearis Different." This paper set a new tone and raisedthe political stakes. The paper outlined three legalactions being taken against the state by thedistrict, the city, and other plaintiffs such as thePhiladelphia chapter of the National Associationfor the Advancement of Colored People. The firstwas a lawsuit filed in the PennsylvaniaCommonwealth Court that claimed the state hadviolated its obligation under the state constitutionto provide a "thorough and efficient" educationfor all children in Pennsylvania. The second was alawsuit filed in the United States District Court forthe Eastern Division of Pennsylvania charging thestate with violating federal civil-rights law byshifting funding away from Philadelphia, thedistrict with the largest concentration of minoritystudents in the Commonwealth and "therebyincreasing their concentration and isolation bothwithin their school district and among students inthe Commonwealth."' The third action was to filea brief in the State Supreme Court to support aprior Commonwealth Court decision that the stateshould pay the city extra funds to support court-mandated desegregation remedies.

The language in the white paper was strong."The School District does not take these actionslightly. Yet, Philadelphia's children deserve bettertreatment from the State...The State's continueddenial of its basic constitutional responsibility

10

simply leaves no alternative. The indisputable factis that the Commonwealth has denied its moraland constitutional mandate to provide a thoroughand efficient education to the City's schoolchildren..." In subsequent speeches, Hornbeckpursued these themes expressing considerablemoral outrage. His implicit charges of racisminflamed the governor and other state leaderswho found the claims outrageous. Battle lineswere drawn, and meaningful discussions betweendistrict and state leaders ceased.

The conflicts between state and city leaders overthe adequacy of the financing of the schools andwho should be responsible for fundingoverwhelmed discussions about the progress ofinstructional reform. Civic and business leaderswho were initially supportive of Children Achievingbecause of its strong accountability systembecame increasingly critical of Hornbeck'sconfrontational style which they saw as an inappro-priate strategy for securing public support or statecooperation.' They were not eager to confront agovernor to whom they were looking for economicsupport for business in the city, despite increasingevidence that Pennsylvania lagged far behindother states in its funding of public schools. (InJanuary 2001, Education Week gave Pennsylvaniaa grade of "D-" on funding equity in comparisonto other states.) Moreover, the leadership of thebusiness community was changing and some ofthe new leaders favored the governor's voucherproposal. In addition, as mentioned earlier, whilethey saw modest gains on student test scores,student performance remained extremely low inthe city and this seemed in conflict with the claimsmade by Hornbeck that student performance wasgreatly improved.

The fair funding cause, ardently led bySuperintendent Hornbeck, became a primary focusof the central staff and drained time and energyfrom the hard work of improving teaching andlearning. In many ways, the funding and fiscalissues were the Achilles Heel of the reform, thepoint of vulnerability that took attention away fromthe schools, and gave those opposed to ChildrenAchieving a way to reframe the issue from inade-quate financial support for public education toHornbeck's leadership style.

19


Strategy Two: Standards

nor z - -P13rs who little or no expcsure to ioc2; or lationaiitiatives promoting instructional reform or who doubted that theirstudents could achieve the standards, the curriculum frameworks

were not enough... Their curriciAim and ins or Agereheavily influenced by the district's standardizes test, the SAT-9.

In their classrooms, narrow test preparation prevailedover the standards."

The architects of Children Achieving believed thatadopting written standards defining whatstudents should know and be able to do wouldcreate a basis for assessing student performance;serve as a guide for teachers, administrators, andparents; and motivate all parties to improve theirwork. The creation of standards was a significantstep for the Philadelphia school district repre-senting a change from accepting any growth inachievement as adequate to striving for mastery.The new standards, partially based on thosedeveloped by national professional organizations,established goals for the skills and knowledgestudents should acquire by defining specificbenchmarks at grades 4, 8, and 10. But thestandards did not prescribe what material shouldbe taught or how it should be taught. This wasleft to the teachers in each school who wereexpected to establish a curriculum that wouldprepare students to reach the standards.

The process used to create the content standardswas inclusive and brought together teachers,administrators, parents, principals, and commu-nity members who served on committees thatmet over several months before reaching finalconsensus and ultimately presenting their workfor Board approval. As these groups workedtogether, they learned much about efforts acrossthe country to promote standards-based educa-tion and the implications of standards for schooland classroom practices. They becamecommitted to standards as an approach toimproving education, but the committees weredisbanded upon completion of the standardsdocuments and the district did not capitalize onthese groups' new knowledge and enthusiasm by

BEST COPY AVAILABLE

using them as advocates and leaders in thereform.

Standards were a major departure from theprevious administration's standardized curriculum.The previous district curriculum outlined thespecific topics to be taught at every grade level,but did not specify expectations for whatstudents should know and be able to do. Withthe adoption of standards, teachers were givenmore latitude about what to teach and how toteach it, but no latitude about the goals forstudent learning. However, teachers varied widelyin their response to the district standards.' Mostused the standards as they had the standardizedcurriculum. Typically, they used them as check-lists, checking off the topics covered, and sawtheir relevance for the content of their courses,but failed to see their implications for theirteaching practice or for student performance.

Part of the problem with the implementation ofstandards was timing. They were developed inthe spring of 1996 and distributed to teachersthat summer. The School Board adopted themthe following December. Since the new accounta-bility system was already in place, teachers wereexpected to implement the standards immedi-ately. Yet no time had been allowed for review ofthe standards, revision of curriculum, purchase ofnew instructional materials, or preparation ofstaff.

Most teachers found that the standards were notspecific enough to guide the development oflessons and asked for more direction. In thesecond year of the reform, this concern waswidespread. Teachers were frustrated by the lack

20

11


of time to review and discuss the standards andthe lack of specificity about what was to betaught at each grade level. They wanted morespecific examples and guidance so they couldunderstand how to put the standards intopractice in their classrooms. They needed help indetermining what materials were aligned with thestandards and time to make the necessary adjust-ments in curriculum.

In response to these concerns, district personneldeveloped curriculum frameworks, which offeredmore specific instructional activities and strategiesfor all subjects by grade level. The curriculumframeworks were intended to help teachersunderstand what they needed to do to have theirstudents achieve the new standards. Although theframeworks did not provide a curriculum, they dididentify constructivism as the pedagogical philos-ophy underlying the district standards.' Theyoffered examples of how to enact that philosophyby providing sample activities and model unitsthat addressed the standards and defined grade-specific skills. The frameworks did help someteachers move toward standards-based instruc-tion. However, many teachers complained thatthe curriculum frameworks did not provide themwith adequate guidance.

In fact, the frameworks did not offer recipes forsuccess. The recommended practices did notalways rest on solid empirical bases, but rather aphilosophy of instruction and a promise of better

12

outcomes. Moreover, they rested on tenuousassumptions about teachers' knowledge ofsubject matter and pedagogical skills. And theyassumed that teachers were able to manage theirclassrooms. Data from classroom observationsand teacher interviews indicated that competentteachers who held high expectations for childrenfound the frameworks to be useful tools. Theyhelped teachers develop curriculum units, lessonplans, and classroom-based assessments.However, for teachers who had had little or noexposure to local or national initiatives promotinginstructional reform or who doubted that theirstudents could achieve the standards, thecurriculum frameworks were not enough. Many ofthese teachers were overwhelmed by classroommanagement problems and by the complexity ofthe changes suggested by the frameworks. Theircurriculum and instruction were heavily influencedby the district's standardized test, the SAT-9. Intheir classrooms, narrow test preparation pre-vailed over the standards.

Bridging the distance between where teacherswere in their knowledge of subject matter,their use of instructional practices, and theirbeliefs about what students can learn (studentswho came with tremendous deficits) and wherethey needed to be, in order to help their studentsreach higher standards, proved to be a dauntingtask in Philadelphia, as it has across the country.

21


Strategy Three: Accountability.,n. an, eechers believed hay. the -f ancewas unfair and held them accountao e for factors cutsicia of their

catroi... A c.reater effort by district officials to educate teachers endthe public about the PRI and to respond to their criticism may have

heiped tv build more confidence in the system.

A good accountability system allows the public tosee the progress being made in schools andleads to accountability for district leaders as wellas teachers and students. It builds the capacity ofschools to become learning organizations byproviding multiple indicators that stimulate reflec-tion and examination of practice, and mid-coursecorrections. Children Achieving's leaders createdan accountability system based on thePerformance Responsibility Index (PRI), a measurethat included several indicators of performance.The index included math, reading, and sciencescores of students on the SAT-9; promotions tothe next grade level in elementary and middleschools; the proportion of ninth-grade studentsgraduating from high school in four years; andstudent and teacher attendance. The systemoperated on a two-year cycle, and the PRIprovided each school with biennial performancetargets. The goal was to motivate school staff totake actions that would improve their results onthese indicators and raise their PRI score. Schoolswere rewarded or sanctioned based on theirprogress toward two accountability goals:meeting or surpassing their PRI performancetarget and decreasing the number of studentsscoring at the lowest levels on the SAT-9 inreading, math, and science.

Admirably, Philadelphia developed one of themost inclusive testing programs in the country.The accountability system offered schools strongincentives for increasing student participation inthe testing program. By including all students,even those who were lowest performing andthose with learning difficulties and languagedifferences, the district risked lowering its overallperformance. The fact that test scores increasedin spite of this increased inclusion was a notableachievement.

The introduction of the accountability systemlaunched a major cultural change in the district. Itfocused everyone's attention on student achieve-ment and school improvement. It providedincentives for school staff to seek out and adoptinnovations that they might have ignored orresisted otherwise. And it initially satisfiedbusiness, government, and civic leaders' insis-tence that the district make data about studentperformance accessible to the public.

But, the PRI had some serious flaws. The indexrelied on a cross-sectional look at studentachievement, focusing on the performancechanges in specific grades rather than looking atstudent growth year to year. Additionally, the PRIrelied on an off-the-shelf commercially-producedtest, the SAT-9, that was loosely aligned withdistrict standards in all subject areas. Using theSAT-9 as the primary measure of student perform-ance instead of looking at multiple measures ofstudent performance that were closely aligned tothe content standards undermined the PRI's credi-bility as a measure of student performance andfocused teachers' attention on preparing studentsfor a standardized test rather than helping themreach the standards.

And since the PRI did not take into account thebackgrounds of students or their entering level ofachievement, schools that had made substantialimprovement prior to 1996 found themselves at arelative disadvantage compared to schools thathad made little progress prior to the introductionof the accountability system. The schools in whichstudents were already making gains in learninghad to reach for even higher levels of perform-ance after 1996 whereas schools with very lowperformance in 1996 had improvement targetsthat were more easily attained.

13BEST COPY AVAILABLE 2 2


A third and more serious problem had to do withthe metric used to determine student progress.Students' actual test scores were not used in thePRI. Instead, students were placed in one of sixcategories of performance (See Figure 4) basedon their scores and only movement from a lowercategory to a higher one counted as a gain forthe school. It was the change in the distributionof students in these categories that countednot the actual change in test scores. Thesecategories were arbitrarily defined and little wasknown about the amount of learning needed toadvance from one level to the next or the relativedifficulty of moving a student from Below Basic 1to Below Basic 2 versus moving from Basic toProficient. Experience in other locales such asKentucky suggests that these advances posedquite different challenges and that students couldmake significant gains but not advance into thenext category. Schools received no credit for theincreased learning of a student who advancedfrom the lowest score in the Basic category to thehighest because actual test scores were not usedin calculating the PRI.

In addition, the index was complicated and noteasily understood by teachers, parents, or thepublic. Some people were able to articulate thecomponents of the index, but the complicatedcalculations used to create the final numbersmade even supporters of accountability skepticalabout the PRI's accuracy, purpose, and impact. Itcreated a false precision that many educatorsmistrusted. And their instinctive mistrust was wellfounded. There was significant measurementerror in the indicators used. The increase on thePRI required to reach the improvement targets setfor the schools were often smaller than theamount of statistical error present in the calcula-tions of the index. To put it simply, schools mighteasily fail to reach their targets or surpass thembecause of statistical flukes. This created a senseof arbitrariness that is not acceptable in systemsthat affect people's lives and professional reputa-tions.

The calculation of the staff attendance factor inthe PRI was also perceived by many teachers aspunitive because long-term illnesses and mater-

14

23

nity leaves were treated as unacceptable absen-teeism.

Philadelphia's teachers were not opposed toaccountability. Teacher surveys indicated that 63percent of teachers believed the PRI had thepotential to benefit students. However, manyteachers simultaneously believed that the PRI wasunfair and held them accountable for factorsoutside of their control. They felt students' livesoutside the classroom had more impact onachievement than they did. A greater effort bydistrict officials to educate teachers and thepublic about the PRI and to respond to their criti-cisms may have helped to build more confidencein the system and greater collective responsibilityfor the results. The district did set up anindependent panel of experts chaired by AndrewPorter of the University of Wisconsin-Madison.This panel made numerous recommendations forstudying the PRI and improving it. While thedistrict was responsive, these changes came toolate to restore teacher confidence in a systemthat had enormous influence on their work andtheir reputations.

There was also a disjuncture between reformgoals and the accountability timetable. The two-year timeframe used by the PRI was too short forschools to implement significant reforms incurriculum and instruction. This added to thefrustration among teachers and principals whowere struggling to make improvements. Considerwhat principals and teachers faced. They wouldreceive their PRI scores for the first year of thetwo-year PRI cycle at the beginning of a schoolyear and they then had only seven months beforestudents would be tested again. Their budgetswere set; the summer professional developmentwas over, so there was little they could do if theyneeded to make a large gain to reach their PRItarget except to directly prepare students for thetest. Test preparation was a rational responseunder the circumstances, and it was the typicalresponse in Philadelphia.

Intensive qualitative research in a sample ofdistrict schools identified schools that werepursuing effective strategies for improvinginstruction, but failed to reach their PRI targets.


FIGURE 4. PERFORMANCE LEVELS ON THE SAT-9Categories ofPerformance

Advanced

Proficient

Basic

Definition

Superior performance beyond grade-level mastery.

Solid performance, meaning students are ready for the next grade.

Partial mastery of the knowledge and skills that are fundamental forsatisfactory work.

Below Basic III Inadequate mastery.

Below Basic II Little mastery.

Below Basic I Very little mastery.

Source: T. Luhm, E. Foley, and T. Corcoran, The accountability system: Defining responsibility for student achievement.Philadelphia: The Children Achieving Challenge, 1998.

These schools were penalized even though theywere on observable, defensible paths to improve-ment. The PRI underestimated the time neededto bring about meaningful change and createdunrealistic expectations and burdens for staff.Rather than creating school cultures that encour-aged organizational learning and internal capacitybuilding, the PRI was a punitive measure thathighlighted school failures. Focusing on prepara-tion for the tests was perceived as the only sureway to avoid embarrassment and sanctions.

Only schools identified as making "low progress"received feedback on the quality of their instruc-tional efforts or were designated as needing theSchool Support Process, a comprehensive inter-

vention that brought additional funds and assis-tance to schools. Reviewing only those schoolsthat failed to meet their targets reinforced theidea that feedback was a form of punishment andthat reflection and change were only necessarywhen there was measurable failure. Rather thanpromote a districtwide culture of continuousprogress, the PRI encouraged many educators toseek quick fixes and adopt coping strategies. TheSchool Support Process could have been an aideto improvement and did, in fact, help someschools improve in real ways, but this process wasavoided rather than embraced because it wasseen as punitive and a symbol of failure.

3EST COPY AVAILABLE

15

24


Strategy Four: Decentralization

''Principa,s and teachers were tiT,',!CI they had more authority and choice,but they didn't feel more empowered. The steady flow of mandates, in

fact, reduced :heir options. School leaders were overwhelmed bymandates and felt Children Achieving was top-down and orescriptive.'

The designers of Children Achieving intended togive principals and teachers more autonomy tomake decisions about curriculum and instruction.This central tenet was articulated early on byHornbeck: "Those who sit closest to the actionare in the best position to decide what mix ofresources...will most effectively accomplish thegoal of raising student achievement."' The radicaldecentralization proposed by the superintendentseemed to be a significant operational change forboth central office and school staff who wereaccustomed to a top-down, prescriptive districtculture.

The reform architects established new structuresto give schools more authority over decision-making. These new structures included clusters,local school councils, and small learning commu-nities. Newly established cluster offices workingwith one comprehensive high school and itsfeeder elementary and middle schools wereexpected to provide focus for improvement initia-tives in the schools, mobilize resources to supportimprovement, build a responsive relationship withneighborhood organizations and communitymembers, provide professional development,coordinate social services, and strengthen K-12curriculum, instruction, and assessment. Localschool councils were supposed to activelyengage parents in the decision-making of theschools, moving them into more influential rolesthan the ones usually offered by home and schoolassociations. Small learning communities, sub-units of schools with typically fewer than 400students across grade levels, were meant toimprove the conditions of teaching and learningand to motivate teachers and students bystrengthening relationships and personalizingschooling.

Central OfficeFrom the beginning, central office staff membersseemed unclear about how to operationalizedecentralization and ambivalent about how itwould impact their roles and responsibilities. TheStrategic Action Design stated that the newfunctions of the central office were to "setstandards, assess progress, monitor for equity,and act as a guide and provider of resources andsupport,"8 but these tasks were complex andchallenging to a central office with limitedcapacity. Central office staff struggled to figureout how to be both "hands-off" and more helpfulto schools. They found the capacity-building roleto be in conflict with the core value of schoolautonomy. Initially, some central office staffmembers interpreted decentralization so literallythat almost every central office action proposedbecame a topic of contention.' There were exten-sive debates about whether to develop a coregroup of consultants to help the schools, whetherto organize the content standards by grade level,how to define best practice, how to organizeprofessional development, and so on.Decentralization posed challenges for thosecharged with both making it happen and withimproving system performance. At times the twogoals seemed in conflict. (See Figure 5 for adiagram of the core beliefs and values and theireffects on centralization and decentralization).

It also became increasingly apparent that decen-tralization placed too heavy a burden on manyschools. School staffs were unready to assume somuch responsibility. Decentralization assumedthat school staff had the know-how and theresources to improve student achievement, butsimply lacked the opportunity or authority tomake the decisions that mattered. But, inPhiladelphia, school staffs often did not have the


17


b.

SCHOOLAUTONOMY

PRIMACY OFRESULTS

-

1;,:

SET STANDARDS

PROVIDE RESOURCES & SUPPORT

HOLD SCHOOLS ACCOUNTABLE

MONTIOR EQUITY

EQUITY ISPARAMOUNT

INCENTIVES ARE DO IT ALLNECESSARY AT ONCE

CORE BELIEFS AND VALUES

capacity to manage the new demands of decen-tralization at the same time that they were tryingto make the changes in classroom practicerequired by standards-based curriculum andinstruction. They were hampered by shortages oftime, resources, and direction. In addition, neithercentral office nor school staffs had access toprofessional development that might preparethem to carry out their new responsibilities. Thecentral philosophy was clearly stated, but thecapacity to put it into practice often was lacking.

A turning point came in 1998 when reformleaders viewed with dismay disaggregated districttest data that showed certain groups of studentswere being left behind. For example, in 1998, thefour-year graduation rate was almost 30 pointslower for students in families on welfare than forall students. Central office staff were concernedthat the reform's equity objectives were beingneglected. These data convinced central officepersonnel that they had to take a more prescrip-tive approach. They concluded that they had tosend stronger messages to schools about whatand how to teach.'18

Cluster Offices

The 22 clusters varied considerably in theircapacity to guide school improvement. Resourceinequities contributed to this variation. The firstsix clusters formed in the 1995-1996 school yearreceived significantly more funding for severalyears than the 16 created in the fall of 1996. Therationale was that the original six were funded atthe level required to be effective and it wasimportant to demonstrate this to the state as partof the district's argument for increased funding. Inaddition, the cluster leaders varied in their abilityto raise external funding. As a consequence, theper-pupil expenditures for cluster staff andservices ranged from $81 to $475 in 1996-1997.Clusters also varied considerably in size, staffexperience, demographic characteristics, and theachievement levels of their student populations.

Clusters were expected to provide and brokerprofessional development through the Teachingand Learning Network (TLN) which was part ofthe Office of Leadership and Learning, theprofessional development arm of the district. The

2 6 3EST COPY AVAILABLE


TLN coordinators and facilitators were based incluster offices so that they could serve the needsof schools and teachers. However, the district'sambivalence about decentralization was apparentas TLN staff also reported to central administra-tion. TLN staff were expected to offer workshopsto help teachers understand and implement thereforms. In fact, much of their time was devotedto communicating new policies and initiativesmandated by central office, causing one TLNcoordinator to declare in 1999, "I'm not rollingout one more reform. Enough is enough." Overtime, TLN staff became somewhat more strategicabout allocating their time. In some clusters theyspent substantial time in elementary schoolsfocusing on early literacy initiatives, whichcontributed to elementary school students'achievement gains. In contrast, their work insecondary schools was often spent orientingteachers to the SAT-9, providing coaching on testpreparation materials and activities, andsupporting the many new and inexperiencedteachers."

Differences in the abilities and strategies of thecluster leaders led to variation in their clusters'success at rallying schools to a common purposeand providing services that met the needs ofschools' staff, especially principals. About a third ofthe cluster leaders defined their role as serviceproviders, supporting school initiatives. About halfdefined their role more proactively, seeingthemselves as catalysts and mobilizers of reform. Afew saw their role as line officers in the districtbureaucracy, serving traditional administrativefunctions. Indeed, central administrative leadersgenerated confusing messages about the roles ofclusters and cluster leaders. One debate thatraged during the first years of Children Achievingwas whether cluster leaders ought to have ratingauthority over principals. Initially, reform leadersbelieved that rating authority would interfere withcluster leaders' ability to coach and mentor princi-pals. But some cluster leaders were adamant thatthey needed rating authority in order to exercisethe influence over principals they considerednecessary to accomplish school improvement.When rating authority was granted, it reinforcedthe message that the cluster was just another rung

in the bureaucratic hierarchy. Interview datashowed that school staff generally saw the clusteroffice as just another form of centralized authority,a way station in the top-down dissemination ofinformation and program mandates.

The most successful cluster leaders were able toleverage funds and resources to provide moreprofessional development for teachers, moretechnology, more instructional materials, andeven additional staff. They kept the focus oninstructional improvement and supported theirprincipals while acting as buffers between thedemands of the central office and the schools.Professional development in these effectiveclusters focused on improving instruction.

One particular cluster exemplifies how this combi-nation of actions affected the progress of itsstudents. From the beginning of his tenure, thecluster leader worked with principals to establishan instructional priority for the cluster and totarget cluster staff and resources to support thatpriority. The building administrators believed thatliteracy ought to be the focus. They were willingto turn back some of their school funds to thecluster so that additional staff could be hired ascoaches for the cluster's teachers. Cluster staffmembers were chosen for their knowledge andexperience in early literacy and supportedadoption of a literature-based approach and astrong phonics program in the schools. Thiscluster's success in teaching reading in the earlygrades eventually captured the attention ofdistrict leaders who subsequently adopted it as asystem-wide program of improvement.Unfortunately, this was a rare instance of strate-gies demonstrated as successful in one clusterbeing replicated in the others.

In general, most cluster offices lacked clear,coherent strategies for instructional improvementand the necessary resources and staffing. Manycluster leaders were unable to leverage additionalfunding and resources to augment the work ofprincipals and teachers. As new mandates wererolled out by the central office, the cluster staffbecame the enforcers of these mandates ratherthan sources of support for implementing thereforms. The work of the cluster offices was

27

19


negatively affected by the sheer number andscope of district initiatives and directives, by staffturnover, by uneven and inadequate resources, bythe resistance of some teachers to change or toparticipate in uncompensated professional devel-opment, and by the low morale and turnoveramong principals who felt ignored and malignedby district and cluster leaders.'

Local School Councils

Local school councils built upon a school-basedmanagement/shared decision-making initiativelaunched by Hornbeck's predecessor, Dr.Constance Clayton, in the late 1980s. That effortfocused on giving more autonomy to principalsand teachers in budget and staffing decisions,but it stalled due to resistance from both centraladministration and the Philadelphia Federation ofTeachers (PFT).13 Under Children Achieving, localschool councils were intended to reinvigorate theoriginal initiative and take it one step further byincluding parents as full partners in the reform.Council members were supposed to make policydecisions at the local school level includingdecisions on the budget, external resources,safety and security, and facilities management."Early in the reform, the PFT, however, objected tothe implementation of school councils asproposed by the district and this resulted in ajoint PFT-District agreement that limited the juris-diction of local school councils to discipline issuessuch as suspension and expulsion and schoolsafety. In addition, the agreement changed thecouncil membership to require that teachers havea majority instead of the proposed 50-50membership.

The district established a stiff criterion that 35percent of all eligible households had to partici-pate in the election of the parent representativesin order for a school council to be certified andhave full authority. This was a formidable obstaclefor most schools and, in the end, many councilswere unable to obtain official certification. Inaddition, the district set broad guidelines forcouncils, but offered only limited training tocouncil members on how to carry them out. Thiswas a problem as most council members lackedexperience for their new roles. For many parents

20

unused to being active in schools, the new rolewas especially challenging and intimidating. Thesuccess of local school councils rested heavily onthe willingness and ability of principals to leadand guide them, but school councils were threat-ening to many teachers and principals since theyimplied a new balance of power that wasunfamiliar and unsettling. The reform architectsunderestimated the amount of support parents,teachers, and principals needed to make stronglocal school councils a reality. In the end, mostschools never formed certified local schoolcouncils and only a few councils exercised signifi-cant influence over their schools.

Small Learning Communities

Like local school councils, SLCs were the exten-sion of a previous reform effort. In 1988, the PewCharitable Trusts provided a $13 million grant forthe restructuring of neighborhood comprehensivehigh schools into smaller units that would providemore intimate environments for teaching andlearning. By the mid-1990s, this reform initiativehad stalled, but under Children Achieving, thedistrict gave it new life and mandated that allschools elementary, middle, and high schools

be divided into SLCs. The focus of SLCs was tobe the improvement of teaching and learning,although from the beginning there was consider-able ambiguity about the authority of SLCs overcurriculum, scheduling, student assignment, andother key decisions affecting teaching.

Analysis of CPRE/RFA survey data generated twoimportant findings about SLCs in elementaryschools: (1) well-implemented SLCs werepositively associated with a school climateconducive to effective teaching, and (2) SLCs werepositively associated with teachers' perceptions ofthe strength of their professional community (ameasure of teacher collaboration and sharedresponsibility for student learning). Because agood school climate and strong professionalcommunity were also associated with increases instudent achievement, these findings suggest thatSLCs made at least indirect contributions topositive learning outcomes for young people.Unfortunately, similar analyses could not beconducted for middle and high schools. However,

28


qualitative data collected in a sample of schoolsindicates that SLCs contributed to safer, moreorderly school environments in secondary schools.

Observations in classrooms and at small learningcommunity meetings and interviews with staff andstudents indicate that the impact of SLCs onschool climate was stronger than on curriculumand instruction. There were several reasons forthis. Having SLCs take responsibility for studentdiscipline contributed to a sense of greater orderand improved communication, but focusing onthese responsibilities too often overshadowed theintended instructional purposes of these neworganizations. Student discipline often becamethe primary responsibility of small learningcommunity coordinators, particularly in middleand high schools. This meant that they had littletime to devote to instructional tasks. Second,SLCs were required to adopt a thematic focus.Hastily adopted and underdeveloped themes didnot produce rigorous curricula and the result wasoften an emphasis on isolated events such asassemblies and field trips. These activitiesexposed students to a theme and built theidentity of the small learning community, butdiluted potentially powerful interdisciplinaryconnections. Third, at the middle and secondaryschool levels, the small learning communitycoordinators could not be expected to lead orjudge the quality of teaching and curriculum workin all of the core subjects. At best, they hadexpertise in one subject. With the elimination ofdepartments, one or two teachers were left ontheir own to develop curriculum in each subjectand they seldom received feedback on theirteaching from an observer qualified in their field.In the middle schools where there were manyfirst- and second-year teachers, this lack ofsubject-matter guidance contributed to a chaoticteaching situation. Teachers at all levels reportedthat they had more frequent informal discussionsabout students with their small learning commu-nity colleagues than they had had prior to thecreation of SLCs. This provided the opportunityto design instruction to meet the individualneeds, abilities, and learning styles of students.But these discussions typically focused on studentbehavior and seldom developed into the system-

atic examinations of students' work needed tohelp students ratchet up their performance.

Interviews conducted with teachers in the middleand high schools and data from the PhiladelphiaEducation Longitudinal Study (PELS) revealed thatthe creation of SLCs also generated newinequities in some schools. When teachers werepermitted to form these communities voluntarily,the result was often an inequitable distribution ofteacher talent across the SLCs. A related problemarose from rostering students for most of theirclasses in the SLCs which meant that not allstudents had access to the full curriculum.Teachers of specialized subjects such as foreignlanguages, and higher mathematics and sciencecourses were not available to all students if thoseteachers were assigned to a small learningcommunity. And data from PELS indicated thatstudents who had performed poorly in middleschools were less likely to select their high schoolsmall learning community which meant that theywere more frequently placed in the SLCs thatoffered less rigorous curricula.' In general, thequality of curriculum offerings varied across theSLCs and the elimination of subject-matterdepartments meant that there were no forums forquality control of curricula in the core subjects.

The stated purposes of SLCs were to strengthenteaching and learning and create stronger bondsand better understanding among teachers andstudents, but SLCs were assigned many adminis-trative and managerial functions. SLC coordinatorswere asked to ensure compliance with mandatesand did not have the leadership, skills, knowledge,time, or resources needed to help teachers makedeep changes in teaching and learning.Developing strong SLCs required more time andstronger incentives than were provided. SLCsneeded more budgetary authority and moreconsistent and larger blocks of time to meet andplan. Small learning community coordinatorsneeded help with discipline and administrativefunctions so that they could devote time andenergy to instruction. And departments or subject-matter networks were needed across SLCs toguide curriculum development, provide qualitycontrol, and ensure equal access to the curriculum.

2921


Clusters, local school councils, and SLCs wereintended to improve the quality of schools buteach ran into significant problems that preventedit from being effective. in the majority of schools.Capacity, acceptance, and authority were thecommon problems. There was also too littleprofessional development to prepare participantsto carry out their new roles. Principals, in partic-ular, were ill prepared to work with local schoolcouncils and SLCs. Accustomed to being incharge, the vast majority of principals wereunable or unwilling to consult and collaboratewith teachers and parents. In addition, resourceswere limited and the complicated and constantflow of district mandates imposed enormousburdens on those responsible for leading the newstructures. They struggled to meet the newdemands, often losing sight of their centralpurpose in the process. Finally, uncertainty andambiguity surrounded the decentralization initia-tives; there was insufficient specification of rolesand relationships and in the face of such confu-sion, people often fell back on what was familiarand comfortable.

22

As district leaders retracted from decentralizationand began to give more direction to school staffs,there was no articulation of a policy change. Theresult was a disconnect between what wasactually happening and the rhetoric from the top.In reality, as the reform progressed, schools lostautonomy as new responsibilities and districtmandates piled up. They also were under tremen-dous pressure to improve student achievement.But district leaders continued to frame the reformin terms of decentralization. Principals andteachers were told they had more authority andchoice, but they didn't feel more empowered.The steady flow of mandates, in fact, reducedtheir options. School leaders were overwhelmedby mandates and felt Children Achieving was top-down and prescriptive. The Philadelphiaexperience does not clarify the ongoing debateover the merits of centralized versus decentral-ized strategies of school improvement or resolvequestions about what powers and responsibilitiesshould rest with the district and what should bedelegated with schools. In the end, Philadelphiawas a confused and muddled amalgam of thetwo approaches.

30


Strategy Five: Leadership and Supportfor Teachers and Principals

'The Jack ot support and respect for prinvpais was feriousChildren Achieving and contributed to mate j'ailure to bring

deep and lasting changes to the district's classrooms."

Adopting standards-based curriculum, instruction,and assessment measures requires deep changesin teaching that experience and research suggestonly occur over extended periods of time andwith intensive support. Under Children Achieving,there was a dramatic increase in professionaldevelopment opportunities for teachers and therichness and depth of these opportunitiesimproved over the course of the reform.Unfortunately, the vast majority of initial offeringsfocused on compliance with mandates, and didnot provide teachers with the subject-matterknowledge or curricular guidance they needed tohelp their students meet the standards. Anexception was the professional developmentoffered by the Urban Systemic Initiative in mathe-matics and science which will be discussed below.As a consequence, these experiences had littleimpact on classroom instruction. But laterprograms, such as the district's successful summercontent institutes, proved more effective inhelping teachers improve their instructional skills.The content institutes built upon what had beenlearned in the Urban Systemic Initiative and manyteachers reported that these professional devel-opment programs deepened their understandingof subject matter and led to changes in theirteaching. Professional development for otherstaff, however, was quite limited. This was particu-larly true for principals who faced newresponsibilities but were offered little support orpreparation.

The increase in professional development oppor-tunities for teachers in Philadelphia was asignificant accomplishment. For years, profes-sional development had been a low priority andteachers typically attended a few sessions eachyear that were usually unrelated to instruction.Under Children Achieving expectations changed.Teachers were expected to improve their practice

and the district was expected to offer programsworthy of their time and attention. However,Children Achieving had many components andmost required teachers to acquire new knowl-edge and skills. The district initially spread itsprofessional development resources across theseinitiatives and across the 22 clusters. The resultwas a lot of activity but much of it was superficial,providing information about mandates and proce-dures rather than opportunities to examine andimprove classroom practice.

Thus, in spite of the increase in professionaldevelopment opportunities, many teachers werepoorly prepared to implement standards-basedinstruction and many held beliefs that rancontrary to it. The professional developmentopportunities should have provided teachers withtools and instructional materials that were alignedwith the standards. In classrooms in which goodinstruction was observed, the teachers were ofteninvolved in sustained professional developmentefforts and had access to high-quality curricularmaterials. Many of these teachers were involvedwith the Urban Systemic Initiative or with well-designed literacy initiatives. In both cases, theprofessional development was focused andextended over time, and supported with goodcurriculum materials.

Ongoing, focused professional development ishard to achieve without support and leadershipfrom principals. Yet few Philadelphia principalssaw themselves as instructional leaders or sawprofessional development as part of their jobsand the reform leaders did not seem to under-stand how this limited the possibilities ofaccomplishing the instructional changes theywere seeking. Instead of preparing principals tobe instructional leaders, central office staff justdemanded more of them. Nowhere in the


23


rhetoric or the design of Children Achieving wasthe essential role of principals as site-basedinstructional leaders recognized or developed.'Principals felt disempowered and disrespected.They received little or no professional develop-ment to help them deal with their newresponsibilities and the demands from centraloffice staff. They were exposed to new pressuresfrom the public release of the PRI scores for theirschools. Public and civic leaders could see whichschools were "working" and which were not andthat burden fell heavily on principals. Morale waslow and the turnover rate high. It is difficult, if notimpossible, to sustain a reform agenda withoutstable leadership in the schools. The lack ofsupport and respect for principals was a seriousweakness in Children Achieving and contributedto its ultimate failure to bring deep and lastingchanges to the district's classrooms.

Central office staff itself struggled to manage theimplementation of the reforms and juggle thedemands of the 10-point plan. As Foley" noted,

24

"Most of the Superintendent's inner circle wereknowledgeable and thoughtful about standards-based, systemic reform, but many other staff atthe central office did not have a clear under-standing of these areas." Shifting central officepersonnel to the cluster offices meant fewerpeople doing more work. Central staff turnoverwas also high, and new people stepped in andlearned as they worked. Little attention was paidto their development needs.

Instead of building the capacity for principals,cluster leaders, small learning community coordi-nators, and central office staff, the leaders ofChildren Achieving just kept asking them to domore. They did not include school leaders inplanning and development and neglected devel-opment of the workforce until it was too late tomake a difference. In the end, a demoralizedschool leadership was disengaged from ChildrenAchieving, disenfranchised from the process, andangry and resentful.

32


Strategy Six: Better Coordination of Resources

"'adequate staffing and tr-aining tFamily Resource Network as budget cutspositions. This added to the perceptions o

they were not being supported in theirhealth and school safety, which they bet:

ability to do their foOs w

efFectiven o thehe a irninarton of mEiny

achers end orincipa!s thatand in areas, such ased critical to their

Children Achieving was intended to improve thequality of support services available to students,helping students and families gain access to abroad range of community services, and coordi-nating activities of service providers. The ChildrenAchieving reformers understood the negativeeffects of poverty on children and on their capacityto learn. As Hornbeck noted in 1994: "Communityservices and supports can make the differencebetween success and failure. Children who areunhealthy, hungry, abused, ill-housed, ill-clothed,or otherwise face the kinds of problems outsidethe school born of poverty will not achieve at highlevels. Therefore it is imperative that initiatives bedramatically expanded to provide the necessaryservices and supports to reduce the impact ofthese major barriers to learning."'

However, resources were scarce and services limitedso district leaders focused on advocacy and coordi-nation with community groups and governmentagencies to broaden and deepen their impact. TheFamily Resource Network (FRN) was the districtoffice assigned this responsibility." FRN staff did notsee themselves as direct service providers, butrather as advocates, working with civic and commu-nity agencies to coordinate youth and familyservices. In addition, they were supposed to consoli-date services within the district itself. The FRNmoved responsibility for student support servicesfrom the central office to cluster offices by providingeach cluster with an FRN coordinator.

In 1996-1997, the FRN identified four goals: toimprove student attendance, student health,school safety, and family and community involve-ment." While central office and cluster staffunderstood how the FRN intended to accomplishthese goals, teachers and principals did not.However, 92 percent of teachers expressed the

BEST COPY AVAILABLE

belief that the FRN could help their students.21Communications about the role of the FRN andresponsibilities were inadequate and led to confu-sion about its purpose. The FRN was notdesigned to be responsive to the daily problemsthat surfaced in schools, but rather to buildcapacity in clusters and schools to make betteruse of existing resources and to work withcommunity and civic organizations.

Leaders of some community and civic organiza-tions that worked with the FRN praised theirefforts. They felt they had a focused point ofcontact within the district for the first time, butothers expressed wariness about working with adistrict bureaucracy that seemed confused aboutwhether its role was delivery or coordination ofservices. This role confusion about delivery versuscoordination was found within the FRN staffthemselves as they struggled to decide whatshould be done by the district and what could behandled best by outside agencies.

The significant improvements in student atten-dance seen in Philadelphia were partially due tothe efforts of the FRN, such as the installation oftruancy courts in schools to address chronic tardi-ness and absence. There were also increases inthe numbers of students who received immuniza-tions, health insurance, and eye examinations.

However, inadequate funding undermined thework. Inadequate staffing and training limited theeffectiveness of the FRN as budget cuts forcedthe elimination of many positions. This added tothe perceptions of teachers and principals thatthey were not being supported in their work andin areas, such as health and school safety, whichthey believed critical to their ability to do theirjobs well.

33 25


Strategy Seven: Civic and Parent Engagement

"As distric leaders became fri adamant in Chair a aaments-fair funding, supportive civic leaders,, especially, were cten t Ig:C voice

their cz.--;ncerns or raise questions. They feared giving ammunition toQoiitical leaders who wanted to discredit the reform

the Superintendent "

Given the financial crisis threatening the district,the architects of Children Achieving understoodthat they needed the support of corporate andcivic leaders, community residents, and parents.And initially, the Children Achieving reform planwas championed by the business and foundationcommunities, civic leaders, and the mayor. Thereform's mandate to change all parts of thesystem all at once made sense to them as theywere eager to get the job done. In addition,public reporting of school-by-school test data anddata disaggregated by race/ethnicity and incomelevel helped Philadelphians understand thegravity of the situation in their public schools.Extensive media coverage and debate over thedesign of Children Achieving and its new superin-tendent made public education a front-page storyin a town where, as a grassroots leader describedit, "there had been apathy and little discussion."'

However, interviews conducted with more than 40of Philadelphia's leaders revealed that, in fact,they saw standards, accountability, and decentral-ization as common sense ideas rather than as atheory of education improvement that wouldneed ongoing assessment and revision. They alsoheld differing ideas about what was wrong withthe city's public schools and competing interpre-tations of whether Children Achieving producedsignificant improvement in the schools.

Business leaders believed that the city's economicfuture rested on the quality of its schools andmany linked the city's economic decline withinadequate public schools. They judged theschool system inadequate because of its inabilityto hold middle-class families in the city. WhenHornbeck approached Greater Philadelphia First,a not-for-profit association of the Philadelphiaregion's largest employers, asking them to house

the Annenberg Children Achieving Challenge, itsboard of directors agreed to take on the respon-sibility, hoping it would give them greaterinfluence with district leaders. Over the course ofthe reform, they were informed about theprogress of Children Achieving on a regular basis,and some board members became personallyengaged and helped raise the matching dollarsrequired by the Annenberg Challenge grant.

But economic circumstances changed and somebusiness leaders became disillusioned with thereform approach and its leader, SuperintendentHornbeck. Several factors contributed to thedecline in support from the business community.First, because Annenberg Challenge funds wereused to build a district infrastructure to supportreform, Greater Philadelphia First leaders couldnot point to specific ways in which the grantfunds made a difference in schools. They alsocomplained that they did not receive adequate oraccurate reporting from the district on how thefunds were spent. Some worried that too much ofthe money was being spent on administration.Additionally, some were frustrated by the failureof reform leaders to outsource more of thedistrict's service contracts to private companies.All of these factors fueled their perception thatthe district was inefficient and not a good stewardof public funds. A turning point in the relationshipbetween the superintendent and the businesscommunity came in 1998, following the negotia-tion of a new contract with the PhiladelphiaFederation of Teachers. Business leaders felt thatHornbeck failed to secure needed changes in theteachers' contract and that this seriously limitedthe possibility of improving performance. Theybegan to doubt the effectiveness and potential ofChildren Achieving.

3 4 BEST COPY AVAILABLE

27


In the meantime, board leadership at GreaterPhiladelphia First changed and the new leaderstook a more conservative stance. As local corpo-rations (e.g., Core States Bank, Scott Paper,SmithKline, and others) were taken over bycompanies based outside of Philadelphia, thecommitment of corporate leaders to the well-being of the city seemed more tentative. Inaddition, Greater Philadelphia First expanded itsmembership base to include regional corpora-tions which meant that the organizationrepresented suburban as well as urban interests.By June 2000, when Superintendent Hornbeckresigned, only four of Greater Philadelphia First'sfounding 23 Chief Executive Officers remained.The new leaders were strong supporters of theArchdiocese system and were attracted toGovernor Ridge's proposal for vouchers whichthey believed would be good for businessbecause it might stem the flow of middle- andworking-class families from the city, therebybuoying the local economy and strengthening thequality of the workforce. They tried to persuadeSuperintendent Hornbeck to work with thegovernor on this issue, but he would not. In 1999,members of Greater Philadelphia First stood withthe governor when he proposed a voucher planfor the state.' The message was clear: thebusiness community, once supportive ofHornbeck's initiative, had lost faith in the localreform effort and lost confidence in its leader.

Civic and grassroots leaders offered the strongestpraise for the Children Achieving reform plan.They expressed pride and gratitude thatPhiladelphia had a plan something they feltother cities lacked. Many saw economic inequalityand racial discrimination as the most significantproblems confronting schools (and the broadersociety) and called for a redistribution ofresources that would provide students fromgroups who had traditionally performed poorlywith better-designed and more intensive supportsto ensure their school success. They perceivedthe city's schools as just one of several key socialinstitutions failing poor children of color.Hornbeck's persistent fight for fair funding madehim a hero to these grassroots leaders even assupport for him crumbled in the business sector.

28

When the reform seemed to falter as discontentamong principals and teachers became morevisible and test scores flattened, many civicleaders began to feel that the plan neededrethinking and modification. But it was difficult togenerate public dialogue about the reform'slimitations or its accomplishments. It was difficultto raise questions, offer a different perspective, ormount an argument in the face of the compellinglogic of "We must change all levels of the systemat once." One civic leader crystallized theproblem: "The completeness of his [Hornbeck's]vision wasn't amenable to questioning. Youcouldn't tamper with any part of it."' Thecentrality of Hornbeck also posed an obstacle toopen discussion and revision. Criticism of theplan was tantamount to criticism of the superin-tendent's leadership. And finally, the largerpolitical and economic contexts played significantroles in discouraging open dialogue about thereforms. As district leaders became moreadamant in their arguments for fair funding,supportive civic leaders, especially, were reluctantto voice their concerns or raise questions. Theyfeared giving ammunition to political leaders whowanted to discredit the reform and theSuperintendent. When it began to appear thatChildren Achieving would not deliver thesustained gains in achievement that its leadershad promised, there was widespread disillusion-ment. Many of the leaders we interviewedexpressed the feeling, "We've done the best andthe most that can be done, let the state takeover. "25

Parents were also viewed as partners in theChildren Achieving reform. Reform leadersbelieved that parents were essential to theirsuccess and parent involvement was a centraltenet of the reform design. To them, parentalinvolvement went beyond traditional roles, suchas being members of parent teacher organiza-tions or attending school events. They wantedparents to be part of school-based decision-making teams and, together with teachers andprincipals, direct school policies and manageresources. The reformers also wanted parents tobecome a political voice for additional fundingand assistance for Philadelphia schools.

35


While their vision seemed clear, they found itdifficult to carry out. Engaging urban parents inthe schools proved to be difficult. Overwhelmedwith trying to provide for their families, facinglong workdays, and feeling disenfranchisementfrom civic life, parents faced severe obstacles togetting involved. There were few precedents forparents to serve as partners in school gover-nance. Resources to educate parents andcommunity members about new expectations androles were limited.

Some principals embraced and encouragedparents to be partners, and parent involvementincreased. Too often, however, principals werenot prepared to engage parents in school leader-ship roles and were threatened by theirinvolvement. As reported in a CPRE/RFA reporton parent involvement, "reformers did not takeinto account how deeply unsettling shifting thebalance of power among schools, parents, andcommunity would be to many principals andteachers. Reform planners underestimated what itwould take for schools, especially in low-income,racially-isolated neighborhoods, to turnthemselves around and work with parents ascollaborators in school reform."'

A few schools successfully worked with interme-diary organizations to enable parents to identifytheir concerns, raise controversial issues, work

through conflicts, and address issues thatimpacted their children. But, these instances wererare and were successful only when there wasstrong leadership in the organization, the school,or from a particular parent. Increasing parentinvolvement was a worthwhile goal, but inPhiladelphia, the impact was marginal, and in theend, had little influence on policymakers orschools.

A reform of the magnitude of Children Achievingrequired internal and external political supportthat could be sustained for many years. Withoutsustained and organized support, the voices ofcritics tended to drown out those of supporters.This was especially true after it became clear thatthe improvements in student performance werecoming more slowly than an impatient public hadbeen led to expect.

Philadelphia's experience shows how difficult it isto build resilient civic coalitions in the harshcircumstances created by inadequate funding. Itreveals the problems that arise for reform leadersas they try to communicate complicatedmessages about standards, accountability, anddecentralization to civic and business leaders whowant "solutions." It points out the importance ofsetting expectations and preparing the public forthe kinds of gains that might be anticipated.

3629


Strategy Eight: "All at Once"'Overloaded by the ref° man y school staff were ::1)

their efforts to improve. They could not translate rnanciatas into dearlydefined and manageable instructional priorities. This resulted in

widespread frustration among teachers and grincipals."

The Children Achieving Challenge was compre-hensive in its scope and ambitious in its goals.The 10-point plan was complicated andsometimes difficult to understand. It placed ahigh burden on school personnel. Each point hadmultiple facets and they were not alwayspresented or rolled out in a coherent fashion.Staff in many schools were overwhelmed andconfused by the various demands made on themby the central office or clusters. Teachers faceddevelopment of new curriculum in every subjectas well as new assessments, new work arrange-ments and relationships, new demands forprofessional development, new procedures forobtaining services for students, new evaluationprocedures, and other changes. They were toldthis had to be done "all at once." What madesense to administrators at the top was perceivedas "hell on the ground" by principals andteachers. What seemed connected and alignedon paper often was fragmented and out of syncin practice. At the heart of the problem weredisjunctures between the philosophy and normsabout teaching and learning that were central toChildren Achieving and those held by most of the

district's teaching staff. Little attention was givento altering the views of teachers. Rather it wasassumed that they would see the wisdom of thereform, embrace its tenets, and change theirbehavior.

Overloaded by the reforms, many school staffswere unable to focus their efforts to improve.They could not translate mandates into clearlydefined and manageable instructional priorities.This resulted in widespread frustration amongteachers and principals. Principals especially feltdisempowered and disrespected as they receivedone directive after another from the clusteroffices. To make matters worse, district leadersseldom sought their input and the supports andresources needed to get the job done were oftennot available or arrived late. The "all at once"theory also contributed to poor sequencing ofthe reform initiatives since all tasks had the samepriority. For example, the accountability systemwas in place several years before the curriculumframeworks offered teachers guidance about thekinds of curricular and instructional strategies thatwould help students meet the standards.

BEST COPY AVAILABLE

3731


Lessons Learned

he refA erns :fatig 3e and resistance a.,,ong teachers, and.---ernp,,trvere.d principals. support from the business community

evaporated and civic leaders became exasperated with the intractabilityof the reform plan and its leader. Children Achieving raised hopes in

Philadelphia, but left the city confused and anxious."

In one respect, the Philadelphia story is like aGreek tragedy. A noble cause, a committedleader, and a grand plan were undone by thefailure of the state and city to provide adequatefunding, by lack of understanding of what it takesto change schools, by poor implementation, andin some cases by the unwillingness of powerfulgroups to work together on behalf of children.

In another sense, it is like a Russian novel.Idealists sought to do too much, all at once. Well-intended people who thought that by movingquickly and broadly they would create agroundswell of support for change instead metwith opposition and apathy. The reforms createdfatigue and resistance among teachers, anddisempowered principals. Initial support from thebusiness community evaporated and civic leadersbecame exasperated with the intractability of thereform plan and its leader.

Children Achieving raised hopes in Philadelphia,but left the city confused and anxious. Is there away to improve urban schools? What is it? Thereare some important lessons to be drawn fromChildren Achieving. They fall into two categories:those that apply to almost all efforts to improveteaching and learning, and those that are specificto systemic approaches to reform.

Remedial Lessons about Teaching andLearning

Some of the lessons learned in Philadelphia havealready been "learned" from the experienceswith other reforms in other cities. Too oftenreformers repeat the mistakes of others, failing tolearn from these previous experiences or fromresearch. This was true in Philadelphia. Amongthese lessons are:

Content standards and accountabilitymeasures offer a foundation for instructionalchange, but produce results only if teachersare provided the time and support necessaryto develop high-quality curricula and to altertheir classroom methods. For many teachersin Philadelphia, closing the gaps between thepre-reform curriculum, their own contentknowledge, and their beliefs about what theirstudents can learn, and the new standards setunder Children Achieving was a difficult task.Given the severe time pressures set by theaccountability system and the inadequateguidance and support provided for them,many opted for drilling students for the test.Achieving the deeper changes in classroompractice and the rigorous curriculum that thearchitects of the new standards hadenvisioned would have required moreguidance, demonstration proofs, richerlearning opportunities, much more time, andgreater understanding and sensitivity to thedaily challenges of teaching urban class-rooms.

Curriculum-based, sustained professionaldevelopment is essential to instructionalchange. Providing teachers with high-qualityopportunities to master content andpedagogy, model curriculum units, andongoing support from accomplished teachersproduces results.

The effects of teacher hiring practices,teacher retention, and teacher transfers onthe distribution of teacher talent and thecapacity of schools to make improvementsmust be addressed. Incentives are needed torecruit and retain new teachers, especially inthe middle grades, and policies are needed


33


to encourage master teachers to practice inthe most challenging schools.

Too much reform can overload the schoolsand teachers. Focus on a few things. Do themwell and demonstrate that better results arepossible. Stay focused, be persistent, andbuild demand for the next set of changes.Pilot new approaches to demonstrate thatthey are effective. As teachers see improve-ments in one area, as they come to believethat students can do the work, they will bemore willing and better equipped to take onthe next challenge.

Different strategies are needed for highschools. One can't hold high school staffsaccountable for meeting high standards whenstudents are arriving in ninth grade lackingskills essential to academic success. Districtsneed a catch-up strategy and they need toprepare high school staffs to implement it.

Win over the teachers. You should not blamethem for social ills they didn't create oraccuse them of not caring or working hardjust because test scores are low. Ultimately,success depends on their commitment andtheir sense of personal accountability.

Prepare staff at all levels of the system for thenew roles and responsibilities called for in thereforms. Structural changes demand thatpeople behave in new ways and they needsupport in learning the new behaviors.

Early intervention to help children acquire thelanguage skills needed for success in school iscritical. Full-day kindergarten appears to havepositively impacted student progress inPhiladelphia, and concentrating on languagearts in the early elementary years may havehad the most impact on student performance.However, the city of Philadelphia failed torespond to the district's efforts to expand andimprove early childhood programs. Thesuperintendent offered a plan but it wasrebuffed by other city agencies. Questions ofwho was in charge apparently took prece-dence over increasing children's access togood programs. As a consequence, an oppor-

34

tunity to lay a solid foundation for theimprovement of the public schools wasmissed.

New Lessons about Systemic Reform

A first lesson about systemic reform concerns thetrade-offs between alignment and autonomy.Children Achieving had at least one seriousdesign flaw. As described in the literature,systemic reform assumes well-aligned accounta-bility and assessment systems and school controland development of curriculum. The idea is that acentral authority (a state or, in Philadelphia's case,the central office) sets content and performancestandards and holds schools accountable, but inturn the schools are granted the opportunity todevelop means of reaching the standards. In thetheory, this includes control over their curriculum.What we believe to be a basic flaw in this theoryis that well-aligned accountability and assessmentsystems either presume or lead to a fairly uniformcurriculum. If a central authority wants to holdschools accountable for what students learn, itsassessment system must test what teachers teachor to put it another way, teachers must know whatwill be tested and it is likely that this is what theywill teach. Therefore, in such a scenario,individual schools and teachers do not determinetheir own curriculum. They teach a curriculumdetermined by the standards and assessments. Awell-aligned system is not tolerant of multiplecurricula, developed by individual teachers andschools.

The School District of Philadelphia lived out thiscontradiction as they worked to implementChildren Achieving. Initially, central office stafffocused on decentralizing authority to schoolsand giving school staff the opportunity to maketheir own decisions. They set standards andestablished an accountability mechanism thatincluded incentives for schools to improve theirperformance. They were criticized because thedistrict test did not perfectly align with thestandards, so they worked to improve the align-ment. Results from testing and other measures,however, continually pointed out wide disparitiesin outcomes and opportunities to learn for low-

39


income students, students of color, and Englishlanguage learners so the central office sought toeven the playing field. As the Hornbeck adminis-tration came to an end, central office staff andthe Board of Education were developing tests inevery subject area for grages 7-12 in order toclearly signal to teachers what to teach. Only ayear after Hornbeck left, the central office wasworking on a districtwide curriculum in

response to teacher demand. The shift from thepromise of autonomy to a common curriculumwas dramatic.

Ironically, throughout the reform effort, personnelin the field perceived most actions of the centraloffice as mandates, even their efforts to decen-tralize authority. Because the decision todecentralize was made centrally, and relatedpolicies were designed with little consultationwith the field, school and cluster personnelexperienced decentralization as a set of centraloffice mandates, rather than opportunities forschool autonomy. And there was little evidencethat the schools were ready and eager to takecontrol of curriculum in Philadelphia. Teachersconsistently called for more specific guidelinesabout what and how to teach.

Philadelphia's leaders equated school autonomywith control over curriculum. One need notpresume the other. School districts consideringsystemic reform efforts might grant schoolscontrol over other aspects of their operationbudgets, staffing, and professional development,for example while maintaining a centralizedcurriculum. Alternatively, school districts coulddevelop a curriculum that covered some propor-tion of the school year, leaving schools andteachers free to determine how to use theremaining time. Districtwide assessments wouldfocus on the mandated curriculum.

A second lesson to be drawn about systemicreform is about standards, how they becomeubiquitous, and how they can undermine theincreased autonomy that they are supposed toprovide to schools and teachers. Under ChildrenAchieving, one of the functions of the centraloffice was to set standards. As the central officestaff recognized the need to provide schools with

more guidance about how to carry out thereforms, they drew on their authority to setstandards to provide it. In addition to content andperformance standards, the central office staffdefined "process" standards that spelled outacceptable practice in a variety of areas. They setstandards for local school councils, small learningcommunities, professional development, teachingpractice, project learning, service learning, thecomprehensive support process, and so on. Thestandards were perceived by central office staff asa benign form of guidance, and as being differentfrom mandates. The label, in their view, impliedadvocacy for quality rather than an exercise ofauthority and control.

However, to school staff, these "process"standards were just prescriptions in another form.And as cluster and central office staff used thesevarious sets of standards to determine whetherschools were complying with the reforms, theyclearly were viewed as regulatory mandates.Calling them standards did not change the factthat they felt like regulations to the school staffs.

Other districts can learn from this experience. Ifthe idea of standards is to define an ideal state ofaffairs, then deviation should be expected, andpermitted, across contexts. This is especially trueof "process" standards. In a decentralizedenvironment, the central office could acknowl-edge that deviations are expected, maybe evenencourage them, and focus on the results ratherthan on compliance. Or they could define param-eters of acceptable behavior and the conditionsunder which deviations would be permitted. It isprobably important to distinguish betweenstandards and regulations. The former implysome professional legitimacy, some basis inknowledge or experience, while the latter aresimply the operationalization of policies that mayor may not have such grounding.

40 35


Tefor; raise.grim reality

amon citizens aboutressured the state

V-1,3 C e of public educethat greeted this

In Conclusion

5 for Philedelp childr , made publicns in the schools, 'o ^,y discussion

e Qf public education in Phi ,delphia andaction to change the political and legal

on in the city...i3ut in the end, the high hopesambitious reform were not fulfilled."

A combination of factors contributed to ChildrenAchieving's failure to produce sustained improve-ments and deep changes in practice. Theseincluded: flaws in the theory of action itself, flawsin implementation, lack of capacity and lack ofattention to building it, and behavior inconsistentwith the beliefs and values underlying the reformeffort.27

Reformers encouraged business, civic, and grass-roots leaders and parents to believe thatstandards, accountability, and decentralizationwould bring improvements in student perform-ance. As achievement data showed that thesestrategies were inadequate, they did not helpbusiness and civic leaders understand that theplan needed to be revised. There was not anunderstanding in the community that the reformplan was a living document and modificationsbased on experience were essential to itsultimate success.

As Christman noted, "...the selling of systemicreform as comprehensive common sense and as apackage that 'all had to be done at once,'undercut the possibility for the input and accom-modations necessary for building alliances forreform. It discouraged critical questions, reflec-tion, and revision all necessary fororganizational learning."' Without strong support

from stakeholders, systemic reform is hard tosustain: progress is too slow, and communitiestoo impatient.

Central office celebratory rhetoric further erodedsupport from civic, political, and business leadersas they struggled to understand the discordancebetween the claims of district leadership, thecomplaints of teachers, and the continuing realityof low student achievement scores. In the end,the complexity of the design coupled with inade-quate resources and support and a failure to winsupport from teachers combined to bringChildren Achieving to an end. This is not to saythat nothing was accomplished. The reform raisedexpectations for Philadelphia's children, madepublic the grim reality of the conditions in theschools, forced a discussion among citizens aboutthe future of public education in Philadelphia,and pressured the state into action to change thepolitical and legal structure of public education inthe city. The elementary schools did show signifi-cant gains in student performance and theintroduction of full-day kindergarten and earlyliteracy programs ensured that the city's childrenwould have additional support as they begantheir school careers. But in the end, the highhopes that greeted this ambitious reform werenot fulfilled.

3EST COPY AVAILABLE

41 37


Endnotes

E. Useem and R. Curran Neild, Teacher staffingin the School District of Philadelphia: A reportto the community. Philadelphia: PhiladelphiaEducation Fund, 2001.

2 School District of Philadelphia, Realitiesconverge: This year is different. Philadelphia:Author, 1997.

3 School District of Philadelphia, Realitiesconverge.

4 J.B. Christman and A. Rhodes, Civic engage-ment and urban school improvement:Hard-to-learn lessons from Philadelphia.Philadelphia: Consortium for Policy Research inEducation, University of Pennsylvania, 2002.

5 Consortium for Policy Research in Education,Research for Action, and OMG Center forCollaborative Learning, Children Achieving:Philadelphia's education reform, progress reportseries 1996-1997. A second-year evaluation,executive summary. Philadelphia: The ChildrenAchieving Challenge, 1998.

6 E. Foley, Contradictions and control in systemicreform: The ascendancy of the central office inPhiladelphia schools. Philadelphia: Consortiumfor Policy Research in Education, University ofPennsylvania, 2001.

7 Foley, Contradictions and control in systemicreform.

8

9

Foley, Contradictions and control in systemicreform.

Foley, Contradictions and control in systemicreform.


11 J.B. Christman, Powerful ideas, modest gains:Five years of systemic reform in Philadelphiamiddle schools. Philadelphia: Consortium forPolicy Research in Education, University ofPennsylvania, 2001.

12 M.N. Spin, School leadership and reform: Casestudies of Philadelphia principals. Philadelphia:Consortium for Policy Research in Education,University of Pennsylvania, 2001.

13 J.B. Christman and P. Macpherson, The fiveschool study: Restructuring Philadelphia'scomprehensive high schools. Philadelphia:Philadelphia Education Fund, 1996.

14 E. Gold, A. Rhodes, S. Brown, S. Lytle, and D.Waff, Clients, consumers, or collaborators?Parents and their roles in school reform duringChildren Achieving, 1995-2000. Philadelphia:Consortium for Policy Research in Education,University of Pennsylvania, 2001.

15 R. Curran Neild and C. Weiss, Philadelphiaeducation longitudinal study: Report on thetransition to high school in the School Districtof Philadelphia. Philadelphia: PhiladelphiaEducation Fund, 1999.

16 School leadership and reform.


18 E. Foley, Restructuring student services:Redefining the role of the school district.Philadelphia: The Children AchievingChallenge, 1998.


20 Foley, Restructuring student services.

21 Foley, Restructuring student services.

22 Christman and Rhodes, Civicurban school improvement.




4 4

engagement and

engagement and

engagement and

engagement and

39


26 Gold, Rhodes, Brown, Lytle, and Waff, Clients,consumers, or collaborators?


28 Christman and Rhodes, Civic engagement andurban school improvement.

40 43


Appendix AAdditional Reading on Children Achieving

The following following publications on the evaluation of Children Achieving are currently available throughCPRE at (215) 573 -0700, or email your requests to [email protected]. Copies can also be downloadedat http://www.cpre. o rg/Resea rch/Research_Project_Ch i Id ren_Ach i eving.htm.

Recruiting and Retaining Teachers: Keys toImproving the Philadelphia Public Schools,by Susan Watson (May 2001)

School Leadership and Reform: CaseStudies of Philadelphia Principals, by MaryHelen Spin (May 2001)

Contradictions and Control in SystemicReform: The Ascendancy of the CentralOffice in Philadelphia Schools, by EllenFoley (August 2001)

Clients, Consumers, or Collaborators?Parents and their Roles in School ReformDuring Children Achieving, 1995-2000, byEva Gold, Amy Rhodes, Shirley Brown, SusanLytle, and Diane Waif (August 2001)

Case Studies of a Systemic Reform in UrbanSchools, by Susan Watson, Rhonda Phillips,Claire Passantino, Jolley Bruce Christman,Nancy Lawrence, Theresa Luhm, and HitomiYoshida (October 2001)

Powerful Ideas, Modest Gains: Five Years ofSystemic Reform in Philadelphia MiddleSchools, by Jolley Bruce Christman(December 2001)

An Analysis of the Effect of ChildrenAchieving on Student Achievement inPhiladelphia Elementary Schools, by ErinTighe, Aubrey Wang, and Ellen Foley(February 2002)

Civic Engagement and Urban SchoolImprovement: Hard-to-Learn Lessons fromPhiladelphia, by Jolley Bruce Christman andAmy Rhodes (June 2002)

The following publications on the evaluation of Children Achieving are available exclusively from CPRE'sweb site at http://www.cpre.org/Research/Research_Project_Children_Achieving.htm.

The Accountability System: DefiningResponsibility for Student Achievement, byTheresa Luhm, Ellen Foley, and Tom Corcoran(Spring 1998)

Children Achieving: Philadelphia'sEducation Reform, A Second-yearEvaluation, Executive Summary (Spring 1998)

Guidance for School Improvement in aDecentralizing System: How Much, WhatKind, and From Where?, by Jolley BruceChristman (Spring 1998)

Making Sense of Standards:Implementation Issues and the Impact onTeaching Practice, by Elaine Simon, EllenFoley, and Claire Passantino (Spring 1998)

Restructuring Student Support Services:Redefining the Role of the School District,by Ellen Foley (Spring 1998)

44 41


Appendix BChildren Achieving Evaluation Research Methods

In 1996, the Consortium for Policy Research in Education (CPRE) at the University of Pennsylvania andits partner, Research for Action (RFA), were charged by the Children Achieving Challenge with theevaluation of Children Achieving. Between the 1995-1996 and 2000-2001 school years, CPRE and RFAresearchers interviewed hundreds of teachers, principals, parents, students, District officials, and civicleaders; sat in on meetings where the plan was designed, debated, and revised; observed its imple-mentation in classrooms and schools; conducted two system-wide surveys of teachers; and carried outindependent analyses of the District's test results and other indicators of system performance. Anoutline of the research methods used by CPRE and RFA follows.

1. 1996-2000 school-level data on indicatorsthat made up the District's PerformanceResponsibility Index including student scoreson the SAT-9, student promotion and gradua-tion rates, student attendance, and teacherattendance.

2. Two census surveys of teachers, the first in1997 and the second in 1999. Teachers wereasked about reform implementation, schoolconditions, and teaching practices. There wasa greater than 60 percent response rate onboth surveys.

3. School indicators describing teacher andstudent characteristics in 1996 and 1999obtained from the School District ofPhiladelphia's Information Services. Thesedata included school enrollment, number ofteachers, the proportion of students quali-fying for free or reduced-price lunch, amongother indicators. These data were used fordescriptive purposes and in hierarchical linearand logistic regression models to help under-stand the relationships among reformimplementation, student outcomes, andschool characteristics.

4. Five years (1995-1996 through 1999-2000) ofqualitative research in 49 schools (26 elemen-tary, 11 middle, and 12 high schools) in 14clusters. Qualitative research included: inter-views of teachers, principals, parents, outsidepartners who worked in the schools, and in afew cases, students; observations of class-rooms, small learning communities meetings,professional development sessions, andschool leadership team meetings; review ofschool documents (School Improvement Plan,budget, etc.); and intensive, multi-year casestudy research in a subset of 25 schools (13elementary, 5 middle, and 7 high schools).

5. Interviews of central office and cluster staffand observations of meetings and otherevents.

6. Interviews of 40 Philadelphia civic leaders(including political leaders, leaders in thefunding community, public educationadvocates, journalists, and business leaders).

In addition, numerous other studies conducted during Children Achieving informed this evaluation.These included: Bruce Wilson and Dick Corbett's three-year interview study of middle school students;an evaluation of the Philadelphia Urban Systemic Initiative in Mathematics and Science conducted byResearch for Action; the Philadelphia Education Longitudinal Study conducted by Frank Furstenbergat the University of Pennsylvania; and the evaluation of the William Penn Foundation's initiative in twoclusters, conducted by the National Center for Restructuring Education, Schools, and Teaching.

45 43

Consortium for Policy Research in EducationThe Consortium for Policy Research in Education (CPRE) unites five of thenation's leading research institutions to improve elementary and secondaryeducation through research on policy, finance, school reform, and schoolgovernance. Members of CPRE are the University of Pennsylvania, HarvardUniversity Stanford University, the 'University of Michigan, and the University

of Wisconsin-Madison.

CPRE studies alternative approaches to education reform in order todetermine how state and local policies can promote student learning.Currently, CPRE's work is focusing on accountability policies, efforts to buildcapacity at various levels within the education system, methods of allocatingresources and compensating teachers, and governance changes like chartersand mayoral takeover. The results of this research are shared withpolkymakers, educators, and other interested individuals and organizationsin order to promote improvements in policy design and implementation.

Research for Action

Research for Action (REA) is a Philadelphia-based nonprofit organizationengaged in educational research and reform. Founded in 1992, RFA workswith educators, students, parents, and community members to improveeducational opportunities and outcomes for all students. REA work fallsalong a continuum of highly participatory research to more traditional policystudies that use qualitative methods and case study analysis. RFA also seeksto inform the national dialogue about strong schools and effective educationthrough reports and journal articles, conference presentations, videos, andcollaboration with other school reform groups.

Nondiscrimination Statement

The University of Pennsylvania values diversity and seeks talented students,faculty, and staff from diverse backgrounds. The University of Pennsylvaniadoes not discriminate on the basis of race, sex, sexual orientation, religion,color, national or ethnic origin, age, disability, or status as a Vietnam EraVeteran or disabled veteran in the administration of educational policies,programs, or activities; admissions policies, scholarships, or loan awards:and athletic or University administered programs or employment. Questionsor complaints regarding this policy should be directed to Executive Director,Office of Affirmative Action, 1133 Blockley Hall, Philadelphia, PA19104-6021 or (215) 898-6993 (Voice) or (215) 898-7803 (TDD).

BEST COPY AVAILABLE

Consortium for Policy Research in Education

Graduate School of EducationUniversity of Pennsylvania

341.0 Market Street, Suite 560Philadelphia, PA 19104-3325Telephone 215.573.0700Fix 215.573.7914

www.cpre.org

Research for Action

international House3701 Chestnut StreetPhiladelphia, PA 19104Telephone 215.823.2500Fax 215.823.2510

WWW. researchforaction.org

iz(

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

National Library of Education (NLE)Educational Resources Information Center (ERIC)

NOTICE

REPRODUCTION BASIS

ERIC

This document is covered by a signed "Reproduction Release(Blanket) form (on file within the ERIC system), encompassing allor classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Release form(either "Specific Document" or "Blanket").

EFF-089 (9/97)

Documents

Reproductions supplied by EDRS are the best that can be ... · Susan Lytle, Bernard McMullan, Ruth Curran Neild, Claire Passantino, Rhonda Phillips, Amy Rhodes, Elaine Simon, Mary