continuousimprovement:

l i t takes more than test scoresAnalyzing state

assessment results isonly the beginning ofeffective data-driven

decision making.

There is no question that the pns-suge of No Child Left Behindin 2001 has impacted schools

ill at least two ways: First and foremost,NCLB has made the use of data to im-prove student achievement imperative;and second, NCLB has increased theneed for continuous improvement pro-cesses within schools.

Summative data just the beginningSchools in our country hear that data

makes the difference in improving stu-dent achievement. Not all schools, how-ever, have felt the positive impact fromwhat they believe is data-driven decisionmaking. The most common reason: Mostschool districts in this country believethey are being data-driven when theyhave analyzed the dickens out of theirstate assessment results.

Some school districts fed they arebeing data-driven when they analyze thedickens out of their state assessment re-

sults ami use some formative assessmentsto help students prepare for the statewidetest. Unfortunately, this is only the be-ginning of data-driven decision making.

Most states' assessment scores canspeak volumes for what is going on intheir schools and districts, and with stu-dent learning. Following assessmentscores of the same groups of students (orthe same students) over time can indicatethe presence or absence ot a continuumof learning that makes sense for the stu-dents. It can give information aboutstudLMit learning as well as the degree towhich standards are being implementedat each grade level (provided the stateassessment truly measures the state stan-dards).

Looking at state assessment results bygrade level over time can give infornia-tion about the impact of the programs

By Victoria L Bernhardt

16 Leadership

being implemented. Disaggregating stateassessment results ctn tell m if our schoolsare meeting clie needs of the students theschools are Littempting to serve.

Test score analyses are important. Infact, in a perfect world, schools woulduse both formative and summative as-sessments to ensure that all students arelearning. If only summative assessmentdata are studied, however, solutions forimproving the scores can come out half-baked.

Forex.iniple. wheTi 1 started workingwitli Lemon Middle School, thestaH'haddetermined that their students' scores inEnglish/language arts and mathemat-ics were lower than the previous years'scores. Since the math scores were thelowest, they decided to "focus" on maththat year.

Gallant effortsIn their focus, they set up several strat-

egies: remediation for the students notmeeting proficiency standards, an after-school program to assist students withtheir math homework, and a requiredmath summer school program for anystudent not passing the state math assess-ment at the proficient or advanced levels.

Unfortunately, their gallant effortsdid not lead to the test score improve-ments they had hoped for. Both math andEnglish/language arts scores went down.They were devastated. As the staff and 1reviewed their data and their solutions,we talked about establishing a continuousimprovement plan. We set out to gather abit more data to see if we could figure outhow to work smarter, not harder, and getbetter results.

We had the student achievement data.We deterniirK'tl that other types of data,including demograpliic, perceptional andschool process data, needed to be gath-ered and analysed. What those data areand what we found in the data analysisat Lemon Middle School are discussedbelow.

Demographic dataDemographics can tell schools all

about who they have as students, whothey have as teachers, and how teachersare aligned to the students. Demograph-

ics begin to tell us about schotil processesand how the school is preparing to meetthe needs of students. Demographics areimportant for setting tlie context of theschool, and they are critical for under-standing all other numbers.

Lemon Middle School's demograph-ics showed that while they had a fairlydiverse student population — 60 percentCaucasian, 30 percent Hispanic/Latinoand 10percent African American—-their

A clear and shared vision andleadership play major parts indata-driven decision making.Iff there is no focus or uniffiedffront in a school, there is alsono continuum off iearning thatmakes sense ffor students,and no structure to increasestudent achievement.

teaching staff was 100 percent Caucasianand SO percent female.

After a recent teacher "buy out" dueto budget decreases, the current teachingstaff had an average of six years of teach-ing experience. The math teachers werethe least experienced, with only threeyears of teaching experience on average.The principal, a male, was in his thirdyear at this school. The attendance ratewas 94 percent for students and 91 per-cent for teachers. The district, in its con-cern about the inexperience of the te.ich-ers and the low test scores, assigned mostof the specialists in the district to "watchover and help" Lemon teachers.

Perceptional dataPerceptional data can tell us about

how students, staffand parents are feelingabout the learning environment and giveinsight into what changes can be made toimprove the learning environment and,ultimately, student learning.

Lemon Middle School students, staffand parents completed questionnairesabiHit the learning environment. Whatthe questionnaires told them was studentsdid not feel teachers thought they cotilddo the work, that quality work was notexpected of them, and that teachers didnot care about them as people — teach-ers only wanted students to get better testscores.

In their open-ended responses, sev-eral students wrote that all they likedabout the school was being outside, tlieirfriends and sports. What they wished wasdifferent was that school would not be soboring, that they could learn cool thingsin cool ways, and that they would not getmath and test scores shoved down theirthroats all the time.

The teacher questionnaire gave evenmore definitive information. Teachersfelt that there was no vision for the school.There might have been a vision a coupleof years ago, but that was before most ofthe current staffwas hired. Teachers didnot feel they had the support they neededto improve student learning, and they didnot collaborate with each other to ensurea continuum of learning that made sensefor the students.

In their open-ended responses, stafftold about the inconsistencies in teach-ing throughout the organization, the factthat staffagreements were never enforcedand that the district was always sendingpeople in to "watch us. not help us."

Commitment from staffThe most often-stated responses to

the question, "What would it take toimprove student learning for all studentsat this school?" included this: AH start"needs to coinniit to doing work in thesame way. including teaching to the stan-dards, assessing and using the results toeffect iniprt)vement. Teachers who arenot willing to put in the time and energyshould not be allowed tt) keep their jobs,or they should have their pay docked.

Parents basically reiterated what thestudents told staff. Parents did not feelthat the staffwas truly committed to stu-dents' learning. Teachers did not seem tobe making progress with what they weredoing. Staff expected way too much help

November/December 2UO4 17

from the parents. Parents did not knowhow to help their children learn. Addi-cionally. each parent worked one or twojobs. When could they help their diil-dren learn?

School processes

School processes are the curriculum,instruction iind assessment strategies usedto teach the content that students are ex-pected to learn. It is important to under-stand what teachers are doing to get theresults they are getting.

Lemon Middle School staff"reviewedwhat they were doing to teach each oftheir subject areas. Most teachers wereusing the curriculum and instructionalmaterials adopted by the district. Whatthey could not tell each other was howmuch of the time they were teaching tothe standards. In fact, they were not re-ally sure what it would look like if theywere teaching to the standards.

Their current processes for helpingstudents were to '"remediate" any student

who did not score proficient or advancedon the state assessment. A lot of the reme-diation was simply re-teaching what hadbeen taught before without determiningwhat the students really needed.

Analyzing the data

These data — student learning, de-mographics, perceptions and school pro-cesses, separately and combined -— telischool personnel what is going on in theschool right now. give clues as to whatneeds to be done to improve, how toimprove to get different results and evenwhat is possible with respect to improve-ment.

Reviewing all the data gave LemonMiddle School staff some commonthinking about what they needed to doto get different results. It also empoweredthem to get different results. Staff knewthey needed a vision and commitmentfrom every teacher to help every studentlearn. They knew they needed to becomemore familiar with and to implement

content standards, and to learn how tomake learning more active and fun. Theyalso needed and wanted accountabilityprocesses and leadership to keep all thestaff on the same page.

Because of the inexperience of thestaff and principal, the district providedthe dollars to help Lemon Middle Schoolhire an outside facilitator to establish aclear and .shared vision and a structure forcontinuous improvement. The structureinchided time to analyze their data andstudent work, and to develop strategiesfor improvement using the results of theiranalyses. Students' test scores in the fol-lowing year were greatly improved.

The need for a continuous improvementprocess

True data-driven decision making isonly partly about data. A clear and sharedvision and leadership play major parts indata-driven decision making. If thereis no focus or unified front in a school,there is also no continuum of learning

LOOK FOR THE WARNING SIGNSLet SELF help you carry the burden of workers' compensation coverage

A cuslodian Injured tiis right knee while lifting a 55-gallondrum and walking down a flight of stain. Subsequently, theclaimant required six surgeries and a cotal knee replacement.Became of a surgical infection, the claim would have beendevastating but with SELF, both the custodian and the districtcould continue with business as usual.

Sell-Insurance is recognized as a viable alternative to commercialinsurance. Since 1989. SELF's Excess Workers'CompensationProgram has provided many California self-insured scticoli andjPAs with reliable excess coverage.

For districti considering self-insurance, SELF can supplementyour program at one of our self-insured retention (SIR) levels:

$350,000 $500,000 $1 million

Benefits of Self-Insurance•Actuarially determined rates• Conirol over claims process• Lower administrative expenses

Schools Excess Liability Fund

Contact us today to find out virhymost California school districtj chooseSELF for excess insurance coverage-Toll Free (866) 45J-5J0O,www.selfjp3.org

18 Leadership

tluit nuikes sense for students, and nostructure to increase student achieve-ment.

It takes strong leadership to inspirea shared vision and to ensure its imple-mentation. It also takes a strong leaderto ensure the analysis and use of data.A continuous improvement process canensure that all professional developmentis focused on implementing the vision;[bat partners, such as parents, understandtheir roles in implementing the visionand helping students learn: and thiit thereis continuous cv.ikuition to know howto improve on an ongoing basis to reachschool goals.

Inspiring a shared visionSchools and school districts ,ire under

intense pressure to improve. With bothlimited resources and limited time to de-velop processes that allow them to movesteadily upward, schools must use data toensure that their improvement is effec-tive and continuous.

It is vitally important that administra-tors — at school and district levels — leadthe way in using data to inform decisionni;iking. Leaders must challenge pro-cesses through tbe study of school results,inspire a shared vision, enable ocbers toact through planning and professionaldevelopment, model the way throughconsistent actions, encourage the beartby reminding teachers of the purpose oi'school and why they got into teachingin the first place, and celebrate successes(Kouzcs&Posncr, 20(12).

True data-driven decision makinggives schools information about theircurrent situation and clues about whatwould help them improve, as well as tbeleadership to see it through.

ReferencesBeriihardt. V.L. (2003). Usiiia Dora to

hnproi'e Stiidctit Learning in BleiiicutarySchools. Larchmont, NY: Eye on Edu-cation, Inc.

Benibardt, V.L. (2(104). t'Vm.i.' Dam to Itn-

provf Sludent Learning in Middle Schools.Larchmont, NY: Eye on Education,Inc.

Bernhardt, V.L. (2004). Usiiis; Data M Im-prove Student Learning in H(i,'/f Schools.Larchmont, NY: Eye on Education,Inc.

Bernhardt, V.L. (2005). l!si,i<^ Data toInipnwe SitidenI Lvaniin^ in School Dis-iricis. Larchmont, NY: Eye on Educa-tion, Inc.

Kouzcs, j.M., & Posner, H.Z, (2002). Theleadership clialleii^e: How fo kee{\^cningextraordinary things do}ie in organizations.(2nd Ed.). San Francisco, CA:Jossey-Bass Publishers.

Victoria L Bemhardt has directed ttie Educa-tion for the Future Initiative since 1991. worliingwith schoois and districts across the country.Bernhardt is the autiior ot 10 books, alipub-

iished by Eye on Education, including theiatest four-book series: "Using Data to Improve

Student Learning."

What good is research-based School improvementwhen the research wasn't based on your School?

Foi more than 35 years, education researchers havegenerated data and reports on how to improvestudent learning. But all this research isn'thelpful to your school until you know howto apply it to your specific situation.

That's why ASCD has developed What Worksin Schoob. a comprehensive framework forschool improvement that helps you identifyand take action against the factors in yourschool or district that block student learningand achievement.

Use These Resources to Base Your School Improvement Efforton What Works

• The What Works in Schools book and video series introduceeveryone in your school to the research-based factors that improvestudent achievement.

• Our What Works in Schools Online Survey helps your schoolor district identify improvement targets and assess your progress.

• Our newest book, Building Background Knowledge forAcademic Achievement, explains how to improve the achievementof students who are rot well prepared for your school curriculum.

ASCD . . . The Trusted Source on Teaching and Learning

Focus Your School Improvement Effort with ASCD Resources!Explore articles, sample book chapters, and video clips online. Visit shop.ascd.org and

search on "what works" or call toll-free 800-933-ASCD (2723).

Association for Supervision and Curriculum development1703 Norif) Beaiiregard SIreel • Alexandria, VA 22311 -1714, USA • WViW.ascd.org

November/lX'ceinbcr 2O(i4 19