Setting the Pace:the Pace:the Pace:the Pace:the Pace - UChicago

1Setting the Pace

Julia B. Smith

BetsAnn Smith

Anthony S. Bryk

CImproving hicago's Schools

November 1998

SettingSettingSettingSettingSettingthe Pace:the Pace:the Pace:the Pace:the Pace:Opportunities toOpportunities toOpportunities toOpportunities toOpportunities to

Learn in Chicago'sLearn in Chicago'sLearn in Chicago'sLearn in Chicago'sLearn in Chicago's

ElementaryElementaryElementaryElementaryElementary

SchoolsSchoolsSchoolsSchoolsSchools

Consortium onChicago SchoolResearch

1313 East 60th StreetChicago, Illinois 60637773-702-3364

2 Improving Chicago’s Schools

Six Reasons Why Instruction Slows Down

examined and reconsidered. We find no connectionbetween slow pacing and improved student learning,a finding echoed in the recent Third InternationalMath and Science Study (TIMSS).1 All we can saywith the evidence at hand is that steady exposure toslow pacing leaves Chicago’s students farther andfarther behind.

Some repetitions more understandably reflectthe fact that certain topics and skills have no obvi-ous or fixed place in the curriculum. An examplefrom our observations was teaching the eight partsof speech. It is common to introduce middle gradestudents to the eight parts of speech, but many highschool language arts textbooks cover them also. Allof us forget what the eight parts of speech are (youcan test yourself ). But there is an important differ-ence between being reminded of them and spendingwhole lessons writing up their definitions. Forget-ting that pronouns are one of the eight parts of speechmay not mean that one doesn’t know what a pro-noun is or how to use it. But many teachers we ob-served and interviewed either feared this to be thecase or believed it wiser to hedge their bets throughthorough reteaching.2

2. Grades and test preparation. Ironically, andpainfully for thousands of teachers, the demands forimproved grade and test outcomes seem to slowinstruction. As the end of each quarter approaches,teachers are increasingly tempted to assign familiar,easy to complete tasks to help their students earnpoints and avoid a failing grade. Time spent in thesefamiliar tasks limits time spent exploring new materialand developing new skills.

Our study of schools and classrooms taught us that manyfactors contribute to slow pacing. Some factors areorganizational, such as very weak grade to gradeinformation about what is actually being taught andlearned. Indeed, we were regularly queried about suchthings by teachers during our school visits. “What arethey teaching down there in those other grades?”

Other factors are human. Practices and habits thatcontribute to slow pacing are rational and well inten-tioned, but they accumulate into patterns that steadilyerode learning opportunities. Below we outline six ofthe most pervasive factors suggested by observationsand interviews with teachers. We offer these factorsto foster understanding and to encourage reflectivediscussion and action.

1. Review and repetition. Starter segments thatreview prerequisite knowledge and skills are a time-tested method for introducing new material. Forexample, when we observed lessons aboutparallelograms, teachers began by reviewingdefinitions of polygons and parallel lines. But whenreview segments last longer than the segments focusedon new material and skills, as we often observed,instructional programs acquire a “two steps back, onestep forward” rate of progress. Teachers’ decisions toreview material frequently stem from the bestintentions, such as a desire to present material in aclear, step-by-step process. As more than one teachersternly reminded us, “People say that repetition isbad, but sometimes it isn’t. I think the kids do betterwhen they repeat things.” Indeed, many teachers weobserved and interviewed believed that slower pacingand repetition helped their students to learn. But howoften this repetition and review occurs needs to be

1William H. Schmidt, Curtis C. McKnight, Pamela M. Jakwerth, Leland S. Cogan, Senta A. Raizen, Richard T. Houang, Gilbert A.Valverde, David E. Wiley, Richard G. Wolfe, Leonard G. Bianchi, Wen-Ling Yang, Seung-Ho Kang, and Edward D. Britton, Facingthe Consequences: Using TIMSS for a Closer Look at United States Mathematics and Science Education (Boston: Kluwer AcademicPublishing, 1998).

2To ease your minds, the eight parts of speech are noun, pronoun, verb, adverb, adjective, article, conjunction, and interjection.

3Setting the Pace

The most predictable and extensive instructionalslowdown occurs during the spring, as teachers stopintroducing new material to review and practice forthe state-mandated achievement tests and the IowaTests of Basic Skills. During the first year of our fieldstudies (1994), this review typically began in March.In each successive year that we visited schools, this“test prep season” began earlier and lasted longer.3

3. Weak homework norms. A great deal of practiceand review occurs in classrooms because it does notoccur at home. Homework is a source of instructionaltime and contributes to instructional pacing for thesimple reason that it extends opportunity to learnbeyond the school day. Furthermore, the commonpractice of beginning homework in class also slowsinstruction by absorbing time when teachers andstudents could work together on new material.

4. Student resistance. Students can slow instructionwhen they resist difficult or unfamiliar material. Weobserved students telling teachers that they had neverbeen taught a particular topic when we had, in fact,observed teachers in earlier grades doing so. Howoften do teachers hear their students protest, “Thisis too hard!” or “We don’t know how to do this” beforestopping to reteach former topics or to providestudents with opportunities to do well by giving themfamiliar tasks? On any given day it may feel wiser orsafer to review and practice old content, postponing

new work for another day when it seems easier topush forward.

5. Battered faith. The constantly shifting prioritiesand mandates of school and district leaders haveeroded many teachers’ faith that it is worthwhile toassume the difficult task of raising and standardizingtheir expectations through formal and informalcollaboration. As one teacher confessed in utterfrustration: “I have been asked to write a differentcurriculum every year, with a different set of teachers,and a different theory of reading. This year, all theycare about is the test; all the other work we did lastyear doesn’t matter any more. Why should I exhaustmyself working for new goals when they will bechanged before we even get there?”

6. Low expectations. In its plainest form, pacingreflects how much teachers aim to accomplish on adaily basis. Teachers who allow blocks of time to gounused cannot make the same progress as those whoare more vigilant about getting started and gettingengaged. Similarly, teachers who set out to accomplishmore during their lessons typically do so. A second-grade class that takes 45 minutes to review sevenwords from the story of the Ugly Duckling does notaccomplish as much as a class that reviews these wordsin 15 minutes and then builds them into somesentences they write together.

3BetsAnn Smith, It’s about Time: School Development and Achievement in Chicago’s Elementary Schools (Chicago: Consortiumon Chicago School Research, 1998).

Setting the Pace: Opportunities to Learn inChicago's Elementary Schools

November 1998

Consortium on Chicago School Research


5Setting the Pace

Setting the Pace:Opportunities to Learn inChicago's Elementary Schools

Julia B. SmithUniversity of Michigan

BetsAnn SmithMichigan State University

Anthony S. BrykUniversity of Chicago

Introduction 1A Visit to Some Chicago Classrooms 3Instructional Opportunities to Learn 9Interpretive Summary 23Endnotes 25References 27

Consortium on Chicago School Research1313 East 60th StreetChicago, Illinois 60637773-702-3364773-702-2010 - fax

November 1998


7Setting the Pace

Introduction

Nineteen ninety-eight completes an important decade of the Chicago PublicSchools’ (CPS) evolving reform movement. The Consortium on ChicagoSchool Research has contributed to this process through a series of reportson school improvement and student learning. In this study, we turn ourattention to one of the most essential concerns of all: the quality of classroominstruction.

At issue here is a critical concern—the opportunities for Chicago’s chil-dren to learn all that is expected of them. We find that while some ChicagoPublic Schools (CPS) students experience instruction that keeps pace withgrade level expectations and test demands, many do not. Moreover, whilesome Chicago teachers are working to align their instructional programsacross classrooms and grades, others operate under widely different as-sumptions about what students should know and be able to do. Especiallytroublesome is the finding that students attending schools in Chicago’smost disadvantaged neighborhoods are much more likely to encounterinstruction that is poorly coordinated and that conveys weak expectationsfor student learning.

Fortunately, there is also good news in this report. We find that someChicago schools—including some that serve disadvantaged neighborhoods—present their students with a progression of challenging instructionalopportunities. In these schools, administrators and teachers are work-ing together to coordinate instructional programs that keep pace withnationally normed expectations for student achievement. Some of thedistinctive characteristics of these schools are also identified in this study.


AcknowledgmentsOur study of classroom instruction involved many researchers, but is particularly indebtedto Eric Camburn, JoAn Chun, Karen DeMoss, Russ Donofry, Gudelia Lopez, and LisaScruggs. It also involved the professional cooperation of hundreds of Chicago Public Schoolsteachers. We appreciate and applaud their willingness to forgo the privacy of their classroomsand to share their insights and experiences.

We would also like to acknowledge the contributions of Benjamin D. Wright, John M.Linacre, and Stuart Luppescu in guiding the development of the content pacing scale. Thiswork is linked to earlier research by Stuart Luppescu on equating multiple forms of the IowaTests of Basic Skills used by the Chicago Public Schools. We continue to be indebted to boththe leadership of the school system and the staff of the Office of Accountability for access todata and advice and assistance in its proper use. The CPS also facilitated the collection of thesurvey data used in this study.

We are grateful for the editorial and production expertise of Consortium staff: Kay KerschKirkpatrick, Sandra Jennings, and Rebecca Williams. The Consortium’s directors and theSteering Committee read and reread this report, and we thank them for their comments.

This report represents a culmination and synthesis of multiple lines of research that havebeen ongoing for several years. This would not have been possible without the sustainedfinancial support for this work from The MacArthur Foundation, The Joyce Foundation,and The Spencer Foundation. Some of the early classroom observation research also wassupported by the Illinois State Board of Education. We would like to thank Al Ramirez,former deputy superintendent, and Connie Wise, Division Administrator of Research andPolicy Development, for their assistance.

We hope this research will inform public understanding and discussion of the challengesteachers face and the supports they need to reach the student learning goals set by the IllinoisState Board of Education. Most important, we hope this research helps to improve the learn-ing opportunities and achievements of Chicago’s students.

9Setting the Pace

A Visit to Some Chicago Classrooms

Imagine yourself in the hallway of a Chicago public school. Pausing outsidea classroom door, you hear a teacher begin a math lesson by saying, “Todaywe are going to study parallelograms. A parallelogram is one example of apolygon with parallel sides.” Without looking into the room to see the ageof the students, what grade level would you guess is being taught? Secondgrade? Fifth grade? Eighth grade? Possibly even tenth grade? If you answeredany or all of the above, you were correct. During our observations of mathinstruction in Chicago, we observed students in all the above gradespresented with introductory lessons on the parallelogram. Each class beganwith either the teacher or a student reading the definition of a polygon anda parallelogram from a math textbook. All of the classes then discussed therequirement that parallelograms have parallel lines. The second-gradestudents went on to draw and measure the perimeter of parallelogramsusing a ruler. In all the other grades, teachers and students spent the rest ofclass learning to measure the angles inside a parallelogram.

When we began observing classes across primary, middle, and high schoolgrades, we did not set out to record and chart students’ exposure to topicand skill sequences. As we observed more and more classes, however, webecame increasingly concerned. We were most troubled when we saw verysimilar lessons taught to students several grades apart, as in the parallelo-gram example above, or the day when two researchers at the same schoolemerged from primary and middle school classrooms to find that they hadobserved comparable lessons on how to write a paragraph. In response tonumerous experiences like this, we began investigating how curriculumcontent progresses from grade to grade in the Chicago Public Schools.

We recognize the need to be careful in our judgments of these observa-tions. Many topics and skills can and should be revisited across students’school careers. Parallelograms can be studied in simple or complex ways,and writing paragraphs can always be improved. But we did not observesuch continuous development and refinement of learned concepts and con-tent. These were not cases of teachers choosing to explore a few topics indepth rather than many topics superficially. Rather, the same topics andskills were being taught again and again, with little to no development intheir content, depth, or complexity.


About Our Survey and Observation DataThe Consortium on Chicago School Research conducted a broad-based survey of Chicago PublicSchools elementary and high school teachers and students in 1994. These surveys covered a widerange of topics, including local school governance, school climate, opportunities for professionaldevelopment and work conditions of teachers, and the dynamics of instruction and classroom life.1

Teachers at each grade also responded to survey items concerning either reading or mathematicsinstruction in their classrooms. In the full survey, there were 6,264 teachers from 384 schools. Theanalyses in this study use information from 2,036 teachers in these schools who were full-timeclassroom teachers and who responded to the mathematics portion of the questionnaire. While thefull sample of teachers was used to generate school information on professional community, onlythe mathematics teacher responses were used in content pacing analyses.

On average, 12 teachers per school responded to questions about mathematics instruction, witha fairly even distribution of grades taught throughout. Most of the teachers who responded to thisportion of the survey were female (84 percent), a portion entirely consistent with the gender distri-bution of the elementary teachers in the city schools as a whole (85 percent female). In addition, 45percent of the teachers identified themselves as white, 40 percent as African-American, and 10percent as Latino/Hispanic, with the remaining 5 percent as Asian-American or “other.” Finally,about 45 percent of the teachers surveyed reported holding a master’s degree or more.

These survey analyses are complemented by classroom case studies that provide more in-depthunderstanding of instruction in the schools. From spring 1994 through spring 1996, a team ofresearchers observed over 800 language arts and mathematics classes in eight Chicago public el-ementary schools and seven high schools. Our sample of schools reflects the most common studentand community characteristics of the Chicago Public Schools.

The observational data in this report reflect the general instructional programs in the CPS. Wedid not observe lessons in magnet programs, honors programs, or remedial and special educationclasses. We observed language arts and mathematics instruction in grades two, five, eight, nine, andten, keeping detailed logs of teaching and learning activities.

Roughly half of the teachers we observed were new or mid-career teachers who had taught be-tween one and nine years. The other half were veteran teachers who had taught for ten or moreyears. The racial distribution of the observed teachers was similar to that of the district’s teacherforce, but included slightly more white teachers and slightly fewer African-American teachers. Nearlyall of the 250 teachers observed participated in one or more in-depth interviews about their teach-ing practices and programs. Observing instruction directly and talking with teachers personallyprovided data and insights that strengthened our findings and enriched our understanding.

All schools and teachers participated in the studies voluntarily. We aimed to record and under-stand rather than critique their work. This positive approach did not obscure the serious problemsand challenges of the general instructional programs of the schools.

1Consistent with the general philosophy of the Consortium as a public informing organization, all datacollected by the Consortium are made available to the public, subject only to concerns about preserving theconfidentiality of respondents. To access the data used in this study, see Bilcer, Luppescu, Sebring, andThum (1996). Contact the Consortium via the World Wide Web at http://www.consortium-chicago.org.

11Setting the Pace

Consider, for example, the observa-tion logs of a set of CPS language artsclasses that we visited (see Figure 1).These classes focused on literary con-cepts such as a story’s setting, maincharacter, and main events. We re-corded dozens and dozens of classesand assignments devoted to these top-ics. In many respects, these lessonswere entirely appropriate. Knowledgeof literary concepts is clearly stated asa learning goal for CPS students, and areview of basic story elements is a natu-ral opening to all kinds of literary dis-cussions. But we frequently observedhigh school students spending as muchtime defining and identifying these ba-sic concepts as their elementary counter-parts (see, for example, the left column inFigure 1), rather than simply noting themfor context and moving on to exploremore advanced literary concepts such asdialogue, figurative speech, hyperbole,satire, symbolism, or personification (seethe right column in Figure 1).

These contrasting examples point tokey problems in the organization of in-struction in the Chicago Public Schools.The students who experienced classeslike those described in the right-handcolumn were exposed to more literaryconcepts and skills than those who tookthe classes described in the left column.In the most basic sense, then, these stu-dents had more opportunities to learnbecause they were required to exploremore disciplinary content. Beyond this,the extra topics and skills studied werevaluable because they built on students’prior knowledge and made progressivelygreater intellectual demands of them. Asa result, these students’ opportunities tolearn were enhanced both by the factthat the instruction was more strongly

Observations of Chicago ClassroomsTeaching Literary Terms

Grade 2

Grade 5

Grade 8

Grade 9

Grade 10

Students begin with appro-priate lessons, but the con-cepts used to study and under-stand new literature do not build on prior knowledge as students progress through the middle grades and high school.

New and increasingly sophisti-cated literary concepts aresteadily added to students'exploration of literature as they advance through the middle grades and high school.

Read The Mitten. Identify andrecord the title, author, setting,and main characters. Review thestory and record events from the beginning, middle, and end ofthe story.

Identify the sequence of events in a story by looking for time-order words such as "first," "next," "then," or "the next day." Summarize the stories according to their main events.

Read Philip Hall Likes Me,I Reckon, Maybe. Do bulletinboard reports describing thetheme, main character, andsetting of the story.

Read The King Who Rained. Payattention to its use of idioms.List these idioms and write downwhat you think they mean.

Read And Now Miguel. Who isthe narrator? What is thesetting?

Select a novel to read and write about. Make note of its use of hyperbole. List some examples. How does hyperbole help us to write creatively?

Name some of the main char-acters of To Kill a Mockingbird.Where does the story takeplace? Who is your favoritecharacter? Write one paragraphexplaining why.

Select one Greek myth and ana-lyze it according to dimensions such as its hero, the call, the quest, the journey, helpers and guides, and transformation.

Read Miss Cynthie. Who is thenarrator? What is the setting?Describe Miss Cynthie in threesentences. Describe David inthree sentences.

Select one African fable and chart it according to the compo-nents of a traditional plot line. Discuss its use of the following: protagonists, antagonists, conflict, foreshadowing, flashback, and irony.

Slow Pacing/Constant Reteaching

Pacing Aimed atStudent Developmentvs.

Figure 1


The Instructional Depth vs. Coverage DebateThe topic of instructional pacing can easily raise confusion and criticism. Cross-national analyses ofcurriculum have criticized United States schools for excessive concern about covering an extensive arrayof content at the expense of in-depth student work on specific content.1 To be clear, the phenomenon weare observing in Chicago is not that of teachers taking time to engage their students in in-depth studiesof a smaller number of key disciplinary concepts. In fact, over 80 percent of the lessons we observedduring our field studies were rated as having little to no depth or complexity in the information trans-mitted or in the understanding students demonstrated.2 In truth, our observations and analysis suggestthat the “depth versus coverage” debate does not often apply to instruction in Chicago classrooms. Ourdata indicate that neither occurs with much frequency. The pacing lag we identify seems an outcome ofheavy topic review and repetition unaccompanied by any increase in instructional depth.

Our concern is not that instruction be mindlessly speeded up or that more is necessarily better.Instead, we believe that students should experience a sequence of instruction that exposes them ina systematic and developmentally challenging fashion to the content on which they are tested.

The Chicago case we illustrate here may well be an acute example of problems associated with math-ematics programs in other urban school systems and, indeed, the country as a whole. In the most recentreport of the Third International Math and Science Study, U.S. math classrooms were notable for thelimited amount of time they gave to introducing new material to their students (about 10 minutes permath lesson) and a persistent focus on basic skills. To quote from the report, “U.S. mathematics instruc-tion was centered on computation and homework. It focused comparatively less on new instruction andmore on review. These practices are consistent with a view of school mathematics as made up of discretebits, to be mastered cumulatively and incrementally, for which constant review is necessary...”3

The importance of more in-depth learning opportunities in the Chicago Public Schools is a seriousconcern. We examine this aspect of instruction in a companion study, titled The Quality of IntellectualWork in Chicago Schools: A Baseline Report.4

1See, for example, studies by Stevenson and Stigler (1992) and Schmidt et al. (1998) for TIMSS curriculumanalysis reporting.

2Ratings of in-depth learning were made using rubrics developed at the Center on Organization and Restructur-ing of Schools by Newmann and Wehlage (1995).

3See Schmidt et al. (1998), p. 19.

4See Newmann, Lopez, and Bryk (1998).

13Setting the Pace

Reading and Exploring a NovelLiterature Progresses, but Intellectual Activities Do Not

Fifth Grade: Charlotte's Web Tenth Grade: Bless Me, Ultima

Students read much of thebook aloud in class.

Students read much of thebook aloud in class.

Students take short testsand quizzes with who,what, and where questionsto review the facts of thestory.

Students take short testsand quizzes with who,what, and where questionsto review the facts of thestory.

The teacher selects thetheme of friendship as afocus of discussion andwriting.

The teacher selects thetheme of superstition asa focus of discussion andwriting.

Students write a shortessay on friendship. They write one draft only.

Students write a shortessay about superstition.They write one draft only.

paced and because lessons were morethoughtfully developed over time.

Figure 2, which documents how afifth-grade and a tenth-grade class ex-plored a novel, offers another view ofthese problems. We observed bothclasses reading high quality, grade ap-propriate works of literature: Charlotte’sWeb in the fifth grade, Bless Me, Ultimain the tenth. In this sense, we could saythat everyone was working at gradelevel. But the assignments given to thetenth-grade students were so much likethose asked of the fifth graders that theolder students’ opportunity to learn wasgreatly minimized. Both classes spentmost of their lesson time reading thenovels aloud in class and answering ba-sic comprehension questions posed bytheir teacher: “What was the rat’s re-sponse to Charlotte’s question?” “Whatdid the grandmother hang over thedoorway?” Both classes had a theme,selected by the teacher, that was tracedthrough the chapters. And both classesof students were given several weeks towrite one draft of a short report relatedto this theme. While these assignmentsseemed reasonably challenging for fifth-grade students, they do not come closeto meeting standards for high schoollearning and achievement. For example,tenth-grade students should read moreat home. They should move beyond thebasic facts of their novel to interpret theauthor’s purpose and message and todiscuss their own thoughts on the story.We also expected significant differencesin the writing assignments given to stu-dents and the standards used to assessthem.

The examples in Figures 1 and 2 arefrom language arts lessons, but we couldalso illustrate these points with data

Figure 2

from the math classes we visited (see Repeat Offenders sidebar). Many ofthe lessons we observed reflected confused or dampened expectationsfor student learning. Indeed, some lessons completely defied expecta-tions when, for example, elementary lessons were more demanding thansome of the middle grade and high school lessons we saw. As our obser-vations of these phenomena accumulated, we began to question whatwas actually guiding teachers’ instructional practice. When, for example,are students expected to know and use the literary concept of a story’ssetting without further instruction? It became clear that, in at least someof the schools we studied, faculty members lacked a shared conceptionof the instructional program overall, and of their own particular set ofresponsibilities for advancing it.


Repeat OffendersSome topics and learning goals seem to have a clear time and place in the schools’ instructional program. Forexample, we consistently observed capitalization taught in the second grade, the metric system in the eighthgrade, and Romeo and Juliet in the ninth. But many other topics do not appear to have a clear time and place.Below are some of the subjects and skills we saw repeated or stretched out across as many as five differentgrades.

Language ArtsUnderstanding fiction versus nonfictionIdentifying and defining setting, main character, and main eventsHow to create a story boardHow to write a formal letterThe eight parts of speechHow to use the commaHow to write a summary paragraphThe structure of a five-paragraph essayHow to write a footnote or bibliography

Additionally, many language arts activities that necessarily repeat do not necessarily develop. The mostcommon examples observed were class presentations, three-minute speeches, summarizing a book by mak-ing a book jacket, and writing a book report. The expectations and standards applied to these products donot systematically rise as students move through middle and high school grades.

MathMoney problemsParallel and perpendicular linesPlace value, 1-1000DecimalsArea and perimeter of polygons and trianglesLong divisionFractions, proportion, and percentHow to use a number lineRight anglesRadius, diameter, and circumference

Both Math and Language ArtsHow to read, create, and color bar graphs and pie chartsHow to read street maps and transportation schedules

15Setting the Pace

Instructional Opportunities to Learn

Our in-depth field studies raised critical concerns about how some Chicagoschools organize and pace instruction and how this structure affects students’opportunities to learn. Now, we turn to a system-level analysis of theseissues. While we could not create a systemwide database of direct classroomobservations, we were able to gather survey reports from Chicago PublicSchools teachers of the content they taught their students. These dataallowed us to investigate the prevalence of the problems identified in thefield work and how they were distributed among the diverse schoolcommunities that comprise the CPS.

Put simply, we wanted our analysis to inform us whether Chicagostudents are given the opportunity to learn all of the content for whichthey are held accountable. We focused our inquiry on mathematicsbecause math instruction is more commonly approached as a system-atic presentation of topics that build upon prior knowledge and thatgrow in complexity.1 Accordingly, it was easier to develop reliable sur-vey items that ask teachers to report their math content emphases foreach elementary grade level.

We organized this research around the de facto math standards of theCPS—the subject matter content of the Iowa Tests of Basic Skills (ITBS).The ITBS has been used by the CPS as a measure of student achievementfor more than two decades but has taken on heightened prominence in thelast few years. Student performance on these tests is now the primary in-formation base for an increasingly high-stakes student accountability sys-tem. Among other uses, students’ test results are now used to mandatesummer school attendance, to deny grade promotions, and to delay entryinto the city’s regular high schools.

Before we analyzed teacher surveys, however, it was necessary to con-duct an extensive content analysis of the actual math items in the ITBS.2

Each test item or question was grouped into one of 48 categories. Thesecategories sort test items according to both mathematic concepts (e.g.,addition, geometric congruence, ratio) and overall difficulty (difficulty levelsreflect the percent of students at a given grade level able to answer an itemcorrectly).3 Though individual items associated with different mathematic


Fig

ure

3G

rad

e L

evel

Mas

tery

Ru

ler

in t

he

Iow

a Te

sts

of

Bas

ic S

kills

Bey

on

d

8th

Gra

de

8th

Gra

de

7th

G

rad

e

6th

Gra

de

How

muc

h w

ater

mus

t be

adde

d to

50

ml o

f 20

% a

cid

to p

rodu

ce a

mix

ture

whi

ch is

12%

ac

id?

Iden

tify

the

root

s of

:

2 +

4 x

- 5

Sol

ve:

5A +

3B

= 1

22A

–

5B =

9

If th

ere

are

40 r

ed b

alls

and

30

blac

k ba

lls, w

hat i

s th

e pr

obab

ility

of

get

ting

2 re

d ba

lls in

a s

ingl

e dr

aw?

Wha

t is

the

equa

tion

of a

line

cont

aini

ng th

e po

ints

(2,

1) a

nd(-

1, 2

)?

Sim

plify

:

2(3

x +

1)

x (

5 x

+ 2

)

Ske

tch

a gr

aph

of:

y =

4 –

4x

If 12

can

s co

st$3

.00,

how

muc

hw

ill 4

0 ca

ns c

ost?

3.4

- .6

01 =

?W

hat i

s 12

%of

80?

-34

– 21

= ?

4 =

?2

3

9

4

+

=?

- 5

+

1 =

?

645

73

+ ?

< 9

Whi

ch s

how

s th

eco

mm

utat

ive

pro-

pert

y of

add

ition

?

Wha

t wou

ld 6

49be

rou

nded

toth

e ne

ares

t ten

?

Wha

t pla

ce is

the

5 in

3.0

052?

Wha

t are

the

prim

e fa

ctor

s of

85?

How

man

ym

eter

s ar

e in

10 y

ards

?

Qu

adra

tic

Eq

uat

ion

sS

yste

ms

of

Eq

uat

ion

s

Sta

tist

ics

and

Pro

bab

ility

Eq

uat

ion

s o

f L

ines

Sim

plif

y A

lgeb

raic

E

xpre

ssio

ns

Gra

ph

Eq

uat

ion

s

Rat

io, P

rop

ort

ion

Per

cen

t P

rob

lem

sO

per

atio

ns

wit

hN

egat

ives

Exp

on

ents

,R

oo

tsO

per

atio

ns

wit

hF

ract

ion

sA

bso

lute

Val

ue

Lo

ng

Div

isio

nS

olv

eIn

equ

alit

yN

um

ber

Pro

per

ties

Ro

un

din

gD

ecim

alP

lace

Val

ue

Pri

mes

, Fac

tors

,M

ult

iple

sC

onv

erti

ng

Mea

sure

men

t

Iow

a Te

sts

of B

asic

Ski

lls it

ems

repr

inte

d w

ith p

erm

issi

on fr

om R

iver

side

Pub

lishi

ng

Not

e: A

ll of

thes

e ar

e m

ultip

le c

hoic

e qu

estio

ns in

whi

ch th

e st

uden

t is

aske

d to

iden

tify

the

corr

ect a

nsw

er fr

om fo

ur p

ossi

bilit

ies

give

n. B

ecau

se o

fsp

ace

limita

tions

, we

pres

ent o

nly

the

ques

tions

.

Ad

van

ced

Wo

rd P

rob

lem

s

3

x

3+(2

+4)

= (

3+2)

+4

3+2+

4 =

4+

3+2

Op

erat

ion

s w

ith

D

ecim

als

17Setting the Pace

5t

hG

rad

e

4th

Gra

de

3rd

G

rad

e

2nd

Gra

de

Ap

pro

xim

atio

nA

rea

Volu

me

Mu

lti-

dig

itM

ult

iplic

atio

n

1st

Gra

de

Sim

ple

Div

isio

nP

lace

Val

ue

and

Reg

rou

pin

gF

ind

ing

Len

gth

,P

erim

eter

Sh

ifti

ng

Bet

wee

nW

ord

s an

d N

um

ber

sW

ord

Pro

ble

ms

wit

hM

ult

iplic

atio

n, D

ivis

ion

Mu

lti-

dig

itA

dd

itio

nM

ult

iplic

atio

nFa

cts

Mu

lti-

dig

itS

ub

trac

tio

nM

easu

rin

gC

om

mo

n O

bje

cts

Co

un

tin

g b

yn

>1

Co

nver

tin

g U

nit

so

f Tim

e

Co

ins,

Val

ue

of

Mo

ney

Telli

ng

Tim

eU

sin

g M

od

els

for

Op

erat

ion

sU

sin

g N

um

ber

Lin

esan

d R

ule

rs

Co

un

tin

gO

bje

cts

Iden

tify

Sh

apes

Nam

e, O

rder

Nu

mb

ers

Sim

ple

A

dd

itio

nS

imp

leS

ub

trac

tio

nId

enti

fyO

per

atio

ns

Wo

rd P

rob

lem

sw

/Ad

d, S

ub

trac

t

Wha

t por

tion

is s

hade

d?2

( ?

+ 4

) =

10

Wha

t num

ber

is th

e cl

oses

tto

the

sum

of 2

98 a

nd 3

004?

Wha

t is

the

area

of t

he c

ircle

2"

32 x

45

= ?

25

5 =

?B

y ho

w m

uch

will

the

valu

e of

the

num

ber

chan

ge if

the

4 is

chan

ged

to a

5 in

742

?

How

long

is th

e pe

rimet

er o

fth

e fe

nce

show

n?

4'

3'1'

2'

How

do

you

writ

e 2,

340

in w

ords

?If

one

penc

il co

sts

15 c

ents

,ho

w m

any

can

you

buy

for

$3.0

0?

43

+ 1

9 ?

Wha

t is

3 x

5? 6

7-

8 ?

How

tall

is th

e do

or?

Fill

in th

e m

issi

ng n

umbe

r3,

6,

? ,

12

How

man

y se

cond

s ar

ein

2 a

nd a

hal

f hou

rs?

Jan

has

3 qu

arte

rs a

nd 2

dim

es. H

owm

uch

mon

ey is

that

?

Wha

t tim

e is

sho

wn?

12

39

6

Wha

t pic

ture

sho

ws

that

hal

f of 6

is 3

?U

se th

e ru

ler

to fi

nd th

e le

ngth

of

the

line.

1+

2+

(spo

ken)

How

man

y?W

hich

is a

circ

le?

(spo

ken)

Whi

ch n

umbe

r is

ei

ghte

en?

8

81

18

(spo

ken)

Wha

t is

4 +

4?

8 -

6 =

?

Wha

t sig

n te

lls y

ou to

subt

ract

?

Ric

k ha

d 9

cook

ies

and

ate

3. H

owm

any

did

he h

ave

left?

. .

,

3200

33

00

1'

Siz

es o

f F

ract

ion

sS

olv

e fo

r O

ne

Un

kno

wn


concepts can overlap in their level ofdifficulty, the elementary math con-tent of the ITBS generally arrangesitself in a progression of concepts thatbuild on one another and grow in dif-ficulty. Figure 3 lays out this sequenc-ing of item content by difficulty. Theruler is anchored in national mathnorms at each grade level and identi-fies the concepts and skills that stu-dents are expected to master year byyear. These math norms are based onactual test results from a nationalsample of students. Notice that thebottom end of the scale assesses mas-tery over relatively simple mathemat-ics concepts of “counting objects,”“simple addition,” and “telling time.”In the mid-range, a student perform-ing on level for fifth grade, for ex-ample, should show competence withsuch skills as “sizes of fractions,” “area,volume,” and “multi-digit multiplica-tion.” (This student should also dem-onstrate competence in the simpleskills that appear below fifth gradeon the ruler, such as “simple divi-sion.”) At the very top of the scale,beyond eighth grade, are algebraicitems that assess mastery over suchtopics as “systems of equations” and“quadratic equations.”

Overall, the math ruler providesa general picture of the grade-to-grade organization of instructionassumed in the ITBS. These are thede facto standards against whichCPS students are being judged bythe accountability system.

Next we turned to elementaryschool teacher survey reports to ex-amine whether students actually havethe opportunity to learn this content.Specifically, elementary teachers were

Pacing of Math Curriculum Slows Downin Upper Grades

Grade Taught

Gra

de L

evel

Con

tent

Em

phas

is

K 1st 2nd 3rd 4th 5th 6th 7th 8th

9th

8th

7th

6th

5th

4th

3rd

2nd

1st

K

Grade-level pacing line

How to Read a Box PlotEach box encloses the middle 50 percent of the teachers' respon-ses. The broken line inside the box is the median; half of the schools are above this line, half are below. The lines extending up and down from the box, called "whiskers," show the top and bot-tom 25 percent of the teacher content coverage reports.

Note: The grade-level pacing line indicates that content coverage is keeping pace with grade level learning, based on equating teachers' reports of content coverage with the Iowa Tests of Basic Skills.

Figure 4

Note: The grade-level pacing line indicates that content coverage iskeeping pace with grade level learning, based on equating teachers’reports of content coverage with the Iowa Tests of Basic Skills.

How to Read a Box PlotEach box encloses the middle 50 percent of the teachers’responses. The broken line inside the box is the median; halfof the schools are above this line, half are below. The linesextending up and down from the box, called “whiskers,” showthe top and bottom 25 percent of the teacher content cover-age reports.

19Setting the Pace

asked to indicate the grade level oftheir class, the particular math con-tent covered that year, and the rela-tive amount of time spent on eachcontent area. Using these data, wewere able to create a scale of con-tent taught, comparable to the ITBSmeasurement ruler in Figure 3. Thisallowed us to directly compare thecontent coverage by Chicago teach-ers with the content students are ac-tually tested on by the ITBS.4 Thiscomparison, in turn, enabled us toassess whether Chicago students areafforded an adequate opportunity tolearn the material for which they arenow held accountable.

The dark black line in Figure 4identifies the results expected if teach-ers’ content emphases align with ITBSgrade level mastery.5 In other words,the line represents math content thatwe should expect teachers to teach ifstudents are being given the opportu-nity to learn what is being tested onthe ITBS. The box plots indicate therange of content emphases actuallyreported by Chicago teachers at eachgrade level. At second grade, for ex-ample, more than half of the teachersindicate content coverage consistentwith (or above) grade level mastery(that is, their content emphases scoreat or above the heavy black line).About a quarter of those teachers re-port instruction well above grade level,moving into the third to fourthgrade mastery range. In contrast,three-quarters of the teachers ateighth grade report content empha-ses below the grade level line, withthe bottom quarter of eighth-gradeteachers indicating a content focusequivalent to about grade three.

The most immediate impression here is that teachers’ own accountsvalidate our classroom observations. Teachers are reporting in our surveyswide variability in students’ exposure to subject matter and frequent rep-etition of topics across one or more years. While some classrooms are work-ing on content consistent with student achievement at or above nationalnorms, the vast majority are not.

We have used the teacher survey reports on content emphases to for-mulate two key indicators of each school’s organization of instruction.

Figure 5

About Half of CPS Schools Maintain Grade-Level Pacing;the Other Half Fall Far Behind

Top 10 percent of schools

Top 50-90 percent of schools

Bottom 10 percent of schools

Gra

de L

evel

Con

tent

Em

phas

is


9th

8th

7th

6th

5th

4th

3rd

2nd

1st

K

Bottom 10-50 percent of schools

Grade Taught

▲

▲

▲

▲

▲

▲

▲

▲

▲

**

* **

* ** *


▲

*


Instructional PacingBy combining the survey responses ofmathematics teachers within a school,we can compute an instructionalpacing trajectory for that school.6

Figure 5 illustrates the range of pacingwithin the Chicago school system. Wesee that in most Chicago elementaryschools, kindergarten teachers reportinstructional emphasis consistent withor above ITBS norms. Moreover,between kindergarten and second grade,teachers in most schools continue toreport a steady introduction of newtopics (as evidenced by the steady climbof their pacing lines), keeping close toor exceeding the ITBS expectations.

Content coverage starts to differen-tiate at third grade. The mathematicscontent tested on the ITBS continuesto advance, yet the pacing of roughlyhalf of Chicago’s schools begins to flat-ten out and fall seriously behind. In-deed, many schools at or above ITBSexpectations in primary school are al-most a year behind by the fourthgrade. By eighth grade, many schoolscontinue to teach material from theprimary and middle grades curricu-lum. Indeed, less than 50 percent ofChicago’s schools keep pace with theITBS during the last three years of el-ementary schooling.

Unfortunately, we do not have thesame kind of data for the high schoolTAP test as we do the ITBS, so wecannot continue this graph throughthe upper grades. But our field obser-vations of hundreds of high schoolmath lessons provide considerable evi-dence that this pacing lag continuesthrough high school, seriously dimin-ishing students’ math achievement andtheir preparation for higher education.

One indicator examines the rate at which teachers in a school introducenew and more difficult subject matter to their students. We call this indi-cator instructional pacing. Instructional pacing is key to understandingwhether topics are being introduced at a rate consistent with the demandsand expectations of the ITBS. Second, we look at the coordination of in-structional content within each school grade, which we call grade-level co-herence. Is there, for example, a distinctive fifth-grade curriculum, or doeseach fifth-grade teacher “do her own thing”? Both good instructional pac-ing and grade-level coherence are necessary to provide students with theopportunities to learn what the ITBS tests.

Pacing Slows Down Dramatically in High-Poverty Schools

Mixed-income schools

High-poverty schools

Gra

de L

evel

Con

tent

Em

phas

is


9th

8th

7th

6th

5th

4th

3rd

2nd

1st

K

Note: These two groups represent the highest and lowest quartiles in terms ofpercent low-income enrollment in the CPS. The mixed-income schools have less than 50 percent poverty. High-poverty schools enroll greater than 90 per-cent low-income students.

Grade Taught


Note: These two groups represent the highest and lowest quartiles in termsof percent low-income enrollment in the CPS. The mixed-income schoolshave less than 50 percent poverty. High-poverty schools enroll greaterthan 90 percent low-income students.

Figure 6

21Setting the Pace

Effects of basic school character-istics. Our concern that all studentsbe provided opportunities to learn thecontent on which they are tested ledus to investigate whether certain typesof Chicago schools are more pronethan others to slow pacing. We exam-ined the effects that a variety of schooland community characteristics mighthave on the pace of instruction. Thesefactors include the prior achievementlevels of the school, racial composi-tion, percent low-income enrollment,school size, and rates of student mo-bility. Not surprisingly, we found thatcurriculum pacing was positively re-lated to the school’s achievement his-tory. Many selective academic magnetschools, for example, have pacing tra-jectories well above the ITBS nationalnorm reference line.

More telling is what we find aftertaking prior achievement into ac-count. Even when we control forprior achievement, the percent oflow-income students enrolled in aschool influences instructional pac-ing.7 To illustrate this effect, Figure6 compares instructional pacing tra-jectories from high-poverty schools(where more than 90 percent of thestudent body comes from low-incomefamilies) with more mixed-incomeschools (where less than 50 percentof the student body comes from low-income families). Chicago studentswho attend schools with high con-centrations of poverty experienceslower instructional pacing that ba-sically flattens out around fourthgrade. While this repetition mayhelp reinforce primary skills, it alsolimits students’ exposure to new top-ics, including topics on which they

Figure 7

Pacing Is Fastest in Integrated Schools and Slowest inPredominately Minority and African-American Schools

Integrated schools (over 30% white)

Predominately Hispanic schools(over 85% Hispanic)

Gra

de L

evel

Con

tent

Em

phas

is


9th

8th

7th

6th

5th

4th

3rd

2nd

1st

K

Racially mixed schools (15% to 30% white)

Predominately minority schools(over 85% mixed minority)

Predominately African-American schools(over 85% African-American)

Grade Taught

*

▲

x

*

▲

x


▲

▲

▲

▲

▲

▲

▲

▲

x

x

x

x

x

x

x x

***

**

* *

*


Equally troubling are the relation-ships between pacing and the racialcomposition of the school (see Figure7). Though the pattern varies somefrom grade to grade, integrated schoolsare the most likely to keep pace withITBS norms, followed by Chicago’s ra-cially mixed schools and predomi-nately Hispanic elementary schools.Predominately minority and African-American schools are the most likelyto suffer slow pacing that lags behindITBS demands.9 These findings echothe results of other Consortiumstudies, which have suggested thatmany of Chicago’s most racially iso-lated schools, especially its African-American schools, have been left behindby reform. These schools continue to facea very steep climb to national norms.10

The evidence in Figures 6 and 7implies that students’ social back-ground has a great deal to do with howmuch material they are given and ex-pected to learn. This gap between whatis taught to students from differentfamily and community backgroundsreveals major differences in students’opportunities to learn within the Chi-cago Public Schools. This finding iscause for serious concern. A significantbody of research has demonstrated thataccess and exposure to math contenthas clear and predictable effects on theachievement of students. Regardless offamily and community background,students who are exposed to moremath content and an overall richermath curriculum have higher levelsof math achievement than peerswhose learning opportunities aremore limited.11

More troubling still, we find thatsuch students are further disadvan-

will be tested. Other inquiries into school life suggest that slow pacingmay also leave students unmotivated and resentful toward school.8 And,as mentioned, our observations found no evidence that slower pacingwas contributing to more in-depth teaching and learning. The class-room life of these students appears to consist of repetitive cycles ofbasic skills instruction.

In contrast to what we found in predominately high-poverty schools,students in schools with mixed-income populations typically follow amath curriculum that keeps pace with the ITBS. Students in theseschools are introduced to significantly more topics and skills in eachgrade than are the students in high-poverty schools. The pattern be-comes apparent between fourth and fifth grade, and the gap grows largerwith each successive year.

Small Schools Are More Likely To Keep Pace

Small schools (enrollment less than 350 students)

Larger schools (enrollment greater than 750 students)

Gra

de L

evel

Con

tent

Em

phas

is


9th

8th

7th

6th

5th

4th

3rd

2nd

1st

K

Grade Taught


Figure 8

23Setting the Pace

taged by other organizational charac-teristics common to their schools. Forexample, Chicago’s high-povertyschools tend to be larger, with morestudents in each grade level. The av-erage school enrollment in a CPS el-ementary school serving a more than90 percent low-income student bodyis 676 students. In contrast, schoolsserving a mixed-income student bodyhave an average enrollment of 435 stu-dents. High-poverty schools also havemuch higher levels of student mobil-ity (34 percent) than mixed incomeschools (10 percent). Our analysesshow that both of these factors—school size and the level of student mo-bility in and out of schools—are alsoassociated with instructional pacing(see Figures 8 and 9 respectively). Thatis, even after we control for the effectsof prior academic achievement andstudent poverty, we still find signifi-cantly stronger pacing in smallerschools and in schools with lower lev-els of student mobility. These findingsillustrate how family and communitypoverty interact with aspects of theschool organization to limit studentlearning opportunities.

An organizational antidote:school-based professional commu-nity. We also considered in ouranalysis how a variety of school or-ganizational characteristics mightinfluence the pace of instruction.12

One organizational characteristicstood out as especially important:the level of professional communityamong teachers. As we have elabo-rated in earlier Consortium reports,three core activities characterizeadult work in a professional commu-nity. First, teachers regularly engage

in reflective conversations that hold teaching practices and student per-formance up to scrutiny. Second, to substantiate these conversations, teach-ers open their classroom doors to share their work with peers and to lend apublic character to their teaching. This openness leads to a third key char-acteristic—peer collaboration. Through shared work, teachers learn fromone another and continue to develop the skills, knowledge, and ideas nec-essary for continuous school improvement. Undergirding these professionalbehaviors is a set of shared beliefs and values, central to which is a collec-tive responsibility for student learning. In sum, schools with strong profes-sional communities provide a normative environment that guides adultbehavior and promotes strong commitments to the welfare and educationof students.13

Note: Schools in the lowest quartile, low-mobility schools, have less than19 percent mobility. The top-quartile (high-mobility) schools experiencemore than 34 percent mobility.

Schools with High Mobility Have Pacing Problems

Low-Mobility Schools

High-Mobility Schools

Gra

de L

evel

Con

tent

Em

phas

is


9th

8th

7th

6th

5th

4th

3rd

2nd

1st

K

GradeTaught


Figure 9


munity—students pay a price. Whileit doesn’t significantly affect studentsin kindergarten through fourth grade,students in succeeding grades experi-ence slower curriculum pacing.

It is important to emphasize thatthis difference remains even when vari-ous aspects of students’ background aretaken into account. To demonstratethe independent influence of profes-sional community on pacing, Figure11 presents results from an analysis ofhigh-poverty schools only. Althoughthe pacing trajectories for these high-poverty schools are somewhat weakerthan the system as a whole, we againobserve a large pacing gap betweenschools that have strong professionalcommunity and those that do not.

In short, it isn’t just the back-grounds of the students that slowsdown instruction, it is also the wayteachers do or do not work togetherin the school. Our results indicate thatwhen teachers do not collaborate intheir curriculum development andteaching, do not regularly confer withone another, and do not reflect on theirpractices and outcomes, slower pacingwith more extensive repetition and re-view is likely to ensue.Conversely, schools with a strongprofessional community are places inwhich the faculty seem much moreable to pursue grade-level learninggoals. This positive relationship offerspromise for schools seeking to developbetter pacing and alignment of theirinstructional programs. While there isno precise recipe for how to developteachers’ professional community, itskey building blocks—common goals,collaborative work, frequent dialogue

We found strong relationships in our analysis between the instructionalpacing of a school and its level of professional community. Figure 10 showsthe results. In this figure, we compare the pacing trajectories of the CPSschools with the strongest measures of professional community (the top25 percent) to those with the weakest measures (the bottom 25 percent).Notice that students who attend an elementary school where faculty enjoystrong professional community tend to receive instruction closely alignedwith the ITBS math content standards (i.e., consistently at or above theITBS grade-level mastery line). On the other hand, in schools where teacherswork in isolation and do not collaborate—i.e., a weak professional com-

Schools with Strong Professional CommunityReport Faster Pacing

Strong professional community schools

Weak professional community schools

Gra

de L

evel

Con

tent

Em

phas

is


9th

8th

7th

6th

5th

4th

3rd

2nd

1st

K

Note: Strong professional community schools comprise the top 25 percent of the CPS elementary schools on the professional community index. Weak professional community schools comprise the bottom 25 percent of the CPS elementary schools on the same index.

Grade Taught


Figure 10

Note: Strong professional community schools comprise the top 25percent of the CPS elementary schools on the professional commu-nity index. Weak professional community schools comprise the bot-tom 25 percent of the CPS elementary schools on the same index.

25Setting the Pace

about improving practices, and a strongsense of collective responsibility—provide the needed social and structuralsupports for teachers to tackle pacingand alignment problems.

Grade-Level CoherenceA school’s instructional pacingtrajectory describes the overallopportunity to learn afforded by aschool as students pass through theelementary grades. This is a school-wide average. Any individual student’sexperience, however, consists ofexposure to a specific set of classroomsacross grade levels. We note thisdistinction because we often found inour field observations considerablevariability in instruction amongclassrooms at the same grade. In suchsituations, a school’s average pacingtrajectory might appear appropriate,but it would not necessarily indicatethat all students attending that schoolexperience consistent instruction. Astudent may be in a classroom that iswell below average one year and aboveaverage the next. For example,consider again the sequence of literacylessons we presented in Figure 1.Students may not move straight downthe columns, but back and forth fromone side to the other as they progressthrough the grades. Such studentsmight lose time studying topics theyalready know one year and encountertopics they are entirely unprepared totackle the next. The more variable thecontent coverage is within any givengrade, the more likely it is thatstudents will receive incoherentinstruction as they pass from grade to

grade.14 Examining the grade-level coherence of instruction allows us toconsider the likelihood that students experience these problems.

We returned to teachers’ content emphasis reports and computed ameasure of the similarity in math content coverage among teachers bygrade level. If most teachers working in the same grade report that theycover a similar set of math topics, their school’s measure of grade-levelcoherence would be high. In contrast, if these teachers’ reports withingrade levels do not overlap much, their measure of grade-level coher-ence would be low.15

Strong Professional Community Improves CurriculumPacing in High-Poverty Schools

Strong professional community schools

Weak professional community schools

9th

8th

6th

Gra

de L

evel

Con

tent

Em

phas

is


7th

5th

4th

3rd

2nd

1st

K

Note: These results are for CPS elementary schools that enroll 90 percent or morelow-income students.

Grade Taught


Note: These results are for CPS elementary schools that enroll 90 per-cent or more low-income students.

Figure 11


Average Pacing and High Coherence

Figure 12a Figure 12b

To illustrate the magnitude of grade level incoher-ence, we grouped together all of the schools with ap-proximately average instructional pacing trajectories andthen examined their grade-level coherence.16 Figure 12asummarizes teachers’ reports of content coverage fromschools in the top quartile in grade-level coherence; Fig-ure 12B summarizes reports from the bottom quartile.Notice that the box plots in Figure 12a indicate rela-tively little spread in teacher reports about content em-phases at each grade level. There is also a steady upwardtrend in the box plots across most grades, demonstrat-ing that teachers in these grades are presenting new andmore difficult topics each year.

As a result, the students in this first group of schoolsare likely to be receiving a coherent program of instruc-tion over time. In contrast, the box plots in Figure 12bdemonstrate extensive variability among classes at thesame grade level. As a result, some students in theseschools might, for example, receive the third-grade cur-riculum in two successive years; others might study the

third-grade curriculum one year and then enter a classthe next fall aimed at fifth-grade material. In short, bothgaps in instruction and repetition of topics may well beexperienced by these students.

Effects of school composition and organizationalfactors. As was the case for instructional pacing, wefound that school composition and organizational char-acteristics had a substantial effect on grade-level coher-ence. Again, even after we controlled for prior schoolachievement, we found that schools with high propor-tions of impoverished families are much more likely tohave incoherent curricula, as are large schools and schoolswith high levels of student mobility. On the brighterside, the positive relationship demonstrated between astrong school-based professional community and instruc-tional pacing also extends to grade-level coherence. Co-herence is more likely in schools where teachers workcollaboratively and share collective responsibility for stu-dent learning.

Average Pacing and Low Coherence

Comparison of Schools with High and Low Within-Grade Instructional Coherence

Grade Taught

Gra

de L

evel

Con

tent

Em

phas

is


7th

5th

4th

3rd

2nd

1st

K

6th

8th

9th


Grade Taught

Gra

de L

evel

Con

tent

Em

phas

is


7th

5th

4th

3rd

2nd

1st

K

6th

8th

9th


27Setting the Pace

Across all our analyses, we find that the weak pacingand coherence that limits students’ opportunities to learnpersistently characterize many of the system’s high pov-erty schools. The patterns here are quite vivid: in someof the math programs we examined, the pace of learningslows to such a great extent that little to no new materialis introduced across the last several years of elementaryschooling. This result means that tens of thousands of

Chicago students attend schools that fail to provideequitable, grade-appropriate opportunities to learn.Moreover, our extensive classroom observations indicatethat this pattern exists in language arts instruction aswell. This finding leaves us with a major challenge: Ifstudents are not exposed to the knowledge and skills theyare tested on, how then are they to achieve the compe-tencies expected of them?

A Growing Research Base Documents the Effects ofProfessional Communities

The findings on professional community presented in this report add to a growing body of literature on itsimportance to urban school improvement. The Consortium first introduced the concept of school-basedprofessional community in its 1995 report, Charting Reform: Chicago Teachers Take Stock. Early work on thisidea emerged out of a collaboration with colleagues at the federally funded Center on Organization andRestructuring of Schools at the University of Wisconsin-Madison.1 About the same time, closely related ideaswere being developed at the federally funded Center on School Context at Stanford University.2

Over the last five years, evidence has emerged confirming the benefits of schools organized around theprofessional norms we have described. For example, Newmann and Wehlage found teachers’ professionalcommunity to be a critical characteristic of restructured schools that successfully engage students in challeng-ing intellectual work.3

In our previous research, we found that weak school-based professional community characterizes the low-est achieving elementary schools in Chicago, as well as most of the city’s high schools.4 In contrast, Chicagoschools with strong professional communities are more likely to have school climates focused on studentlearning. Evidence is also beginning to appear that strong professional community among teachers may playa key role in sustaining long-term efforts to improve student achievement.5

1See Kruse, Seashore-Louis, and Bryk (1995).

2See Talbert and McLaughlin (1994).

3See Newmann and Wehlage (1995). See also McLaughlin and Talbert (1993).

4See Sebring et al. (1995).

5See Designs for Change (1998). See also Bryk, Sebring, et al. (1998).


29Setting the Pace

Interpretive Summary

Every day Chicago teachers have to juggle dozens of considerations aboutthe young people they teach and the learning goals they strive to reach.Unforeseen events arise; students fail to grasp new material in the allottedtime. But what happens when the juggling, borrowing, and repeatingextends past one or more school years? Our research argues that unalignedand incoherent instructional programs emerge. Students who pass throughthese programs experience delays, repetitions, and/or skips in coreknowledge and skills in ways that seriously diminish their chances for successin school and, in particular, on the tests used to measure their knowledgeand their progress.

A First-Step: Learning StandardsIn the current debates on school improvement, the reform most directlyassociated with instructional pacing and coherence is the standardsmovement. Indeed, both the Illinois State Board of Education and theChicago Public Schools have issued official learning goals and standardsthat articulate what students are expected to know at various grade levels.The Chicago Public Schools are even working to develop daily lesson plansthat illustrate these standards. In our view, well designed and properlypromoted learning standards can assist teachers in reorganizing theirteaching and expectations into more coherent and effective programs ofinstruction. They may also foster educational equity by promoting morecommon learning opportunities for students, regardless of their backgroundand the individual schools and classrooms they attend.

It is critical to remember, however, that curriculum standards arenot new to the Chicago schools. Official “scope and sequences,” not tomention thousands of textbooks outlining grade level curricula, havebeen issued for years. Indeed, the ITBS itself constitutes a long-stand-ing and powerful form of curricular guidance. Our research makes clearthat external guidelines and mandates do not, by themselves, preventtroubling differences in teaching and learning from occurring. As nec-essary and challenging as it is to develop high quality learning stan-dards, standards documents must be understood as only the first stepin a series of changes that actually influence how schools organize andconduct instruction.


A Need to Strengthen Schoolsas Organizations If, in spite of the best laid plans, it is common for teachersto slow down instruction, what might keep thecurriculum moving forward? What will help teacherspush students into new learning? Our research suggeststhat individual teachers need social support to counterthe resistance put forth by students as well as the doubtsthey themselves have about what their students have beentaught and are capable of learning. This study finds thatsome of the most promising supports reside in keyorganizational characteristics and qualities of the school.

In particular, good teacher communication and col-laboration play a vital role in bringing standards frompaper to practice and in shaping how instruction de-velops and progresses across classrooms and grades.Professional interaction and reflection among teach-ers can provide perspective and clarity, whether it isthrough informal sharing of information or formalanalysis of what is being taught and produced in eachclassroom and at each grade level. These are the op-erational dynamics of a school-based professional com-munity at work.

We also find that closer physical and social proximityamong members of a school community can offer pow-erful assistance. Small schools with fewer students in eachgrade facilitate social familiarity and shared knowledgethat reduce some of the doubts and ambiguities that canslow instructional programs. To be sure, small schoolsize does not guarantee more positive relations and in-structional coordination. Such processes, however, aremuch easier for committed adults to sustain in such anenvironment.

Perhaps most troublesome are the negative relation-ships we found between high levels of student mobilityand instructional pacing and coherence. Currently, whennew students enter a school, little information about theirpast instruction accompanies them, even if they are trans-ferring from another CPS school. Typically, they are sim-ply assigned to the classroom at the age-appropriate gradelevel with the most vacant seats. Rather than undertak-ing a careful diagnosis of new students’ knowledge andskills in order to make the best instructional placement

possible, these new students are routinely distributedaround so that “all teachers get their fair share.” Thus,we neither assure instructional continuity throughsystemwide curriculum pacing and coordination northrough individualized assessment and placement. As aresult, many of these new students experience less school

success than their more stable peers. Just as troubling,teachers in high mobility schools resort to even slowerpacing to accommodate the ever changing range of stu-dent knowledge and experience in their classroom—anunderstandable response, but one that unfortunately af-fects all of the students in the classroom.17 To be clear,this problem is not an issue that individual classroomteachers can solve alone; rather, it demands coordinatedschool- and system-level action.

Fortunately, the school characteristics that we haveidentified as significantly related to grade-level pacingand instructional coherence—stronger professional com-munity that sustains rich systems of teacher sharing andreflection, smaller school size, and lower levels of stu-dent mobility—can all be influenced by policy action.Positive developments in each of these areas can advancestudents’ opportunities to learn in a more systematicmanner. Moreover, the future success for institutional-izing new standards for teaching and learning across theChicago schools may well rest on school- and system-level attention to these key organizational concerns.

31Setting the Pace

Endnotes

1See Stodolsky (1988).

2For the purposes of this research, the test form used by the CPSin 1994, identified locally as CPS94, was used. Since the ITBSis not designed to be standards-based in the way that we areusing it in this study, the link between content emphasis andscoring at national norms does vary some from form to form.Since the survey data were collected in 1994, the choice of thistest form seemed most sensible. This test form was reused bythe CPS in 1996 and 1998.

3The results presented here are based on Rasch test item analy-sis. Within this test theory, mastery level is defined as a 75 per-cent probability of getting any particular item correct. Formally,each item has a difficulty measure in logits which also repre-sents a 50 percent probability of a correct response. By adding1.1 logits to this difficulty level, we estimate the mastery level.For a further discussion of our use of these test equating tech-niques, see Bryk, Thum, Easton, and Luppescu (1998).

4We performed a separate Rating Scale Analysis (see Wright andMasters, 1982) on the teacher content reports, where we an-chored the content category difficulties based on the results fromthe ITBS Rasch Analyses. This made the content emphasis scalecomparable to the ITBS difficulty scale, allowing us to producean “initial pacing trajectory” based on the content emphasis datafor each school. Then, in order to establish a precise equatingbetween content emphasis reports and actual student assessedcompetencies, we examined the initial pacing trajectories for asample of schools with students scoring at national norms.This allowed us to introduce a final adjustment to the pacingtrajectories that permits the direct equating of content em-phasis reports and ITBS test results. For a further discussionof these analyses, see Smith and Bryk (Forthcoming).

5This line is empirically derived based on teachers’ reported con-tent emphases in schools where students score near nationalnorms.

6The survey data from each math teacher in the school werescored in terms of content emphases. A two-level HLM analysis(teachers within schools) was then undertaken. At level 1 (withinschools), teacher’s content emphasis was predicted by grade level.This produced a pacing trajectory for each school that was al-lowed to vary at level 2.

7These results are based on a two-level HLM analysis where aninstructional pacing trajectory is calculated for each school atlevel 1 and the shape of those trajectories is predicted by a vari-ety of characteristics at level 2, including prior achievement leveland percent low income enrollment.

8See, for example, Marks (1997).

9These findings are confirmed in HLM analyses like those usedfor examining percent low-income effects. The illustrative re-sults presented in Figure 7 are overall group differences, unad-justed for other factors.

10See Bryk, Easton, Kerbow, Rollow, and Sebring (1993), 19-20. Also see Sebring et al. (1995), and Sebring et al. (1996).

11See, for example, Porter (1994). Also, see Schmidt et al. (1998),and Stevenson and Stigler (1992).

12The school organizational measures considered here derivefrom the Consortium’s work on “Essential Supports for Stu-dent Learning.” They include key aspects of school leader-ship, school-community partnerships, school climate, andprofessional community and development. For more details seeSebring et al. (1995), and Sebring et al. (1996).

13See Sebring et al. (1995). We note that while it is possible thatteachers could apply their collective attentions and efforts toissues unrelated to curriculum pacing and the demands of theITBS, it would be difficult for a faculty to plan and align itscurriculum and its pacing without the supportive dynamics of aprofessional community in the school.

14To study this problem fully, we would like to have longitu-dinal student data over time that tracks the specific subjectmatter and skills to which students are exposed. Unfortunately,the current Consortium database consists only of cross-sectionalinformation on this topic. We note that it is possible for indi-vidual students to receive coherent instruction in schools char-acterized by weak grade level coherence if the school deploysability grouping or tracking. We intend to investigate thisfurther in future research.


15To compute the grade-level coherence measure, we began byestimating for each school, using a two-level HLM analysis, anempirical Bayes pacing trajectory based on teachers’ contentemphasis reports by grade (level 1) and an unconditional modelat level 2. We then computed the residual variance of teacherreports around this pacing trajectory for each school. High re-sidual variance means grade-level incoherence; low residual vari-ance means high grade-level coherence.

16A subset of 22 schools that had pacing trajectories approxi-mately equal to the systemwide average were selected. These 22schools were divided into thirds according to their coherencelevels. The figures display content emphasis reports by gradelevel in the top and bottom third of these schools, respectively.

17For further discussion of the dynamics and consequences ofschool mobility in Chicago, see Kerbow (1996).

33Setting the Pace

References

Bilcer, Diane King, Stuart Luppescu, Penny BenderSebring, and Yeow Meng Thum (1996). The Chart-ing Reform Survey Series: Public Use Data. Chicago:Consortium on Chicago School Research.

Bryk, Anthony S., John Q. Easton, David Kerbow,Sharon G. Rollow, and Penny A. Sebring (1993). AView from the Elementary Schools: The State of Reformin Chicago. Chicago: Consortium on Chicago SchoolResearch.

Bryk, Anthony S., Penny Bender Sebring, John Q.Easton, Stuart Luppescu, Yeow Meng Thum, JennyNagaoka, and Diane King Bilcer (1998). ChicagoSchool Reform: Linkages between Local Control, Edu-cational Support, and Student Achievement. Paperpresented at the annual meeting of the AmericanEducational Research Association, San Diego.

Bryk, Anthony S., Yeow Meng Thum, John Q. Easton,and Stuart Luppescu (1998). Academic Productivityof Chicago Public Elementary Schools. Chicago: Con-sortium on Chicago School Research.

Designs for Change (1998). Chicago Elementary Schoolswith a Seven-year Trend of Improved Reading Achieve-ment: What Makes These Schools Stand Out. Chicago:Author.

Kerbow, David (1996). Patterns of Urban Student Mo-bility and Local School Reform. Journal of Educationof Students Placed at Risk, 1 (2), 147-169.

Kruse, Sharon D., Karen Seashore-Louis, and AnthonyS. Bryk (1995). Professionalism and Community:What Is It and Why Is It Important in Urban Schools?An Emerging Framework for Analyzing School-basedProfessional Community. In Karen Seashore-Louisand Sharon D. Kruse, Professionalism and Commu-nity: Perspectives on Reforming Urban Schools. Thou-sand Oaks, CA: Crown.

Marks, Helen M. (1997). Student Engagement in In-structional Activity: Patterns in the Elementary,Middle, and High School Years. (Revision of StudentEngagement in the Classrooms of Restructured Schools.Madison, WI: Center on Organization and Restruc-turing of Schools, 1995.) Unpublished.

McLaughlin, Milbrey W., and Joan E. Talbert (1993).Introduction: New Visions of Teaching. In DavidCohen, Milbrey W. McLaughlin, and Joan E. Talbert,Teaching for Understanding: Challenges for Policy andPractice. San Francisco: Jossey-Bass.

Newmann, Fred M., Gudelia Lopez, and Anthony S.Bryk (1998). The Quality of Intellectual Work in Chi-cago Schools: A Baseline Report. Chicago: Consortiumon Chicago School Research.

Newmann, Fred M., and Gary G. Wehlage (1995). Suc-cessful School Restructuring: A Report to the Public andEducators. Madison, WI: Center on Organization andRestructuring of Schools.

Porter, Andrew C. (1994). The Uses and Misuses ofOpportunity-to-Learn Standards. Educational Re-searcher 24, 21-27.

Schmidt, William H., Curtis C. McKnight, Pamela M.Jakwerth, Leland S. Cogan, Senta A. Raizen, RichardT. Houang, Gilbert A. Valverde, David E. Wiley,Richard G. Wolfe, Leonard G. Bianchi, Wen-LingYang, Seung-Ho Kang, and Edward D. Britton(1998). Facing the Consequences: Using TIMSS for aCloser Look at United States Mathematics and ScienceEducation. Boston: Kluwer Academic Publishing.

Sebring, Penny Bender, Anthony S. Bryk, John Q.Easton, Stuart Luppescu, Yeow Meng Thum, WinifredA. Lopez, and BetsAnn Smith (1995). Charting Re-form: Chicago Teachers Take Stock. Chicago: Consor-tium on Chicago School Research.


Sebring, Penny Bender, Anthony S. Bryk, MelissaRoderick, Eric Camburn, Stuart Luppescu, YeowMeng Thum, BetsAnn Smith, and Joseph Kahne(1996). Charting Reform in Chicago: The StudentsSpeak. Chicago: Consortium on Chicago School Re-search.

Smith, Julia B., and Anthony S. Bryk (Forthcoming).An Organizational Analysis of Students’ Opportunity toLearn Mathematics.

Stevenson, Harold W., and James W. Stigler (1992). TheLearning Gap: Why Our Schools Are Failing and WhatWe Can Learn from Japanese and Chinese Education.New York: Simon and Schuster.

Stodolsky, Susan S. (1988). The Subject Matters: Class-room Activity in Math and Social Studies. Chicago: TheUniversity of Chicago Press.

Talbert, Joan E., and Milbrey W. McLaughlin (1994).Teacher Professionalism in Local School Contexts.American Journal of Education 102, 123-153.

Wright, Benjamin D., and Geoff N. Masters (1982).Rating Scale Analysis. Chicago: MESA Press.

35Setting the Pace


Notes

37Setting the Pace

This report reflects the interpretations of the authors. Although the Consortium's Steering Committee

provided technical advice and reviewed an earlier version of this report, no formal endorsement by these

individuals, their organizations, or the full Consortium should be assumed.


Consortium on ChicagoSchool Research

MissionThe Consortium on Chicago School Research is an independent federationof Chicago area organizations that conducts research on ways to improveChicago's public schools and assess the progress of school improvement andreform. Formed in 1990, it is a multipartisan organization that includesfaculty from area universities, leadership from the Chicago Public Schools,the Chicago Teachers Union, education advocacy groups, the Illinois StateBoard of Education, and the North Central Regional Educational Labora-tory, as well as other key civic and professional leaders. The Consortiumdoes not argue a particular policy position. Rather, it believes that goodpolicy is most likely to result from a genuine competition of ideas informedby the best evidence that can be obtained.

Melissa RoderickUniversity of Chicago

Penny Bender SebringUniversity of Chicago

Dorothy ShippsUniversity of Chicago

Mark A. SmylieUniversity of Illinoisat Chicago

Directors

Anthony S. BrykUniversity of Chicago

John Q. EastonConsortium onChicago School Research

Albert L. BennettRoosevelt University

Kay Kersch KirkpatrickConsortium onChicago School Research

Steering CommitteeRachel W. Lindsey, Co-Chair

Chicago State University

Angela Pérez Miller, Co-ChairLatino Institute

John AyersLeadership for Quality Education

Tariq ButtChicago School Reform Board of Trustees

Michael E. CarlNortheastern Illinois University

Karen G. CarlsonAcademic Accountability Council

Molly A. CarrollChicago Teachers Union

Victoria ChouUniversity of Illinois at Chicago

Joseph HahnChicago Public Schools

Anne C. HallettCross City Campaign

for Urban School Reform

G. Alfred Hess, Jr.Northwestern University

John K. HoltonHarvard School of Public Health

Richard D. LaineIllinois State Board of Education

James H. LewisChicago Urban League

Donald R. MooreDesigns for Change

Jeri NowakowskiNorth Central RegionalEducational Laboratory

Charles M. PayneNorthwestern University

Barbara A. SizemoreDePaul University

Linda S. TafelNational-Louis University

Beverly TunneyChicago Principals and

Administrators Association

Paul G. VallasChicago Public Schools

Consortium on Chicago School Research

1313 East 60th Street, Chicago, IL 60637

773-702-3364 773-702-2010 - fax

http://www.consortium-chicago.org