Y6 MMP Evaluation SNA Report

8/8/2019 Y6 MMP Evaluation SNA Report

1/32

Year 6 Evaluation Report:

Report on School-Based Learning Communities

Prepared by

Carl E. Hanssen, Ph.D.Hanssen Consulting, LLC

September 2009

________________________This material is based upon work supported by the National Science Foundation under Grant No. 0314898. Any

opinions, findings and conclusions or recommendations expressed in this material are those of the author and do

not necessarily reflect the views of the National Science Foundation (NSF).


2/32

Milwaukee Mathematics Partnership School-Based Learning Communities

i

Table of Contents

Table of Contents....................................................................................................................... i

List of Tables ............................................................................................................................ ii

List of Figures........................................................................................................................... iiIntroduction............................................................................................................................... 1

Overview of Evaluation Activities........................................................................................ 1Social Network Analysis........................................................................................................... 2

Statistical Results .................................................................................................................. 4

Sociograms............................................................................................................................ 6Learning Team Observations.................................................................................................. 18

Overall Team Functioning .................................................................................................. 19

MMP Impact ....................................................................................................................... 20Conclusions............................................................................................................................. 22

Appendices.............................................................................................................................. 24


3/32


ii

List of Tables

Table 1: Social Network Analysis Statistics ..........................................................................4

List of Figures

Figure 1. Distributed Leadership Continuum..........................................................................6School A ..................................................................................................................................7

School B ..................................................................................................................................8

School C ..................................................................................................................................9School D ................................................................................................................................10

School E ................................................................................................................................11

School F ................................................................................................................................12

School G ................................................................................................................................13School H ................................................................................................................................14

School I ................................................................................................................................15School J ................................................................................................................................16School K ................................................................................................................................17

Figure 2. Learning team models............................................................................................18


4/32


1

Introduction

Hanssen Consulting, LLC is pleased to provide this addendum to the Year 6 Evaluation

Report for the Milwaukee Mathematics Partnership. This report provides results and analysis

from data collected in selected MPS schools during the 2008-09 school year. Specifically,

these data address the evolution of school-based learning communities for mathematics.Combined with our Year 6 report dated August 2009, this evaluation is intended to satisfy

National Science Foundation requirements that all funded Math Science Partnership projectsconduct a rigorous evaluation of their activities.

The Year 6 evaluation has been guided by the Evaluation Plan submitted to NSF and datedDecember 23, 2005 and builds on the activities and results from Years 3 through 5. The

evaluation plan outlines two broad objectives for the MMP evaluation. They are

1. to help the MMP better serve its constituents and improve its effectivenesstoward reaching stated goals and objectives, and

2. to serve the broader mathematics education community by documentingaccomplishments and disseminating critical findings based on project activities.

The broad objectives may be termed as formative and summative, respectively.

Overview of Evaluation Activities

During Year 6, the evaluation focused on revisiting Social Network Analysis that was

conducted in schools in past years. We were able to successfully complete the analysis in all

eleven schools that were subjects of the evaluation case studies in 2006 (Year 3). The results

reported here focus on changes that were observed in the 11 case study schools we collecteddata from in 2006.

We also conducted learning team observations in approximately 25 schools across thedistrict. These observations were not intended to support the development of case studies,

but rather were intended to (a) verify or suggest alternative learning team models to those

observed in the past and (b) offer additional explanatory information to support the socialnetwork analysis efforts.

Unlike in past years, we did not conduct classroom observations or administer theMathematical Knowledge for Teaching (MKT) assessment to teachers in targeted schools.

We have discontinued this activity because we felt we were not learning anything new.

Finally, it is important to understand that this work was set against the context of a majorprogram shift. The 2008-09 school year was the first full year where over 100 math teacher

leaders were released from the classroom. Of our sample of 11 schools, 10 had a releasedMTL in 2008-09. The data presented below, we believe, were greatly influenced by this

program change and suggest some fundamental changes in school-based learning

communities.


5/32

Milwaukee Mathematics Partnership School Learning Communities

2

Social Network Analysis

Social Network Analysis (SNA) is used to describe school-based networks. The purpose of

this analysis is to gain an understanding of the nature of collaborative relationships for

mathematics education in a given school. For the purpose of this analysis, collaboration wasdefined as communication about mathematics on the assumption that communication is a key

element of collaboration and often the first step leading to more meaningful collaborative

efforts (e.g., working together on a project, planning mathematics lessons, or developing theschool-wide learning plan for mathematics). Over the past four years of the MMP, thisanalysis has served as a starting point for (a) understanding the nature of school-based

collaborative networks and (b) measuring change in collaboration about mathematics over

time.

The analysis presented in this report takes a second look at the 11 schools that were subjects

of the analysis in 2006 (Year 3 of the evaluation). Our primary purpose was to identify

changes that might have occurred over time. For these 11 schools, we present descriptivestatistics and sociograms (i.e., SNA maps) below.

Methods

SNA is conducted using a pencil-paper survey (see Appendix A) that is administered to all

mathematics teachers and other mathematics education stakeholders (e.g., Learning Teammembers) in a school. In 2009, a self-administration packet was used to facilitate

administration of the SNA survey. Survey respondents were asked to indicate who they

communicated with about mathematics, the frequency of that communication, and the extent

to which they collaborated with each individual named. Based on past work, respondentswere asked to limit their list of individuals to 15 people; the vast majority of respondents

indicated fewer than 10 individuals on their surveys. Though some respondents named morethan 15 people, we found that most limited their lists to well below the 15-person limit.

Data for each school were compiled in two spreadsheetsthe first contained matrix data

which indicated who chose who. The second contained demographic data for each uniqueindividual named across all of the surveys submitted by a school. A typical school data set

indicated a ratio of approximately 1:2-3 of survey respondents to unique individuals named.

This open SNA approach has the added benefit of allowing us to assess the extent to which aschools mathematics education network extends beyond the school.

Data were analyzed using UCINET software.1

Four descriptive statistics were calculatedtwo measures of network density, a measure of the perceived importance of the MTL, and a

measure of the perceived importance of the MTS assigned to that school.

1 Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). UCINET for Windows: Software for Social Network

Analysis. Harvard, MA: Analytic Technologies.


6/32


3

Density is defined as the total number of ties divided by the total number of possible ties in a

network and is represented as a percentage. For example, if a network contains threeindividuals, the total number of possible ties is six, which would occur if everyone in the

network named all of the others in the network. If only five ties were articulated by the

survey respondents, then the network density would be .83 or 83%.

Density was calculated twicefirst using just the individuals who work in the subject school

and secondly using all the individuals named, which included individuals outside of theschool. In interpreting these statistics, the difference between network-wide density and the

density within school yields important information about, for example, the degree to which a

school extends beyond its walls to improve mathematics education or whether a school takes

a more insular approach.

MTL Importance and MTS Importance are evaluated using the Freeman Degree Centrality

measure2. This metric is an indicator of how important an individual is to the networka

higher number indicates that other individuals named that individual frequently and thus they

are perceived as important to the network. Normalized scores are reported so that valuesacross different networks can be compared. Essentially, however, higher numbers indicategreater importance in the network. This measure is commonly referred to as the In Degree.

Sociograms were also prepared for each school. A sociogram provides a graphical depiction

of the social network. The key features are nodes (i.e., individuals), and ties (i.e., linksbetween individuals depicted by lines). The key for interpreting these sociograms is as

follows:

1. Distance is important. Closer nodes are more tightly connected than nodes that arefurther apart, even though those nodes may not be directly linked. Distance should alsobe interpreted in light of the total number of people in the networknetworks with

fewer people may appear less connected but may in fact be statistically denser.

2. Ties are depicted by lines with arrow heads. Ties are not necessarily reciprocal; anindividual could name someone who in turn did not name them. The arrow direction

indicates who named who.

3. Individuals from the subject school are colored red and those outside the school aregreen. Node IDs from 2006 and 2009 sociograms are not necessarily the same person.

4. Roles are depicted as follows:MTL=Diamond; yellow represents MTL at the schoolMTS=Down Triangle; blue represents the MTS assigned to the school

Principal=Overlapping Triangles

Literacy Coach=Up TriangleTeachers=Squares

Other Individuals=Circles

2 Freeman, L. C. (1979). Centrality in Social Networks: Conceptual Clarification. Social Networks 1, 215-239.


7/32


4

Statistical Results

Table 1 indicates the descriptive statistics for the 11 schools that are the subjects of this

study. Note that School B was the one school in our sample that did not have a released

MTL in 2008-09. The results for this one school, though, do not appear different than the

other schools. The subsequent pages depict the 2006 and 2009 sociograms for each school.

Table 1: Social Network Analysis Statistics

Grade Level School Statistic 2006 2009

K-5 A n 19 21

Total Named 45 41

Network Density 9.6% 6

Density in School 28.8% 15.9%

MTL Role In Degree 7.34 33.5

MTS Role In Degree 6.25 2.00

K-5 B* n 13 13Total Named 31 24





K-5 C n 17 18

Total Named 46 40





K-5 D n 22 32

Total Named 45 65 Network Density 5.8% 5




K-5 E n 14 13

Total Named 52 24





K-8 F n 22 27

Total Named 55 60





K-8 G n 32 33

Total Named 61 86






8/32


5

Grade Level School Statistic 2006 2009

K-8 H n 32 26

Total Named 80 50




MTS Role In Degree 6.01 5.31K-8 I n 17 17

Total Named 40 37





K-8 J n 27 30

Total Named 91 76





6-8 K n 16 18Total Named 48 70





All Schools Mean SD Mean SD

N 21 6.7 23 7.4

Total Named 54 17.6 52 20.9

Network Density 7% 3% 7% 4%

Density in School 18% 10% 15% 5%

MTL Role In Degree 13.8 7.2 30.7 7.6

MTS Role In Degree 5.3 4.9 6.0 4.8

Two important trends are evident based on these results. First, the average density of the in-

school network was lower in 2009 than in 2006, though this difference was not statistically

significant (p=.24). At the same time, the average MTL Role In Degree measure was

substantially higher in 2009 than in 2006. This increase was statistically significant (t=6.81,p=.00). Negligible differences were seen in this small sample of schools on the other

measures.

Combined, these results suggest the possibility of a fundamental shift in how school-based

networks function; further we believe that this shift may be due to the emergence of the

released MTL-model as the dominant approach to math teacher leadership across MPS; eventhe one school that did not have a released MTL exhibited similar changes from 2006 to

2009. This shift can best be described in the context of the distributed leadership continuumthat was described in 2008 (see Figure 1 below).


9/32


6

Figure 1. Distributed Leadership Continuum

This continuum suggests an evolution of distributed leadership within a school that leads to

shared leadership amongst many teachers and administrators. This shared leadership may be

indicated by a statistically dense network that has many individuals communicating withmany other individuals. On a sociogram, this type of network is tightly webbed with many

ties between many different individuals.

What the change in the statistical pattern from 2006 to 2009 shows, however, is that while

school networks are becoming slightly less dense, the importance of the MTL has

dramatically increased within many schools. This is shown by most teachers indicating that

they communicate with the MTL in that school. For the released MTL, opportunities tointeract with individual teachers to help improve their performance have likely increased. At

the same time, this one-on-one interaction may be reducing the need for larger groups of

teachers to communicate and collaborate with one another on a regular basis. The long-termimpact of this may be that teachers abdicate responsibility for mathematics leadership to the

MTL thus stalling the progression of distributed leadership below position 5 on thecontinuum.

The positive aspect of these observations is that in most schools, the MTL is viewed as themost important individual in terms of mathematics communication and collaboration. The

impact of the MMP in terms of positioning the MTL as a school-based leader is clearly

demonstrated through this analysis. The sociograms depicted below also clearly show thecentral position of the MTL within most school-based networks.

Even though we had one school in our sample that had a non-released MTL, and this school

demonstrated similar characteristics to the schools with released MTLs, we do not believethis invalidates our conjecture about the impact of the released MTL on school learning

communities. Similarly, the impact of these trends on mathematics student achievement and

overall school-improvement efforts is not known at this time. We anticipate gaining agreater understanding of the impact of the MTL release model through our Phase II research.

Sociograms

The following pages depict the 2006 and 2009 sociograms for the 11 schools described in

this study.


10/32


7

School A

2006 2009

The critical shift in this network is the dramatic decrease in in-school density from 28 to 15 percent. As a result, the 2006 networkappears much more webbed compared with the 2009 network. At the same time, the importance of the MTL to the 2009 network ismuch greater than in 2006, increasing from 7.3 to 33.5. This is an indication that the MTL was one of the few people named by most

respondents in 2009 as opposed to being one of many people named by respondents in 2006.

The other key change in these networks is the movement of the MTS (#29) from a relatively central position in 2006 to a peripheral

(#25) role in 2009. In 2009, the MTS was named only by the MTL and the principal as someone with whom they communicated.


11/32


8

School B

2006 2009

In this school, the MTL was not released in 2008-09. In 2006, the network was dominated by the isolate group at the right of thesociogram. In 2009, the isolate group is gone and the school exhibits other changes, most notably the increased centrality of the MTL

in the network in relation to the Literacy Coach (Red Up Triangles in both sociograms). The density of the in school network has also

improved from 2006 to 2009 11.7 to 17.3 percent but this is likely a statistical function of the disappearance of the isolate group.


12/32


9

School C

2006 2009

The MTL and principal (Red Overlapping Triangles) positions have essentially shifted in this school from 2006 to 2009. This

suggests that in 2006, the principal was seen as the primary mathematics instructional leader while the MTL may have been stillstruggling to gain credibility. The MTL role In Degree has increased from 22 in 2006 to 33 in 2009. At the same time, in-school

network density has decreased from 28 to 23 percent though it is still quite high. This suggests overall stability within the school

though leadership responsibility for mathematics has clearly shifted to the MTL.


13/32


10

School D

2006 2009

This school has exhibited some important changes between 2006 and 2009. First, both the number of survey respondents and numberof people named has increased for this school. The statistical impact of this is similar overall network density (5.8% in 2006 versus

5.5% in 2009) yet a sharp decrease in in-school network density (28% versus 12%). At the same time, the MTL role In Degree hasincreased while the MTL remains relatively central to the network. Additionally, the 2009 network exhibits several gatekeepers who

control communication to the MTL from an isolated group (at the left of the 2009 sociogram).


14/32


11

School E

2006 2009

The dramatic change in this network is the sharp reduction in the number of different people named by the survey respondents. In

2006, 14 survey respondents named 52 different people, or a ratio of 1:3.7. In 2009, 13 survey respondents named only 24 people, or

a ratio of 1:1.8. As a result, the network appears less dense even though statistically, it is slightly denser with overall network densityincreasing from 4% to 13% and in-school density increasing from 15% to 19%. At the same time, MTL In Degree has increased

dramatically from 17 to 46 during this period where the school appears to have become more insular.


15/32


12

School F

2006 2009

This network maintained some of its core features from 2006 to 2009. The MTL, MTS, Principal, and Literacy Coach are all central

figures in both networks. The in-school network is tightly webbed, though less dense overall. Another important feature of the 2009

network is the presence of two MTSs at the center of the network, indicating that many individuals within the school reportcommunicating with both MTSs directly. Among all schools in our sample, this one appears to take best advantage of the MTS

resource.


16/32


13

School G

2006 2009

The key difference between 2006 and 2009 for this school is the slightly less dense network which is likely a function of the

significantly higher number of different people named by the survey respondents86 in 2009 versus 61 in 2006. This suggests that

the school is extending its network outward by, perhaps, taking advantage of well-established MMP and district resources. Incontrast, the MTS role In Degree declined slightly from 3.74 to 1.41. As with other schools, however, the MTL role In Degree

increased substantially from 6.67 to 19.29.


17/32


14

School H

2006 2009

This network is one of only a few where in-school density and MTL role In Degree, has increased from 2006 to 2009. The importantchange, however, is that substantially fewer people were named on the surveys (80 in 2006 versus 50 in 2009). A second important

change was the school went from having two MTLs in 2006 to one in 2009. This is a likely explanation for the dramatic increase in

the MTL role In Degree from 11.7 in 2006 to 32.2 in 2009 and illustrates the impact of focusing attention on one versus two differentindividuals.


18/32


15

School I

2006 2009

This network has been strengthened in multiple ways between 2006 and 2009. First, the in-school network density has increased from9% to 12%, suggesting increased communication amongst school personnel. Second, the MTL position has shifted to the center as

well as exhibiting a statistical improvement from 17.3 in 2006 to 37.2 in 2009. Lastly, the MTS role In Degree has increased in this

network from 3.9 to 11.6, indicating greater utilization of this district resource.


19/32


20/32


17

School K

2006 2009

This school has demonstrated improvements in key areas. The overall density of this network has increased due to the elimination of

the isolate group observed in 2006. At the same time, the MTL role In Degree has increased from 12 to 20, which reflects the MTLsmovement to the center of the sociogram. Consistent with 2006, however, is the lack of importance of the MTSin both years, only

one person, the MTL, reported communicating with the MTS (blue triangle) which reflects this individuals lack of visibility at theschool.


21/32


18

Learning Team Observations

SNA of selected schools provides a quantitative means of examining changes in school-based

networks over time. Additional insights can be gained from looking at data collected by

observing learning teams. As in past years, the evaluation team conducted observations oflearning teams in selected schools. These observations adhered to a pre-defined protocol (see

Appendix B) that addressed (a) overall team functioning and (b) evidence that a school has

adopted MMP principles and ideas.

We conducted observations in 25 different schools. Some of these schools were the same

ones in which we conducted social network analysis but many schools were different.

Unlike in past years, our goal this year was not to create pseudo-case studies of a small

number of schools, but rather to determine if the primary trends that were evident in our pastlearning team observations were continuing, had stalled, or were changing.

As reported in Years 3-5, the primary learning team trends that we observed can be depictedby the following learning team models.

Figure 2. Learning team models

Previous results had shown that schools assumed one of the two general model types

depicted above. Meetings tended to be either authoritarian or participatory. At the same

time, our general conclusion was that high-achieving schools tended to adopt a participatorymeeting approach. A high level of participation was often a manifestation of shared (i.e.,

distributed) leadership for mathematics (and other) school improvement efforts. Learningteam meetings in these schools tended to focus on discussion, problem solving, and solutiondevelopment. Authoritarian meetings, in contrast, tended to reflect a school learning

community where individuals or small teams worked in relative isolation and then reported

the results of this work to the larger learning team. In these schools, adoption of proposedpractices was often difficult because the proposed change rarely had buy-in from the larger

group before it was attempted.


22/32


19

In addition to the learning team models discussed above, we looked for evidence that MMP

ideas and principles were being adopted within a school, irrespective of the model exhibitedwithin a school. In the past, MMP ideas have been manifested through learning teams

discussing CABS, examining student data, and reporting back information and outcomes

from district-wide MTL meetings. We were interested in determining if these same trends

were continuing or if the MMP was manifesting in different ways across the district.

The following are results from these observations and a discussion of the key themes thatwere observed.

Overall Team Functioning

Based on our observations of learning team meetings during the 2008-09 school year, the

following dominant themes emerged.

Schools have refined the mechanics of learning team meetings. Most observedmeetings were guided by an agenda, had a clear meeting leader, and finished in theallotted time. We observed that the most effective meetings had relatively shortagendas (3-5 items). Though few schools take this approach, many of the best

meetings we observed were held during the school day which placed a premium on

completing the agenda and reduced wasted time.

Fewer and fewer meetings are being led by the Principal. We observed that manymeetings are led by the Literacy Coach, the MTL, an Implementer, or a District

Curriculum Generalist. This year, we especially observed that more meetings were ledby the MTL. This may be a by-product of the MTL release model and an indication

that principals are taking advantage of having another resource that is not committed tothe classroom 100% of the time.

Learning teams tend to be representative of the school overall and participation atmeetings is widespread. Most teams have a mix of grade level and functional

representation (e.g., math, reading, special education, art). Few meetings are

dominated by only a few individuals as most learning team members appear to beknowledgeable and committed. Our observations conveyed a sense, however, that the

learning team viewed itself and was, in fact, often comprised of the top performing

teachers in a school. There were often references to bringing the others along, or

convincing the others to do. during meetings. This suggests that the learning teamis typically comprised of school leaders but that not all school staff share the teams

enthusiasm for school improvement efforts.

Fewer meetings would be termed working sessions; the most common meeting formatwas what could be called report and discuss. This format demanded that one

individual provide a report on another meeting, project, event, etc. The team wouldthen ask questions or discuss the implications of the report. These meetings often took

on an administrative tone and appeared less focused on student learning. Decisions,

where they occurred, were often implied or vague to the point that it was difficult to


23/32


20

detect when actual decision-making took place. Rarely were key decisions or to dos

made explicit at the end of a meeting.

Fewer and fewer meetings consisted of either working sessions or professionaldevelopment. Where working sessions occurred, the focus was usually student data

and the goal of the session was to develop ideas for the School Education Plan orSchool Improvement Plan. Professional development, where it occurred, usually

consisted of a team member bringing back information from a conference or otherdistrict-wide meeting.

Overall, based on our observations it appears that the dominant trend in learning team

meetings is that few meetings actually focus on student learning as opposed to administrativeissues. The participatory-authoritarian distinction appears less valuable as a way of

classifying learning team meetings as most meetings are participatory in nature and fewer

and fewer meeting leaders assume an authoritative stance. This latter observation may bedue largely to a reduction in principal-led meetings versus meetings led by other staff

members, such as the Literacy Coach or the MTL.

MMP Impact

Most observed learning team meetings reserved a portion of the meeting for the math

report. Along with reading, this was the dominant curricular topic at most schools. Overall,this was not unexpected for two reasons. First, because the school knew the MMP evaluator

would be in attendance, it was likely that a purposeful effort was made to provide evidence

of MMP activities in the school. Secondly, reading and math dominated the discussion atlearning team meetings because these are the areas of greatest emphasis within many schools.

Still, there was ample evidence that the MMP is having an impact in many MPS schools.

Based on our observations, the following evidence of MMP impact emerged as most

prevalent.

Many schools use learning team meetings to scan or provide an overview of student testresult data. Most often, these data are then sent back to grade level groups for detailedanalysis that would lead to differentiated instruction or tactics for improving

performance of specific students. The MMP has continually encouraged schools to

examine student data and it is clear that most are now doing so. This impact, though, is

not limited to mathematics but to other curriculum areas as well.

Schools are developing CABS, using CABS, and are focused on adopting commonCABS within grade levels. While there is evidence of CABS-focused discussions, theclearest evidence of MMP impact is the process that learning teams and other teachers

are using to score CABS and to summarize descriptive feedback for the classroom.

Most often this detailed work is not done in learning team meetings but the process forconducting the work and reporting back the results of these efforts is a frequent topic of

the MTLs math report. Given the heavy emphasis on these activities at MTL meetings

during the 2008-09 school year, it is promising that these efforts are being directly


24/32


21

translated to the school-based community.

Several teams discussed math portfolios as an additional form of assessment. Whilethis is a relatively new initiative, the use of portfolios has been promoted by the MMP.

MTLs have been oriented to different types of student portfolios in advance of a

district-level expectation that schools begin implementing the use of portfolios todocument student achievement.

Schools are aware of The Learning Team Continuum and some have used this as atool to self-evaluate their performance. The Continuum is a tool developed through

MMP efforts and used by the MPS Curriculum Specialist to help a school evaluate its

progress against key developmental stages (e.g., what proportion of a schools staffunderstands the importance of learning targets, or what proportion regularly examines

student work to monitor achievement and progress). There is evidence that learning

teams have discussed this tool and are using it to stay focused on their improvementefforts.

Learning teams reference district Math Teaching Specialists (MTS) in multiple waysmany school staff members have personal relationships with MTSs. Other times, there

are references to bringing in a particular MTS to conduct a professional development

session at a school. Awareness and utilization of these district resources that were

established by the MMP demonstrates positive impact.

Despite these positive impacts, it is also clear that little real school improvement work is

occurring at learning team meetings. This work is most often occurring in grade levelmeetings, during staff meetings, at Banking Day, or in separate subject-focused meetings

held outside the context of the learning team. What is evident at the learning team level isthe reporting back about the results of these other meetings. As a result, a school that

references MMP ideas and principles in the learning team setting is likely applying those

ideas in various other settings. Future work will examine how those ideas and principles arebeing applied in practice.


25/32


22

Conclusions

The MMP continues to have a strong impact on school-based learning communities across

the Milwaukee Public Schools and there is ample evidence from both the social networkanalysis and learning team observation to buttress this claim. The 2008-09 school year,

however, was an important year for the MMP because it brought a significant programchange to MPSthe adoption of the release MTL model in over 100 MPS schools. This

significant change has impacted efforts to develop distributed leadership in MPS schools.

For example

1. Social network analysis reveals that school networks are less densethat is participantsdo not report widespread communication with their peers at the same level as they didin 2006. At the same time, the relative importance of the MTL to every school-based

network has increased dramatically. One hypothesis for this shift is that having a

released MTL enables more one-on-one and small group work between individualteachers and the MTL, rather than requiring all teachers to be engaged in collective andcollaborative process improvement efforts. Thus, teachers may feel as if they can shift

leadership responsibility primarily to the MTL rather than assuming a leadership role

themselves.

2. Evidence from learning team meetings further suggests that the real work of improvingmath education is occurring in ancillary meetings such as grade level meetings orspecial math-focus meetings. While this is overall positive, one unintended impact may

be the isolation of small teams of teachers. Thus, grade-level teams may be very strong

and communicate extensively, but this communication may not extend beyond ones

grade-level colleagues. Thus, cross-grade or school-wide communication may be lessimportant in a school where an MTL is released.

3. Learning team meeting observations provide strong evidence that the MMPprofessional development modelwhere the MTL receives development and then

brings that new knowledge back to the schoolis occurring. Whether this

development occurs in a learning team meeting (rarely), in an ancillary meeting (morefrequently), or individually (commonly) is not really a topic of concern. And, now that

many MTLs are released, this process can be allowed to flourish. At the same time,

this may provide a dis-incentive for teachers to assume responsibility for theirdevelopment, one manifestation of which might be the desire to attend fewer outside

courses or development opportunities. From the perspective of social networks, oneindication of this phenomenon is survey respondents indicating that they communicate

with fewer people. Across the 11 schools for which social network data were reportedabove, only four reported an increase in the total number of individuals named as part

of the networkboth inside and outside the school.

4. While there is strong evidence that the MTL is clearly the math leader in manybuildings, there is also evidence that MTLs are being asked to lead learning team


26/32


23

meetings that have a broader focus than just mathematics. Care should be taken to

ensure that MTLs can retain their focus on mathematics and not get swept into theeveryday administrative milieu of their schools.

Overall, many of the schools, where we conducted social network analysis and observed

learning teams, exhibited changes between 2006 and 2009. Our analysis suggests that muchof this change was driven by the emergence of the released MTL model as the dominant

approach for leading school-based math improvement efforts. It is too early to say whetherthese changes will have positive or adverse effects. Future work will continue to monitor

progress and trends that can help answer this key question.


27/32


24

Appendices


28/32

Hanssen Consulting, LLC 1324 Lake Drive SE Suite 6 Grand Rapids, MI 49506 616-808-2867 Fax 616-808-2866 http://www

APPENDIX A: MILWAUKEE MATHEMATICS PARTNERSHIP (MMP) SOCIAL NETWORK

Your Name: Your School:

1. List up to 15 individuals you have communicated with about mathematics education in the past several months. Pleaseindividual. For this survey, communicated with means any form of communication such as face to face, notes, email, o

anything having to do with math curriculum, assessment, classroom teaching, or related topics.

2. Indicate whether this person works at your school.3. Indicate the role of each individual using the numeric codes provided. Please choose from among the list of roles provid4. For each individual you list, indicate the frequency of communication about math by placing an X in the appropriate c5. For each individual you list, indicate the extent of collaboration about math education by placing an X in the approp

collaboration is defined as actively working together to solve a problem, complete a task, or engaging in some type of

4. Frequency of com1. First and Last Name of individuals

you have communicated with about

mathematics education in the past several

months.

2. Does this

person work

at your

school?

(circle one)

3. Role of this individual:Please choose from among this list:1MTL 2Principal

3Literacy Coach 4Teacher

5Other Admin 6MTS

7Teacher in Residence 8MPS Admin/Staff

9UWM Staff 10Other11Math Dept Chair 12Math TeacherOnceperyear

Afewtimesperyear

O

ncepermonth

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

25


29/32

Hanssen Consulting, LLC 1324 Lake Drive SE Suite 6 Grand Rapids, MI 49506 616-808-2867 Fax 616-808-2866 http://www.hanssenconsulting.com

Demographic Questions

What is your role? (check all that apply)

Indicate the grades you teach (check all that apply)

How many years have you been teaching at MPS?

How many years have you been teaching at your school?

How many years have you been a certified teacher?

What is your gender?

What is the highest degree you currently hold?

MTL Literacy Coach AP Math Teacher Implementer Curriculum Generalist Learning Team Member Principal Other

K3, K4, or K5 5th

grade 10th

grade 1

stgrade 6

thgrade 11

thgrade

2nd

grade 7th

grade 12th

grade 3

rdgrade 8

thgrade

4th

grade 9th

grade

Male Female

Bachelors

Masters Masters plus Ed.D. Ph.D.

26


30/32

APPENDIX B

Milwaukee Mathematics Partnership Learning Team Observation Protocol

External Evaluation

Introduction The purpose of this protocol is to facilitate observations of MPS school-based learning

teams. These observations are one part of the MMP Evaluation. This work continues

and builds on work conducted in Years 3-5 of the first MMP grant.

The premise for learning team observations is that learning team functioning, in general,

and team attention to mathematics, specifically, is related to teacher behavior, which in

turn influences student achievement. Thus, learning teams are one link in the logic chainwhich presumes that the work of the MMP influences the math teacher leader, which

influences the learning team, which influences school attention toward mathematics.

Instructions Two domains are addressed in this protocol(1) overall team functioning and (2)

mathematics-related issues. For each domain, there are several underlying concepts that

represent that domain. For each concept, 3-5 indicators are listed that are representative

of each concept. For each concept area, provide a rating and comments to support yourrating. In addition, a summary rating for each domain is requested.

Please also complete a brief summary of what occurred during the meeting in the spaceprovided below.

School: Date: Time:Number present:

Meeting leader:Primary MTL: Released (y/n) Second MTL: Released (y/n)

Observer:

Meeting Summary (length of meeting, topics discussed, key follow-up steps, etc.)

27


31/32

I. Team Functioning

A. Leadership

-There is a clear team leader

-The leaders authority is apparent

-Leader has team members respect

5 There is strong team leadership

4

3

2

1 Team leadership is lacking

Leadership comments:

B. Participation & Representation

-team members actively participate

-multiple viewpoints are represented

-conflict is not counterproductive

5 Broad participation & representation

4

3

2

1 Limited participation & representation

Participation & Representation comments:

C. Organization & Structure

-there is a clear agenda-the meeting is organized and well-planned

-objectives are appropriate for time allowed

5 Clear organization & structure

43

2

1 No clear organization or structure

Organization & Structure comments:

D. Meeting Focus

-the meeting focused on teaching and learning (e.g.,

student data, curricula, instructional issues)

-the meeting focused on administrative issues

(e.g., discipline issues, logistics,

5 The meeting focused on teaching and learning

4

3

2

1 The meeting focused on administrative issues

Meeting Focus comments:

E. Results & Actions

-the meeting has clear outcomes

-results are relevant

-next steps and action items are clear

5 Obvious results & clear actions

4

3

2

1 No clear results or next steps

Results & Actions comments:

F. Overall Functioning 5 High functioning

43

2

1 Low functioning

Comments:

28


32/32

II. MMP-Specific Issues

A. Coherent vision for mathematics

-one curriculum

-no individual teacher entrepreneurs

-curriculum consistent within and across grades

5 There is a clear coherent vision for math

4

3

2

1 There is no vision for math

Coherent vision comments:

B. Integration

-math is discussed at the meeting

-math is discussed in context with other issues

-math is perceived as equally important to other

subjects

5 Math is integral to the school

4

3

2

1 Math is not integral to the school

Integration comments:

C. Math Leadership

-The MTL(s) leads discussions on math

-The MTL(s) is respected-If not the MTL, then someone else is the clear

leader for math

5 The MTL is clearly the math leader

4

32

1 There is no apparent leader for math

Math Leadership comments:

D. Overall MMP Issues 5 Clear adoption of MMP perspectives

4

32

1 Has not adopted MMP perspective

Comments:

Documents

Y6 MMP Evaluation SNA Report