Download pdf - Community School Evaluations

A RESOURCE GUIDE FOR

UNDERSTANDING COMMUNITY SCHOOLS Community School Evaluations

October 2012

Prepared by:

Iris Hemmerich

Urban Strategies Council

1

©Urban Strategies Council, October 2012

Community School Evaluations

Table of Contents

A Resource Guide for Understanding Community Schools .......................................................................... 2

Updating the Resource Guide ................................................................................................................... 4

Additional Community School Resources ................................................................................................. 4

Our Community School work with Oakland Unified School District ............................................................. 5

Community School Evaluations: Literature Review ...................................................................................... 6

Introduction .............................................................................................................................................. 6

Review ....................................................................................................................................................... 6

1. Methodology .................................................................................................................................. 6

2. Indicators and Measures of Success ............................................................................................... 7

3. Outcomes........................................................................................................................................ 7

Conclusion ................................................................................................................................................. 8

1. Methodological Flaws & Challenges .............................................................................................. 8

2. Promising Practices ........................................................................................................................ 8

3. Concluding Remarks: Being Intentional about Equity .................................................................... 9

Community School Evaluations: Annotated Bibliography .......................................................................... 10

2


A Resource Guide for Understanding Community Schools

INTRODUCTION

Urban Strategies Council has collected and reviewed more than 175 evaluations, case studies,

briefs and reports for use by those considering or planning a community school or community

school district. Our intention is to provide interested individuals and stakeholders the

resources they need to better understand the unique structure and core components of

community schools. The promising practices, recommendations, tools and information shared

in this document have been culled from documents representing the last 20 years of research

and documentation of community schools across the United States.

We highlighted 11 content areas that we believe to be the most foundational for understanding

community schools. Within each of the content areas, you will find:

1. A literature review: The literature reviews for each content area are organized

around core questions and provide a synthesis of the most commonly identified

solutions and responses to each question, as well as highlights, promising practices,

challenges and recommendations.

2. An annotated bibliography: We gathered and annotated literature in each of the

content areas to underscore key themes, some of which include: best practices,

exemplary sites, models and tools. The annotations vary by content area in order to

draw attention to the most pertinent information. For example, the Evaluations content

area includes annotations of the evaluation methodology and indicators of success.

The 11 content areas include the following:

1. Community School Characteristics

Provides a general overview of the structure, function, core elements, programs and

services of a community school.

2. Planning and Design

Explores the general planning and design structures for community schools, and

discusses the initial steps and central components of the planning and design process, as

well as strategies for scaling up community schools.

3. Equity Frameworks and Tools

Examines literature and tools that can be adapted to an equity framework for

community schools. We included equity frameworks and tools that explore

disproportionality and the monitoring of disparities and demographic shifts.

3


4. Collaborative Leadership

Addresses how to build, strengthen and expand the collaborative leadership structure at

community schools. Collaborative leadership is a unique governance structure that

brings together community partners and stakeholders to coordinate a range of services

and opportunities for youth, families and the community.

5. Family and Community Engagement

Explores how community and family engagement operates as well as the challenges for

actualizing it at the site level. Family and community engagement is a unique

component of community schools in which the school, families, and community actively

work together to create networks of shared responsibility for student success.

6. Data Collection and Analysis

Addresses the outcomes measured at community schools, methods for collecting data

at community schools, and short and long term indicators.

7. Assessment Tools

Includes tools used to measure outcomes at community schools.

8. Community School Evaluations

Provides evaluations of community school initiatives with special attention paid to

methodology, indicators of success, findings and challenges.

9. Community School Funding

Explores how to leverage revenue streams and allocate resources at community schools.

10. Budget Tools

Includes tools that support the process of budgeting and fiscal mapping.

11. Community School Sustainability

Explores promising practices for creating sustainability plans, partnership development

and leveraging resources for the future.

4


UPDATING THE RESOURCE GUIDE

Urban Strategies Council will continue its efforts to update the Resource Guide with the most

current information as it becomes available. If you know of topics or resources that are not

currently included in this guide, please contact Alison Feldman, Education Excellence Program,

at [email protected]. We welcome your ideas and feedback for A Resource Guide for

Understanding Community Schools.

ADDITIONAL COMMUNITY SCHOOL RESOURCES

National:

The Coalition for Community Schools

http://www.communityschools.org/

The National Center for Community Schools (Children’s Aid Society)

http://nationalcenterforcommunityschools.childrensaidsociety.org/

Yale University Center in Child Development and Social Policy http://www.yale.edu/21c/training.html

Regional:

The Center for Community School Partnerships, UC Davis

http://education.ucdavis.edu/community-school-partnerships

Center for Strategic Community Innovation http://cscinnovation.org/community-schools-project/about-cscis-community-schools-

project/community-school-initiative-services-coaching-and-ta/’

5


Our Community School work with

Oakland Unified School District

Urban Strategies Council has a long history of working with the Oakland Unified School District

(OUSD) to support planning for improved academic achievement. Most recently, we helped

develop and support the implementation of OUSD’s five-year strategic plan, Community

Schools, Thriving Students. Adopted by the Board of Education in June 2011, the plan calls for

building community schools across the district that ensure high-quality instruction; develop

social, emotional and physical health; and create equitable opportunities for learning. Urban

Strategies Council has worked with the school district, community members and other

stakeholders to support this system reform in several ways:

� Community Schools Strategic Planning: Urban Strategies Council facilitated six School

Board retreats over a 14-month period to help develop the strategic plan. As part of that

process, the District created 14 task forces to produce recommendations for the plan, with

Urban Strategies Council facilitating one task force and sitting on several others.

� Full Service Community Schools Task Force: Urban Strategies Council convened and co-

facilitated the Full Service Community Schools and District Task Force, which created a

structural framework and tools for planning and implementation, and produced a report

with a set of recommendations that formed the foundation of the strategic plan.

� Community Engagement in Planning: Urban Strategies Council partnered with the district

to educate and engage more than 900 school and community stakeholders on how

community schools could best serve them.

� Planning for Community Schools Leadership Council: Urban Strategies Council has been

working with OUSD’s Department of Family, School and Community Partnerships to lay the

groundwork for building an interagency, cross-sector partnership body that will provide

high-level system oversight and support, and ensure shared responsibility and coordination

of resources towards the vision of healthy, thriving children supported through community

schools.

� Convening Workgroups: Urban Strategies Council continues to partner with the District to

convene and facilitate several workgroups developing specific structures, processes, and

practices supporting community school implementation, as well as informing the eventual

work of the Community Schools Leadership Council.

� African American Male Achievement Initiative: Urban Strategies Council is a partner in

OUSD’s African American Male Achievement Initiative (AAMAI), a collaboration supporting

efforts to close the achievement gap and improve other key outcomes for African American

males in OUSD. Urban Strategies Council has developed data-based research; explored

promising practices, programs and policies inside and outside the school district; analyzed

the impact of existing system-wide policies; and developed policy recommendations to

improve outcomes in various areas identified by the AAMAI Task Force.

� Boys and Men of Color: Urban Strategies Council is the Regional Convener for the Oakland

Boys and Men of Color site, which adopted community schools as a vehicle to improve

health, education and employment outcomes for boys and men of color.

6


Community School Evaluations: Literature Review

Introduction

Evaluations of community school initiatives are integral to understanding the impact of the

community school strategy to improve student learning while building stronger families and

healthier communities1. The evaluations expose the challenges and limitations of assessing the

community school strategy, which is critical for designing a more accurate evaluation in the

future. Furthermore, they reveal the effectiveness of supports and operational elements and

help inform future improvements for those areas. We used four central research questions to

guide the literature review of community school evaluations:

1. What methodology was employed for the evaluation?

2. What were the indicators and measures of success?

3. What were the evaluation findings?

4. What were the challenges and limitations throughout the evaluation process?

A compilation of recent research on community school evaluations published in 2009 shows

trends in student improvement on reading and math standardized test scores2. Other

significant trends include reduced drop-out rates and improved attendance3, improved

behavior and youth development4, and greater parental engagement.5 While there is some

research on the community schools evaluation process (most notably the “Evaluation Toolkit”

compiled by the Coalition for Community Schools), what seems to be lacking in research and

scholarship is an explicit focus on how to integrate equity into the evaluation process.

Review

1. Methodology

A combination of qualitative and quantitative assessment methods were employed in all of the

community school evaluations. The most common qualitative approach was the use of surveys

1 Coalition for Community Schools. “What is a Community School?” Coalition for Community Schools, 2012. Web.

24 April 2012. <http://www.communityschools.org/aboutschools/what_is_a_community_school.aspx>. 2 A study of Communities in Schools, a national community school model found: net increases of +6.0% in grade 8

math and +5.1% in grade 8 reading scores for high-implementing community schools over their matched

comparison group. Net increases in math scores for all grades over their comparison groups (+2.5% urban, +3.3%

rural). Net increases in math for schools predominantly serving traditionally-low performing populations. 3 Communities in Schools found net increases of +0.2% in elementary, +0.1% in middle, and +0.3% in high school

for high-implementing community schools over their matched comparison group. 4 Blank, M., A. Melaville, and B. Shah. “Making the difference: Research and practice in community schools.”

Washington, DC: Coalition for Community Schools, Institute for Educational Leadership, 2003. Web. 24 April 2012. 5 Coalition for Community Schools. “Community Schools Research Brief: 2009.” Coalition for Community Schools,

2009. Web. 24 April 2012.

<http://www.communityschools.org/assets/1/AssetManager/CCS%20Research%20Report2009.pdf>.

7


to better understand issues such as school climate, trust between students and adults, and

feelings of personal improvement among students. Interviews and student focus groups were

also commonly used to gain a better understanding of the aforementioned issues. In many

cases, key documents were reviewed to assess the alignment of community school initiatives

with their shared vision and goals.

All of the evaluations utilized a public database or other data management system to collect

and analyze quantifiable data. The type of data collected ranged from student academic

achievement and attendance to individual program participation. The most common evaluation

design was the longitudinal study, in which the improvement of a cohort was documented over

a period of time. Most frequently, a cohort of community school participants was compared

with non-participants over a period of time. In a few cases, randomized controlled trials (RCTs)

were employed and pre/post cohorts of participants and non-participants were examined. A

few evaluations used state benchmarks to measure the success of community school

participants against other students in the district or state. The majority of the evaluations took

place during a 1-3 year period.

2. Indicators and Measures of Success

The most common indicators of success were standardized test scores for math and reading

subject areas, attendance rates and survey results. Other frequently used indicators include

suspension and expulsion rates, disciplinary infractions and graduation credits (for high school

students only). There were a few evaluations that looked at increases in parental engagement

as an indicator of success; however, measurement of parental engagement was difficult and

oftentimes done arbitrarily through voluntary surveys.

One evaluation which stood out was the Cincinnati Community Learning Centers, which

employed a “Learning Partner Dashboard” to measure student success. The aforementioned

indicators of success were used but individual student data was disaggregated and student

improvement was correlated with rates of participation in specific programs6.

3. Outcomes

Of the 25 studies and over 70 school initiatives reviewed, the most common outcome among

community schools was improved academic achievement on math and reading standardized

tests. Findings indicate that the vast majority of students participating in community school

programs outperformed free/reduced lunch students in state math and reading tests at

comparison schools. Community schools also increased the number of students meeting

proficiency levels and state benchmarks for reading and math. In many evaluations, academic

6 Mitchell, Dr. Monica. “Community Learning Centers: Year in Review 2010-2011.” Cincinnati Public Schools,

INNOVATIONS in Community Research, 2011. Web. 19 December 2011.

<http://news.cincinnati.com/assets/AB1820921121.PDF>.

8


achievement had a positive correlation with the number of days students attended community

school programs.

Other significant findings from community schools include increased Academic Performance

Index scores, increased credits toward graduation and graduation rates, higher attendance

rates, more schools meeting Annual Yearly Progress and lower rates of disciplinary infractions.

Survey outcomes were positive and the majority of students who responded to surveys felt

supported academically, welcome and safe at school. Significant survey outcomes include

improved student mental health, increased parental engagement and increased feelings of

collective trust. It is important to note that only a few programs cited better access to

healthcare or improved general health in their outcomes.

Conclusion

1. Methodological Flaws & Challenges

Collecting student data from school sites seemed to present one of the greatest challenges to

the evaluation process. Issues of parental consent likely contributed to inconsistencies or

inaccuracies in student surveys aimed at obtaining demographic information or participant

feedback data. The students that did receive parental consent were not obligated to fill out

surveys and the voluntary responses may have skewed the results.

Another challenge presented itself in the actual methodology of most evaluations. Many

evaluations did not have a control group and it was not certain that comparison cohorts

mirrored the circumstances of one another. The lack of a control for the differences between

participants and non-participants in community school programs indicates there may have

been unexplored disparities in student data. Unexplored disparities among participants and

non-participants include factors such as prior and current student grade point averages, test

scores, race and socioeconomic status, all of which could have altered the findings.

2. Promising Practices

The “Learning Partner Dashboard” database used by the Cincinnati Community Learning

Centers7 stood out as the most promising tool for collecting and analyzing individual student

data and improvement. Data was disaggregated by multiple “priority factors”, some of which

include race, non-proficiency on standardized tests, five or more behavior referrals and five or

more absences. Individual student data was then assessed in relation to rates of participation in

specific programs. This allowed for the centers to not only evaluate individual student success

but the success of specific programs.

7 Mitchell, Dr. Monica. “Community Learning Centers: Year in Review 2010-2011.” Cincinnati Public Schools,

INNOVATIONS in Community Research, 2011. Web. 19 December 2011.


9


3. Concluding Remarks: Being Intentional about Equity

The findings of the evaluations are consistent with previously published research on community

schools and align with the community school vision. What is missing from the evaluations,

however, is an intentional look at which students are benefitting from the community school

strategy and which students continue to experience disparate outcomes. The vast majority of

the evaluations conclude that outcomes are improving for the overall student population, but

few evaluations address the experiences or outcomes of student subpopulations. Community

schools need to be more intentional about integrating equity into the evaluation process in

order to develop a complete understanding of who is benefitting from services and how they

need to be targeted. A more comprehensive system of tracking and disaggregating student data

would produce more accurate findings and more equitable outcomes in the future.

10


Community School Evaluations: Annotated Bibliography

The Community School Effect: Evidence from an Evaluation of the Tulsa Area Community

School Initiative

Adams, Curt M. The Oklahoma Center for Educational Policy, November 2010. Web.19

December 2011.

<http://www.csctulsa.org/files/file/Achievement%20Evidence%20from%20an%20Evaluation%2

0of%20TACSI.pdf>.

The 2008-2009 report evaluates the achievement effect of the community school model in

Tulsa, Oklahoma. The Tulsa Area Community Schools Initiative (TACSI) is a participant in the

national Coalition for Community Schools. TACSI programs intend to target low-income

students; however, the evaluation did not disaggregate data by socioeconomic status, race or

other indicators. Instead, fifth grade student data was collected from 18 TACSI schools and 18

non-TACSI schools with comparable demographics.

The report suggests that the comparable school demographics (across indicators of poverty,

average teacher experience, school size, and student ethnicity) in TACSI and non-TACSI schools

reduced the probability that achievement differences were the result of confounding factors or

selection bias. It also suggests that bringing the community school model to scale in TACSI

schools enhances student achievement and narrows the achievement gap. When isolating the

poverty effect, results indicate that students in TACSI schools significantly outperformed

free/reduced lunch students in state math and reading tests than the comparison schools. The

study also found that collective trust mediated the relationship between student poverty level

and achievement.

� Methodology:

1. Cross-sectional and ex post facto data collection and analysis on state math and

reading scores for a sample of fifth grade students at 18 TACSI and 18 non-TACSI

schools

2. Surveys

3. Use of a poverty differential to account for individual achievement

4. Use of Optimal Design 2.0 to test the power of the sample in detecting significant

achievement differences

5. Use of multilevel modeling with HLM 6.04 to evaluate the achievement effect

6. Use of Community School Development Scale to measure development of the

community school model in TACSI schools (four levels of community school

diffusion: inquiring, emerging, mentoring, and sustaining)

� Indicators & measures of success:

1. State math and reading exam scores

2. Survey results

� Tools: Sample Beacon scorecards (Appendix C)

11


Evaluation of the San Francisco Beacon Initiative

Social Policy Research Associates. Social Policy Research Associates, November 13, 2008. Web.

19 December 2011.

<http://www.sfbeacon.org/00_Evaluations/Evaluations/2008_Beacon_Evaluation_Report_SPR

_Full_Report.pdf>.

The 2007-2008 report evaluates the San Francisco Beacon Centers’ alignment with the Beacon

vision to promote youth and family centers in public schools as beacons of activity for the

surrounding neighborhood. While the eight Beacon Centers in San Francisco are not

community schools, they similarly provide a myriad of services and programs and function as

community hubs. Beacon programs intend to target African American, English Language

Learner, Latino, Pacific Islander, Samoan, and Special Education students; however, the

evaluation did not disaggregate data by race or other indicators.

The majority of youth surveyed felt that people at Beacon respected their culture and heritage,

but few replied that Beacon helped them learn about their culture and heritage. Elementary

school Beacon participants decreased their number of suspensions, while the number of

suspensions for non-Beacon elementary school youth increased. However, the Average Daily

Attendance (ADA) of middle and high school youth decreased across all levels of after school

Beacon participation. There were also inconsistencies with Beacon youth California Standards

Test (CST) proficiency. Non-Beacon elementary school youth actually had higher percentages of

CST proficiency than Beacon youth for ELA and Math; Beacon middle school youth had higher

percentages of CST proficiency than non-Beacon youth; and non-Beacon high school youth had

higher percentages of proficiency than Beacon youth on both tests.

*Reviewer’s comment: The survey response that Beacon did not help students learn about their

culture and heritage suggests that Beacon Centers may need to pay more attention to issues of

race, ethnicity and culture in programming. The inconsistencies in CST proficiency may be due

to differences in the student subpopulations that attend and don’t attend Beacon Centers.

� Methodology:

1. Key informant interviews

2. Surveys

3. Student focus groups

4. Contract Management System (CMS) program for collecting and analyzing data on

attendance, suspension rates, and CST scores in ELA and Math


1. Attendance

2. Suspension rates

3. CST scores in ELA and Math

4. Survey results

� Tools: Sample Beacon scorecards (Appendix C)

12


Hartford Community Schools Evaluation: Findings from Year 2 (September 2009-November

2010)

HPS and ETO Data Analysis.” The OMG Center for Collaborative Learning. The OMG Center for

Collaborative Learning, February 2011. Web.19 December 2011.

<http://www.communityschools.org/assets/1/AssetManager/Hartford%20Community%20Scho

ols%20summary%20report%2009-10.pdf>.

The 2009-2010 report evaluates the success of the community schools approach in Hartford

Community Schools by the extent that the schools have realized student achievement gains.

Hartford Community Schools provide an integrated approach to K-12 education, focusing on

high academic standards and the provision of social and health supports to improve student-

learning outcomes, and strengthen families and the community. The schools did not target a

specific student subpopulation and did not disaggregate data by race, income or other

indicators. Data was aggregated and compared between participants and non-participants.

In 2010, a higher percentage of community school students scored proficient or above on the

Connecticut Mastery Test (CMT) Reading test than other students in Hartford Schools. A greater

percentage of students participating in the afterschool programs moved at least one level up on

the math and writing CMT tests than students who did not participate. Enrollment in the

afterschool portion of the day reached 25 percent of the community schools’ population.

Attendance rates averaged above 70 percent for all students enrolled in afterschool programs

in any given month over the past academic year. Furthermore, the vast majority of students

who responded to the 2009-2010 survey felt supported academically and welcome at school.

*Reviewer’s comment: The lack of a control for the differences between participants and non-

participants in afterschool programs implies that there are unexplored disparities in the data

(ex: prior and current course grades, test scores, race and socioeconomic status).

� Methodology:

1. Data collection and analysis of CMT Math, Reading and Writing scores from Hartford

Public Schools

2. Data collection and analysis of ETO data including afterschool participation,

attendance, and activity data from the Hartford Office of Youth Services

3. School Climate Survey


1. Attendance

2. CMT Math, Reading and Writing proficiency levels

3. School Climate survey results

13


Final Report: 2009-2011: Hartford Community Schools Evaluation

The OMG Center for Collaborative Learning. The OMG Center for Collaborative Learning,

October, 2011. Web. 12 March 2012.

<http://www.communityschools.org/assets/1/AssetManager/HCS%20Final%20Report%20(2-6-

12).pdf>.

The 2011 final report presents findings, three-year summary assessments and

recommendations for moving Hartford Community Schools forward. Systems-level community

school implementation, school-level implementation and progress toward outcomes at

Hartford Community Schools are explored. Hartford Community Schools provide an integrated

approach to K-12 education, focusing on high academic standards and the provision of social

and health supports to improve student-learning outcomes, and strengthen families and the

community. The schools did not target a specific student subpopulation and data was

compared among participants and non-participants.

Outcomes were explored at the school-level, systems-level, student, family and community

level. At the school-level, outcomes included: more robust community school programs;

increased capacity of lead agency staff; demonstrated value to school leadership; and increased

data quality. Systems-level outcomes included: increased clarity about the importance of a

systems-level focus; district-wide emphasis on school-community partnerships; and expansion

of funding and school partners in HCS. School, student, family and community outcomes were

charted and color-coded according to progress. Most outcomes were positive and met their

indicators of success. Increased interaction between school and lead agency staffs was

identified as making strong progress. Improvements and increasing community school

programming was also identified as making strong progress. Two outcomes, increased family

participation and changes in family behavior, proved to be immeasurable due to a lack of data.

*Reviewer’s comment: The lack of a control for the differences between participants and non-

participants in afterschool programs implies that there are unexplored disparities in the data

(ex: prior and current course grades, test scores, race and socioeconomic status).

� Methodology:

1. In-person local leadership interviews

2. In-person focus groups with community school directors and lead agency managers

3. Follow-up phone interviews with the HCS director and HFPG program officer

4. Secondary data requests from Hartford Public Schools and Hartford Office of Youth

Services School Climate Survey


1. Increased interaction between school and lead agency staffs

2. Increased use of data

3. Changes in community school programming

4. Changes in classroom/school environment

5. Increased student access to afterschool programs

6. Increased utilization/participation in afterschool programs

14


7. Increased outreach and programs for families

8. Increased family engagement

9. Changes in family behavior

10. Increased awareness of Hartford Community Schools in the community

11. Increased community access to Hartford Community Schools

12. Increased community involvement in schools

13. School Climate survey results

14. Attendance

15. CMT Math, Reading and Writing proficiency levels

� Tools:

1. Appendix A: Hartford Community Schools Partnership Structure

2. Appendix C: Key Milestones, Activities, and Contextual Changes

15


Community Learning Centers: Year in Review 2010-2011

Mitchell, Dr. Monica. Cincinnati Public Schools, INNOVATIONS in Community Research, 2011.

Web. 19 December 2011.


The 2010-2011 annual performance report evaluates the outcomes of individualized student

services related to tutoring, mentoring, afterschool programs and college access at the

Cincinnati Public Schools Community Learning Centers (CLC). CLC is a district-wide initiative

designed to provide academic reinforcements for students and develop community-centered

“hubs” of services and resources through co-located partnerships. The centers specifically

target students who have one or more “priority factors”. These factors include non-proficiency

in Ohio Achievement Assessments (OAA) Math scores or Reading scores, five or more absences,

five or more tardies, or five or more behavioral referrals. Data is disaggregated by race and

other indicators through the comprehensive “Learning Partner Dashboard” tool.

Data for 2010-2011 show that academic achievement has improved in CLC schools since the

onset of Resource Coordination. Students receiving student support services, including

tutoring, mentoring, college access and afterschool, all showed positive academic trends from

2009-2010 to 2010-2011. “Priority factor” students with tutors made important gains on OAA

Math and Reading scaled scores from 2009-2010 to 2010-2011. “Priority factor” students with

College Access services made important gains on OAA Math scaled scores from 2009-2010 to

2010-2011. Greater parent volunteerism was positively linked to fewer absences, fewer

behavioral referrals, and lower math and reading priority factors.

� Methodology:

1. Use of “Learning Partner Dashboard” database to track and analyze individual

student needs and service partner coordination

2. Data collection and analysis of OAA Math and Reading scores from Cincinnati Public

Schools

3. Surveys


1. OAA Math and Reading scores for “priority factor” students related to tutoring,

mentoring, afterschool programs and College Access

2. Behavioral referrals for “priority factor” students related to tutoring, mentoring,

afterschool programs and College Access

3. Tardies for “priority factor” students related to tutoring, mentoring, afterschool

programs and College Access

4. Absences for “priority factor” students related to tutoring, mentoring, afterschool

programs and College Access

5. Survey results

16


SUN Community Schools FY 2010-2011

Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,

Multnomah County, Oregon, 2012. Web. 14 May 2012.

<http://web.multco.us/sites/default/files/sun/documents/suncsannual_outcomes1011.pdf>.

The 2010-2011 report evaluates students who attended at least 30 days of Schools Uniting

Neighborhoods Community School (SUN CS) programming, who had signed parental releases

and could be matched to district data. SUN CS provide school-based educational support,

recreation, social and health services, and parent engagement to students and their families. In

2010-2011, there were 60 SUN CS sites at 23 elementary schools, 13 middle schools, 6 high

schools, and 18 schools serving grades K to 8 across Multnomah County, Oregon.

Over 75% of SUN CS students showed improvement in state math and reading scores.

Additionally, nearly 75% of SUN CS students were meeting their reading benchmarks or on

track to reach their benchmarks in three years. Fewer students were on track to meet their

math benchmarks, but the report suggests that this is largely due to the more rigorous cutoff

scores in 2010-2011. SUN CS students attended an average of 94.5% of school days, which is

above the state benchmark of 92%. 17% of SUN CS students were classified as chronically

absent compared to the districts average of 32%. 79% of 12th grade SUN CS students graduated.

*Reviewer’s comment: SUN CS did not employ an equity framework, according to information

obtained from an interview with Peggy Samolinski, the Division Director for SUN System in

Multnomah County. SUN CS use 19 “risk factors” which they believe research has proven to

impact school success; they fall under the umbrella categories of children living in poverty,

children of color, and English learners. Due to complications with parental consent, some staff

learned that students were at-risk only after the students started participating in the programs.

This created difficulties in pre/post data collection and analysis. SUN CS measured the impact of

the program based on participants versus non-participants. They also compared the CS

participants’ outcomes with district and state benchmarks in attendance, reading, math, and

credits earned for graduation (if in high school). At the site level, SUN CS employ ServicePoint, a

web data base that collects information with an activity point module.

� Methodology:

1. Use of ServicePoint at site level

2. Collection and analysis of Oregon Assessment of Knowledge and Skills (OAKS) Math

and Reading score data for SUN CS participants and non-participants

3. Comparison of SUN CS participants’ outcomes with district and state benchmarks in

attendance, reading, math, and credits earned for graduation (if in high school)

4. Surveys


1. OAKS Reading and Math test scores related to individual improvement and state

benchmarks

2. Attendance related to district level

3. Graduation credits (if in high school)

17


SUN Community Schools FY 2009-2010


Multnomah County, Oregon, 2011. Web. 19 December 2011.

<http://web.multco.us/sites/default/files/sun/documents/suncsannualoutcomes0910.pdf>.

The 2009-2010 report evaluates students who attended at least 30 days of Schools Uniting

Neighborhoods Community School (SUN CS) programming, who had signed parental releases

and could be matched to district data. SUN CS provide school-based educational support,

recreation, social and health services, and parent engagement to students and their families. In

2009-2010, there were 60 SUN CS sites at 23 elementary schools, 14 middle schools, 7 high

schools, and 16 schools serving grades K to 8 across Multnomah County, Oregon.

Over 75% of SUN CS students showed improvement in state math and reading scores. In

addition, almost three quarters of SUN CS students were meeting their benchmarks by eighth

grade in reading and math. SUN CS students attended an average of 94.3% of required school

days, which is above the state benchmark of 92%. About 15% of SUN CS students were

classified as chronically absent compared to 25% across the district. Of 12th grade SUN CS

students, 81% graduated.


1. OAKS Reading and Math test scores related to individual improvement and state

benchmarks



18


Social and Support Services for Educational Success FY 2009-2010



<http://web.multco.us/sites/default/files/sun/documents/sssesannualoutcomes0910.pdf>.

Social and Support Services for Educational Success (SSSES) is part of Schools Uniting

Neighborhoods Community Schools (SUN CS) and provides age appropriate and culturally

specific academic support, case management and skill building activities. The most common

types of service include case management, education, skills training and recreation. The

outcomes reported are for youth who participated in at least 45 days and 15 hours of services

during 2009-2010.

The report finds that SSSES agencies are serving their intended group of children living in

poverty, children of color, and English learners. While students are making progress in reading

and math, only 41% met the state reading benchmark and 42% met the state math benchmark.

However, SSSES students attended an average of 91% of required school days, which is almost

at the state benchmark of 92%. High school students made significant progress in earning

credits, but only 53% earned enough to be on track to graduate in four years. Of 12th graders

participating in SSSES, 53% graduated. Students made progress across six types of case

management goals and over 90% of each goal type showed at least partial progress.

*Reviewer’s comment: The three page report focuses on outcomes and not methodology for

data collection and analysis. According to information obtained from Peggy Samolinski, the

Division Director for SUN CS, there is no equity framework integrated into the evaluation of

SSSES. There is no way to ensure specific student subpopulations participate in the services;

however, the report states that those being served are “at-risk” youth.


1. Case management goals: (1) academic; (2) attendance; (3) self-esteem; (4) social

skills; (5) relationships; and (6) basic needs

2. OAKS Reading and Math test scores related to state benchmarks

3. Grade Point Average

19


Parent Child Development Services FY 2009-2010



<http://web.multco.us/sites/default/files/sun/documents/pcdsannualoutcomes0910.pdf>.

The 2009-2010 report evaluates the services and outcomes on the 476 children who

participated in the Parent-Child Development Services (PCDS) program. PCDS is part of SUN

Community Schools in Oregon and provides parent education, support groups, and young child

playgroups (age two to five). PCDS uses the “Parents as Teachers” curriculum, which is an

evidence-based curriculum designed to teach parenting skills and knowledge. The majority of

the service hours are spent on parent skill training.

The report finds that the PCDS program is serving its intended population of children living in

poverty, children of color, and parents whose primary language is not English. Half of the

families stay in the program for at least one year. Research in the national Parents as Teachers

evaluation shows that two years of home visitation combined with one year of pre-school is

most effective at getting students ready to start kindergarten. Younger pre-school-aged

children stay in services for longer than other children. The program helps ensure that children

are up-to-date on immunizations, developmentally screened and referred to early intervention

services if needed. Service exit surveys for parents also show very positive feedback.

*Reviewer’s comment: The three page report focuses on outcomes and not methodology for

data collection and analysis. According to information obtained from Peggy Samolinski, the

Division Director for SUN CS, there is no equity framework integrated into the evaluation of

PCDS.


1. Kindergarten readiness (does not state specific measure)

2. Survey results (parent satisfaction with services and skill-building)

20


Executive Summary: SUN Service System Results - Fiscal Year 09/10



<http://web.multco.us/sites/default/files/sun/documents/executivesummary0910.pdf>.

The 2009-2010 executive summary provides an overview of Schools Uniting Neighborhoods

Community Schools (SUN CS) progress and outcomes. SUN CS provide school-based educational

support, recreation, social and health services, and parent engagement to students and their

families. In 2010, SUN CS programs served a notably higher percent of children living in poverty,

children of color and English learners compared to the school districts’ average. Outcomes were

reported for students who participate in SUN CS for 30 days or more.

The average daily attendance rate of SUN CS students was 94%, which exceeded the state

benchmark of 92%. SUN CS students’ average gains were equal to or higher than expected for

fourth, sixth and seventh grades in reading and math. Benchmarks state that high school

students need to earn 6.0 credits per year to be on track for graduation in four years and

students in SUN CS earned an average of 6.2 credits per year. About 99% of parents who

participated in Parent Child Development Services reported acquiring new skills. Almost all of

the families threatened by homelessness (94%) remained in permanent housing 12 months

after receiving rent supports.

*Reviewer’s comment: The report focuses on outcomes and not methodology for data

collection and analysis. According to information obtained from Peggy Samolinski, the Division

Director for SUN CS, there is no equity framework integrated into the evaluation of SUN CS.

� Methodology:

1. Collection and analysis of Oregon Assessment of Knowledge and Skills (OAKS) Math

and Reading score data for SUN CS participants and non-participants

2. Comparison of SUN CS participants’ outcomes with district and state benchmarks in

attendance, reading, math, and credits earned for graduation (if in high school)

3. Surveys


1. OAKS Reading and Math test scores related to state benchmarks



4. Survey results

5. Permanent housing (no specific measured stated)

21


Summary of the Children’s Aid Society Community School Results to Date

Children’s Aid Society. Children’s Aid Society, January 2006. Web. 19 December 2011.

<http://www.aypf.org/documents/SummaryoftheChildrensAidSocietyCommunitySchoolsResult

s.pdf>.

The report summarizes various evaluation outcomes of CAS community school programs and

services by outside entities over a 13-year period. The goal of the Children’s Aid Society (CAS) is

to affect youth, families, and schools by reducing barriers to learning so that children are

physically, emotionally and socially prepared to learn.

The outcomes for youth show improvements in reading and math proficiency on standardized

tests. Additionally, academic achievement has had a positive correlation with the number of

days students attended community school extended-day programs. Teachers reported

improvements in behavioral conduct and qualitative data showed improvements in student

mental and physical health. Some of the most significant outcomes for families include much

higher parental involvement in CAS Community Schools than in comparison schools and

increases in the quality and size of parent social support networks. The mothers participating in

the Early Head Start program reported decreases in depression and stress over the course of

participation in the program. One of the most significant outcomes included teachers in

community schools being able to spend more time on teaching than their counterparts in

comparison schools.

*Reviewer’s comment: The Children’s Aid Society report focuses on outcomes and not

methodology for data collection and analysis.


1. Reading and math proficiency levels on standardized tests

2. School report card data

3. Grade point average

4. Attendance

5. Survey results

6. Parental involvement (no specific measure stated)

22


Communities in Schools National Evaluation: Five Year Summary Report

ICF International. Communities in Schools, October 2010.

<http://www.communitiesinschools.org/media/uploads/attachments/Communities_In_Schools

_National_Evaluation_Five_Year_Summary_Report.pdf>.

The 2010 report measures national outcomes in Communities in Schools (CIS) over a five year

period in order to understand how and why community schools work. CIS is a national

federation of independent 501(c)3 organizations that consist of a national office, state offices,

and local affiliate offices serving students in 3,400 schools. Their mission is to engage

community partners and volunteers in order to effectively address both the academic and

human service needs of students. There was no specific student subpopulation targeted in the

evaluations and no disaggregated data by race.

“High implementer” CIS schools had considerably greater effects on reducing dropout rates and

increasing on-time graduation than their non-CIS comparisons and other CIS schools (i.e.

“partial implementers”). Results from the school-level quasi-experimental study indicate that

CIS students experienced consistent improvements in attendance and state-mandated test

scores. Fewer CIS case-managed students dropped out of school during their 9th grade year

than students in the control group. Students who received CIS services for two consecutive

years had more favorable outcomes in all categories than students who received a single year

of CIS service. CIS schools, regardless of urban, suburban, or rural location, outperformed their

comparison “match” schools on most outcomes.

� Methodology:

1. Use of a National Evaluation team to collect, analyze and assess gaps of all CIS data

to date

2. Critical Processes Survey to assess site level process data and create a community

school rubric for “high implementer” or “partial implementer” schools

3. School Level Quasi-Experimental Study (using propensity score matching to compare

CIS with non-CIS sites)

4. External Comparison Study

5. Natural Variation Study to determine key attributes separating successful CIS schools

from the unsuccessful CIS schools on a given outcome

6. Randomized Controlled Trials (RCT’s) to determine the impact of CIS case-managed

services on individual student outcomes


1. Attendance

2. Graduation

3. Drop-out

4. Suspension

5. Behavior referrals

6. Math and ELA performance

23


Comprehensive Evaluation of the Full-Service Community Schools Model in Pennsylvania:

Lincoln and East Allegheny Middle Schools

LaFrance Associates, LLC. LaFrance Associates, LLC, September 2005. Web. 19 December 2011.

<http://www.eisenhowerfoundation.org/docs/Pennsylvania.pdf>.

The 2004-2005 report evaluates the implementation of the full-service community school

(FSCS) model and its program outcomes at Lincoln and East Allegheny Middle Schools. Lincoln

and East Allegheny Middle Schools are among a cohort of schools in five U.S. states that are

participating in the Eisenhower Foundation Full-Service Community Schools replication

initiative. The demographic characteristics of participants were surveyed in the evaluation;

however, data was not disaggregated by race.

The report suggests that a higher level of participation in academic FSCS programs is associated

with better attendance outcomes. Participation in after-school programming also appears to

lead to positive outcomes for youth, especially in their behavior at school, academic

achievement, satisfaction with school, and positive peer and adult relationships. More than

eight in ten parents of FSCS students who responded to the parent survey said that the FSCS

activities have helped their child enjoy school more. Furthermore, FSCS participants have more

positive peer relationships as a result of their participation in after-school programs and have

expressed a greater feeling of safety in school.

*Reviewer’s comment: There may be inconsistencies or inaccuracies when relying on student

surveys to obtain demographic information of FSCS participants. Not all students fill out surveys

and parental consent is required. This could potentially skew results indicating which student

subpopulations are benefitting from FSCS.

� Methodology:

1. Data collection and analysis of school records through management information

systems (MIS)

2. Observations of program operations


4. Youth focus groups

5. Surveys

6. Youth journal-writing exercise and photo project

7. Logic model indicators

8. Randomized Control Trials

9. Quasi-experimental pre/post comparison cohort of FSCS afterschool program

participants and non-participants



2. Attendance

3. Disciplinary action report cards

4. Standardized test scores

5. Survey results

24


Comprehensive Evaluation of the Full-Service Community Schools Model in Maryland: General

Smallwood Middle School

LaFrance Associates, LLC. LaFrance Associates, LLC, August 2005. Web. 19 December 2011.

<http://www.eisenhowerfoundation.org/docs/Maryland.pdf>.


(FSCS) model and its outcomes at General Smallwood Middle School in Maryland. General

Smallwood is one of a cohort of schools in the U.S. participating in the Eisenhower Foundation

Full-Service Community Schools replication initiative. The demographic characteristics of

participants were surveyed in the evaluation; however, data was not disaggregated by race.

At General Smallwood Middle School, there were no significant findings for the relationship of

FSCS program participation with attendance. Overall, the students who participated in FSCS

activities were those with more absences and behavioral incidences than students who did not

end up participating. FSCS participants did show an improvement in math grades which was

higher than the improvement shown for non-participants. Moreover, participation in after-

school programming appears to lead to positive outcomes in youth behavior at school,

academic achievement, satisfaction with school, and positive peer and adult relationships.





� Methodology:


systems (MIS)




5. Parent focus group

6. Surveys








2. Attendance



5. Survey results

25


Comprehensive Evaluation of the Full-Service Community Schools Model in Iowa: Harding

Middle School and Moulton Extended Learning Center


<http://www.eisenhowerfoundation.org/docs/Iowa.pdf>.


(FSCS) model and its outcomes at Harding Middle School and Moulton Extended Learning

Center in Iowa. Harding Middle School is one of a cohort of schools in the U.S. participating in

the Eisenhower Foundation Full-Service Community Schools replication initiative. The

evaluation focuses on the change experienced by students and parents and impacts at the

school level. The demographic characteristics of participants were surveyed in the evaluation;

however, data was not disaggregated by race.

Students who participated in FSCS showed more improvement in their grades than non-

participants. FSCS participants were almost six times more likely to have shown improvement in

their attitude toward school than students who did not participate. High participants benefited

more than those who participated less. Participation in the enrichment programs is associated

with a year-to-year increase in English grades. Additionally, FSCS participants have greater

positive adult relationships as a result of their participation in after-school programs and

expressed satisfaction with school.





� Methodology:


systems (MIS)




5. Surveys








2. Attendance



5. Survey results

26


Comprehensive Evaluation of the Full Service Community Schools Model in Washington:

Showalter Middle School


<http://www.eisenhowerfoundation.org/docs/Showalter2.pdf>.


(FSCS) model and its outcomes at Showalter Middle School in Washington. Showalter Middle

School is one of a cohort of schools in the U.S. participating in the Eisenhower Foundation Full-

Service Community Schools replication initiative. The demographic characteristics of

participants were surveyed in the evaluation; however, data was not disaggregated by race.

One of the most significant findings is that FSCS participation leads to decreased behavioral

incidents at Showalter Middle School. Those who participated in FSCS afterschool programs

improved their behavior significantly more than those who did not participate. Holding all else

equal, those who participated in FSCS afterschool programs improved their math grades more

than non-participants. Survey results also indicated that participation in FSCS programs led to

greater orientation towards learning, more positive peer relationships and improved self-

esteem.





� Methodology:


systems (MIS)




5. Surveys








2. Attendance



5. Survey results

27


Three Years into Chicago’s Community Schools Initiative (CSI): Progress, Challenges, and

Lessons Learned

Whalen, Ph.D Samuel P. College of Education, University of Illinois at Chicago, June 2007. Web.

19 December 2011.

<http://www.aypf.org/documents/CSI_ThreeYearStudy.pdf>.

The report analyzes trends in how Chicago’s Community Schools Initiative (CSI) is building

community school capacity and realizing important benefits for the schools, students, and

families since 2001. CSI includes 110 elementary and high schools and builds upon the core

features of the federal 21st Century Community Learning Centers Program (CLC). These core

features include open resources for afterschool and community use, parent and community

engagement, and social and family support services. Some of the schools include neighborhood

elementary and high schools, magnet and specialty schools, and charter and contract schools.

The evaluation did not disaggregate data by race or other indicators.

Overall enrollment of students in “out of school time” (OST) increased by 17% between 2005

and 2006, from an average of 156 students to over 183 students per school. A large number of

OST program participants improved their reading and math grades over the course of the year.

More significantly, the average of 49.3 days of attendance per student in 2006 far exceeded the

CLC criteria for regular attendees. The inclusion of students with special needs among CLC

program participants increased 59% from 2005 to 2006. Trends in standardized test results

indicate that CSI schools have steadily closed the gap in achievement between themselves and

the district between 2001 and 2006.

� Methodology:

1. Collection and analysis of individual student participation data, standardized test

performances and school level summary statistics

2. Surveys


4. Analyses of school improvement plans


1. Grade point average in reading and math

2. Attendance (school and OST)

3. Disciplinary infractions

4. Illinois Standards Achievement Test (ISAT) scores in math, science and reading

5. Survey results

28


Report of the Evaluation of the Polk Bros. Foundation’s Full Service Schools Initiative

Whalen, Ph.D Samuel P. Chapin Hall Center for Children at the University of Chicago, April 2002.

Web. 19 December 2011.

<http://www.communityschools.org/assets/1/AssetManager/Polk_Bros_ExecutiveSummary.pd

f>.

The 2002 report summarizes the findings of a three year evaluation of the Full Service Schools

Initiative (FSSI) in Chicago, a pilot project funded by Chicago’s Polk Bros. Foundation. The

purpose of FSSI is to test a research-based framework for expanding school-based and school-

linked services that broaden support for children’s well-being and academic achievement. The

evaluation did not disaggregate data by race or other indicators.

Results indicate that the FSSI framework successfully expanded resources in support of student

growth and learning during the afterschool hours. Academic performance improved at all three

schools during FSSI. In most cases, the rates of improvement exceeded CPS and equaled or

marginally exceeded comparison schools. Furthermore, FSSI schools increased the range and

diversity of their community alliances. The number of teachers involved in planning or providing

after-school activities increased by more than 20% at all three schools. Across the schools,

between one-third and one-half of enrolled students participated in an after-school program.

However, FSSI schools also showed lower student mobility than comparable schools and

steeper rates of decline in mobility.

� Methodology:

1. Collection and analysis of aggregate student achievement, truancy, student mobility

and attendance data for Chicago Public Schools

2. Surveys


4. Focus groups

5. Observations


1. Student mobility, attendance and truancy related to demographically similar schools

and the Chicago Public School District


3. Increased afterschool resource opportunities

4. Increased community partnerships

5. Increased teacher involvement in FSSI programs

6. Increased student participation in FSSI programs

7. Increased supportive adult relationships for students

8. Survey results

29


Achievement Plus Evaluation: 2009-2010

Mueller, Dan, Katie Broton, and Edith Gozali-Lee. Wilder Research, September 2010. Web. 19

December 2011.

<http://www.wilder.org/download.0.html?report=2341>.

The 2009-2010 report evaluates how Achievement Plus fits into community school efforts

nationally, how well community partnerships are working, and how well students at these

schools are performing. Achievement Plus is a private/public partnership with the primary goal

of improving student achievement in low socio-economic areas of the city through academics,

extended afterschool programs, and learning supports for families, students and community

members at schools. Programs intend to target students of low socioeconomic status; however,

the evaluation did not disaggregate data by income, race or other indicators.

Results from the 2010 state-mandated Minnesota Comprehensive Assessments (MCA) indicate

that proficiency levels in reading and math are improving. At one school, the math proficiency

rate increased substantially and exceeded the statewide rate in 2010. Additionally, results of

interviews with community partner representatives indicate that partners believe they

contribute to children’s school success by eliminating or reducing barriers to children’s learning.

There are currently nine Achievement Plus community partners filling a need identified by

school staff or neighborhood residents.

� Methodology:

1. Collection and analysis of student achievement data

2. Literature reviews of community schools

3. Analysis of service records



1. Minnesota Comprehensive Assessments (MCA-II) in math and reading

2. Key informant interview results

3. Partner alignment with children’s needs

30


Community Schools—Results That Turn Around Failing Schools: Test Scores, Attendance,

Graduation and College-Going Rates

Coalition for Community Schools. Coalition for Community Schools, May 2010. Web. 19

December 2011.

<http://www.communityschools.org/assets/1/AssetManager/Turning_Around_Schools_CS_Res

ults2.pdf>.

The 2010 report summarizes research outcomes that were gathered from community schools

across the nation from 2007-2009. The Coalition for Community Schools is an alliance of

national, state and local organizations involved in education, youth development, health and

human services, and community schools among other areas. There is no evaluation

methodology provided in the report. Consequently, the report does not mention if an equity

framework was employed in the evaluation process. Some of the most significant findings from

community schools across the nation include higher scores on math and reading standardized

tests, increased Academic Performance Index scores, increasing graduation rates, more parent

involvement, more schools meeting Annual Yearly Progress, higher attendance rates and lower

rates of disciplinary infractions.



2. Academic Performance Index

3. Annual Yearly Progress

4. Attendance

5. Graduation

6. Drop-outs

7. Behavioral referrals

8. Parental involvement

31


Raising Graduation and College Going Rates—Community High School Case Studies

Axelroth, Rita. Coalition for Community Schools and National Association of Secondary School

Principals, August 2009. Web. 19 December 2011.

<http://www.eric.ed.gov/PDFS/ED506727.pdf>.

The report provides an individual overview of eight community high school’s programs and

services, partnerships, leadership and results. There is no evaluation methodology provided in

the report. The eight featured schools include George Washington Community High School,

Fannie Lou Hammer High School, Oyler Community Learning Center, Parkrose High School,

Foster High School, Community Links High School, Little Village Lawndale High School and Sayre

High School. Across the schools, at least 60 percent of students are eligible for free or reduced-

price lunch, while most of the schools have rates of 90 percent or more. They also represent

some of the poorest and most ethnically diverse schools in the country.

Some of the most significant results across all eight schools include positive changes in

academic achievement and preparation for post-secondary success. Attendance has

dramatically increased among all eight community schools, exceeding the district goal in most

cases. Graduation and college-acceptance rates have also increased and at most schools they

exceed the district-wide average. At all eight schools there have been significant decreases in

dropout rates.



2. State benchmarks

3. Attendance

4. Graduation

5. Drop-outs

6. College enrollment

7. Survey results

32


Evaluation of Community Schools: Findings to Date

Dryfoos, Joy. Coalition for Community Schools, 2000. Web. 19 December 2011.

<http://www.communityschools.org/assets/1/AssetManager/Evaluation%20of%20Community

%20Schools_joy_dryfoos.pdf#xml=http://prdtsearch001.americaneagle.com/service/search.asp

?cmd=pdfhits&DocId=737&Index=F%3a%5cdtSearch%5ccommunityschools&HitCount=17&hits

=5+6+7+4a+36a+410+49a+4a4+608+659+b55+fa0+2a83+3426+44ef+45fc+461c+&hc=1394&re

q=findings+to+date>.

The report provides research findings from 49 evaluations of community school initiatives. The

programs vary in their length of existence, evaluation method and duration. One limitation in

the evaluation process is that some reports aggregated findings for hundreds of schools and not

at state, community, and local site levels.

The most common achievement among the schools was academic improvement, with 36 of the

49 programs reporting academic gains in reading and math standardized test scores over a two-

three year period. In at least eight of the cases; however, the outcomes were limited to

students who received special services, such as case management or extended day sessions.

Nineteen programs reported improvements in school attendance and several mentioned higher

teacher attendance rates. Eleven programs reported a reduction in suspensions and eleven

programs reported reductions in rates of disruptive behavior in the classroom. At least 12 of

the programs reported increases in parent involvement. Only a few programs cited better

access to health care, lower hospitalization rates, higher immunization rates, or access to

dental care.

� Methodology (varied for each evaluation):

1. Collection and analysis of student achievement data through management

information systems (MIS)

2. Use of control cohorts

3. Use of non-representative small samples

4. Use of comparison studies with “match” schools or participants vs. non-participants



2. Attendance

3. Graduation

4. Suspension and expulsion

5. Access to support services

6. Parental involvement

7. Survey results

33


Community Schools Collaboration Evaluation Report

Murray, Mary, and Jessica Ganet. MEM Consultants, September 2010. Web. 28 June 2012.

<http://cscwa.org/sites/default/files/pdfs/CSC%20Evaluation%20Report%202009-

2010%20FINAL.pdf>.

The 2009-2010 report evaluates the impact of expanded learning activities in 14 community

schools in the Tukwila and Highline School Districts in King County, Washington. The

Community Schools Collaboration (CSC) implements full service community schools in 16

schools in the Tukwila and Highline School Districts in south King County. Although not explicitly

stated, CSC intends to serve free/reduced lunch students, English learners and students of

color. While demographics were documented, the evaluation did not disaggregate outcomes by

race or other indicators.

CSC had over 2,000 students that attended programs and over 1,000 that attended programs

for 30 days or more. High school aged CSC participants reported feeling high expectations for

their future. Student survey results indicated that CSC increased school motivation in three

ways: by increasing excitement to participate in CSC afterschool options, by fostering

confidence in the classroom, and by providing options that help student avoid disciplinary

problems. Teachers reported improvement in homework completion and quality and academic

performance among CSC participants over the course of the school year. Findings also

demonstrated that CSC supports students to fulfill graduation requirements, informs them

about post-secondary opportunities and provides college student role models.

� Methodology:

1. Program attendance records

2. Student and teacher surveys

3. Student focus groups


1. Attendance and program attendance

2. Focus group results

3. Survey results


34


The Economic Impact of Communities in Schools

ESMI. ESMI, May 2012. Web. 5 July 2012.

<http://www.communitiesinschools.org/about/publications/publication/economic-impact-

communities-schools>.

ESMI conducted a five-year national evaluation of the Communities In Schools’ (CIS) model to

help at-risk students stay in school and graduate. CIS works within the public school system to

determine student needs and establish relationships with partners to provide needed resources

(in over 100 high schools). The evaluation measures the overall economic impact of CIS rather

than individual community school results.

ESMI found that the net present value of the CIS benefits in 113 high school-serving affiliates

exceeds the total investment costs by almost $2.6 billion. They calculated the average annual

return to society resulting from CIS’ investment at 18.4%. Every one dollar of CIS investment

created about eleven dollars of economic benefit. ESMI also found that it will take nine years

before all investment costs are fully recovered. Further, the analysis found that the CIS model

created social savings by increasing students’ disposable income by $63 million annually and

reducing social costs due to smoking, alcoholism, crime, welfare, and unemployment.

Moreover, the evaluation showed CIS’ intensive case managed services have produced the

strongest reduction in dropout rates of any existing fully scaled dropout prevention program

that has been evaluated, and that CIS’ model is effective across states, school settings (urban,

suburban, rural), grade levels, and student ethnicities.

� Methodology:

1. Cost-benefit investment analysis of CIS’ high school-serving affiliates (benefits

include higher earnings for students who progress through high school and graduate as

well as social or taxpayer savings created and captured based on the student’s increased

academic achievement)

� Indicators & Measures of Success:

1. High school graduation and drop-out rates\

2. Cost-benefit ratio

35


Positive Student Outcomes in Community Schools

Sebastian Castrechini and Rebecca A. London. Center for American Progress and John W.

Gardner Center for Youth and their Communities, February 2012. Web. 20 July 2012.

<http://www.rwc2020.org/uploads/positive_student_outcomes.pdf>.

The 2007-2011 evaluation conducted by the John W. Gardner Center for Youth and their

Communities (JGC) and the Redwood City School District (RCSD) analyzes student participation

and outcomes in five K-8 schools in the Redwood City School District. Students enrolled in the

five schools were 89 percent Latino, 68 percent were receiving subsidized meal plans and 67

percent were English learners. The evaluation disaggregated student demographic information

in order to compare outcomes by a number of factors and understand their impact on specific

populations. Community school programs at each school were grouped into the following three

categories: Family Engagement; Extended Learning; and Support.

Supplemental community school programs reached more than 70 percent of the students

enrolled at those schools, including high rates of students who were English learners, eligible

for subsidized meals and had parents who had not completed high school. English learner

students that consistently participation in community school programs showed gains in English

language development scores. The evaluation also found that in the elementary grades,

language development gains were tied to family engagement participation, but continued gains

during middle school were associated with frequent extended learning program participation.

Furthermore, students with family engagement in elementary school entered middle school

reporting that their school provided a supportive environment more than students without

family engagement. In middle school, frequent participation in extended learning programs was

linked to increases in students’ perceptions of their school as a supportive environment. Feeling

supported at school was associated with gains in math achievement for all students and English

language development scores for English learners.

� Methodology:

1. Youth Data Archive (to match and compare individual student data across agencies)

2. Surveys


1. Attendance

2. Discipline

3. Math and English California Standards Test (CST) scores

4. California English Language Development Test (CELDT)

5. Student demographic data

6. Program participation

7. Survey results

� Tools: Community school results framework mapped to indicators and potential data

sources (pgs. 35-36)