AFTER SCHOOLEVALUATION REPORT

www.experienceBELL.org

AFTER SCHOOL 2011-12 Report Card

experienceBELL.org

30th à 35th Percentile Rank Increase, Reading

27th à 32nd Percentile Rank Increase, Math

91% Parents reporting that they are more engaged with their child’s education because of BELL

89% Parents expressing satisfaction with BELL’s services

K-8 Grades Served

87% Average Daily Attendance

6,184 Scholars

93% Teachers expressing satisfaction with BELL’s program

32%

27%

35%

30%

0% 10% 20% 30% 40% 50%

BELL (Building Educated Leaders for Life) exists to transform the academic achievements, self-confidence, and life trajectories of children living in under-resourced, urban communities. In the BELL After School program, scholars in grades K-8 engage in after-school literacy and math instruction and hands-on enrichment courses. The program also engages parents and positive adult role models from the community. BELL partners with schools to expand learning time and reach the scholars who need it most.

During the 2011-12 school year, 6,184 scholars participated in BELL After School at 48 schools in Baltimore, MD; Boston, MA; Newark, NJ; and New York City, NY. BELL measures program outcomes against three major goals for scholars: improved academic performance, enhanced self-concept and self-efficacy, and increased social skills. Additionally, evaluation activities seek to assess parent engagement and satisfaction with the program.

PERCENTILE RANK INCREASE

In BELL After School, scholars increased their percentile rank scores. Percentile rank scores range from a low of 1 to a high of 99, with 50 representing the middle score and denoting average performance. They provide a relative measure that compares BELL scholar performance to national norms. On average, when enrolling in BELL After School, scholars were underperforming: they entered at the 30th percentile in reading and the 27th in math. In BELL After School, scholars increased their percentile rank scores to the 35th and 32nd percentiles in reading and math, significantly reducing their deviation from the norm. Reducing this deviation is, in effect, equivalent to narrowing the achievement gap.

Elementary School Scholars

•  % Rank Increase, Literacy: +5%

•  % Rank Increase, Math: +7%

Middle School Scholars

•  % Rank Increase, Literacy: +4%

•  % Rank Increase, Math: +5%

NORMAL CURVE EQUIVALENT SCORES

Scholars gained new literacy and math skills at a faster rate than their peers during the school year, according to an positive NCE score on standardized diagnostic tests. Normal curve equivalent (NCE) scores show a student’s relative position compared to others in the same grade and tested at the same time of year.  A gain in NCE indicates that the student has “grown” more than the norm group.  The average student demonstrates no change, for a NCE score of zero. BELL scholars are targeted for underperformance in school and need to learn at an accelerated rate in order to succeed academically (i.e., demonstrate a positive change in NCE scores).

Average Percentile Rank Increase by Subject

Reading

Math

AFTER SCHOOL 2011-12 Evaluation Report

Fall

Spring

Fall

Spring

Together, increases in percentile rank scores and positive NCE scores confirm that participation in the BELL After School programs boosts scholars’ engagement in learning and academic achievement. This conclusion is further validated by teacher and parent survey data.

95%

94%

89%

95%

91%

88%

93%

80%

81%

87%

91%

89%

90%

92%

88%

85%

93%

90%

93%

87%

89%

0% 25% 50% 75% 100%

Parents reporting that they would recommend BELL to other parents."

Parents reporting that they would enroll their child in BELL again. "

Parents expressing satisfaction with BELL's services."

Parents reporting that their child enjoyed BELL."

Parents reporting that they felt more engaged in their child's education."

Parents reporting that scholars cooperate more with other children."

Parents reporting that scholars are better at group actvities."

Teachers reporting that scholars were better at self-control."

Teachers reporting that scholars were better with peer relations."

Teachers reporting that scholars were better at group activities."

Parents reporting that BELL improved their child's ability to express their ideas."

Parents reporting that BELL improved their child's attitude toward school."

Parents reporting that BELL improved their child's attitude toward learning. "

Parents reporting that BELL improved their child's self-confidence."

Teachers reporting that BELL improved scholars ability to express their ideas."

Teachers reporting that BELL improved scholars' ability to ask for help"

Teachers reporting that BELL improved scholars' self-confidence."

Parents reporting that BELL improved scholars' math skills"

Parents reporting that BELL improved scholars' reading skills"

Teachers reporting that BELL improved scholars' math skills"

Teachers reporting that BELL improved scholars' reading skills"

Parent Engagement & Satisfaction

Parent & Teacher Observations of Community Skills

Parent & Teacher Observations of Scholars' Academic Performance

Parent & Teacher Observations of Self-Efficacy Skills

TEACHER & PARENT SURVEY DATA

METHODOLOGY

TEST DATA: The following standardized, validated instruments were administered pre- and post-program and used to measure program outcomes: the fourth edition of the Stanford Diagnostic Reading Test (SDRT-IV) (Harcourt Educational Measurement, 1996), a diagnostic assessment of grade-level literacy skills, was administered to all scholars; and the fourth edition of the Stanford Diagnostic Mathematics Test (SDMT-IV) (Harcourt Educational Measurement, 1996), a diagnostic assessment of grade-level mathematics skills, was administered to scholars in grades 2 and higher.

SELF-EFFICACY DATA: BELL uses the Youth Outcome Toolkit, a validated, youth-respondent instrument, to measure changes in scholars’ academic behaviors, sense of self, positive core values, and life skills. These attitudes, skills, and behaviors are highly predictive of future school persistence and success. In addition, at the beginning and end of BELL programs, scholars in grades 3 and higher complete a brief survey designed by BELL to assess their academic aspirations. experienceBELL.org

EVALUATION ADVISORY BOARD BELL is nationally recognized for its rigorous approach to program evaluation. Its evaluation activities are guided, advised, and endorsed by an external, interdisciplinary group, BELL's national Evaluation Advisory Board. The responsibility of the Evaluation Advisory Board is to oversee the application of principles and standards established in the evaluation and assessment fields.

Dr. Duncan Chaplin, Senior Researcher Mathematica Policy Research, Inc. Dr. Leslie Goodyear, Senior Research Associate Educational Development Center, Inc. Dr. Denise Huang, Project Director & Senior Researcher National Center for Research on Evaluation, Standards, and Student Testing (CRESST) Ms. Priscilla Little, Research & Evaluation Consultant Evaluation Consultant Dr. Beth Miller, Director of Research & Evaluation Nellie Mae Education Foundation

TEACHER SURVEYS: BELL developed a teacher and tutor survey instrument to assess program implementation and efficacy from the perspective of instructional staff. Teachers and tutors complete the anonymous survey at the end of the program, which includes questions regarding scholars’ progress, training, curricula, BELL staff and service, and parent engagement.

PARENT SURVEYS: BELL developed a parent survey instrument to assess parent observations of scholar improvement, program quality, and parent satisfaction. Parents complete the anonymous survey on-site during the final week of the program.

93%

92%

90%

92%

92%

93%

0% 25% 50% 75% 100%

Teachers reporting that they would recommend BELL to parents."

Teachers reporting that they found working for BELL personally rewarding."

Teachers reporting that BELL After School developed their professional skills."

Teachers reporting that BELL's team displayed a positive attitude."

Teachers reporting that BELL's team modeled BELL's mission."

Teachers reporting satisfaction with their experience in BELL After School."

Teacher Engagement & Satisfaction