National Mentoring Summit

January 2013

Agenda

• Mentor Video• Program Description • Research Overview• Key Findings • Conclusions and Continual Improvement• Lessons Learned: Mentoring for Academic Gains• Action Planning

Video: Be a Mentor

• http://www.youtube.com/watch?v=YyK_03DccSY

Program Model

• Afterschool, Summer, High School Placement

• 650 hours/year, + 1000 hours in school• Weekly academic mentoring• Accelerated lessons, college trips,

career exposure• Positive culture and youth

development approach

RCT Study Design$4 million over 7 years

Four-year results published in 2013

By MDRC

Two-year results and Summer

Snapshot published in 2011

by P/PV

Researchers test & survey:

4 summers and 3 school years + one 9th grade

follow up

50% accepted through lottery

50% assigned to control group

Over 3 years (2006, 2007,

2008),951 students &

families apply and interview for 3

cohorts

Only 2% of nonprofits have this “gold-standard” randomized research

Randomized Control Trial (RCT) Study: funded by Wallace, WT Grant, Smith Richardson, Spencer, Atlantic Philanthropies, Bank of America

Randomization – in Brief

1. Hear about HA at school, neighborhood, etc.

2. Interview99% - enter lottery.

3. Lottery50% enterprogram:“treatment”

“Treatment”Access to full program – may never attend.

“Control”Never able to access program.But given list of other after-school & summer options

Annual testing ($120 payment), surveys,

Follow-up from researchers

Researchers

• Dr. Jean Grossman– Princeton University, P/PV, MDRC

• Dr. Leigh Linden– Columbia University, University of

Texas- Austin• Dr. Carla Herrera– P/PV

Outcome Measures Collected• Hypothesis:

- With high dosage and structure, Higher Achievement would have a measurable impact on academic outcomes and high school placement, by first impacting attitudes and behaviors.

• 2 key outcomes:– Standardized Test Scores– Application, Acceptance to, and Matriculation at Competitive High Schools

Other outcomes of interest:– Behavior– Academic Attitudes– Perceptions of Peer and Adult Support– Participation in HA and Other OST Programs– Engagement in Academic Activities and High School-Related Activities

KEY FINDINGS

“Higher Achievement’s intensive year-round program had a

significant impact on youth’s standardized reading and math test

scores.”

“The longstanding Higher Achievement model is making a

difference in the academic lives of motivated, at-risk students who

could easily fall through the cracks.”

Two-year follow-up study

No summer learning loss.

Also - no impact on test scores, compared to control group,

over the course of one summer: 2010

High levels of attendance and retention, during tricky middle

school years: 97% of summer participants also in

after-school

Summer 2010 Snapshot

Four-year follow up study

Math gains persist at significantly higher levels than

the control group. Reading gains level with control group.

Significant impact on certain types of family engagement in

education

Significant impact on placement in competitive

private schools and avoiding low-performing high schools

Academic Impacts

First-Year FU

Second-Year FU

Fourth-Year FU

Math problem-solving 0.03 0.10* 0.11*

Reading comprehension 0.02 0.08† 0.04

* = Statistical significance.

Academic and Enrichment Activities

ActivityFirst-Year FU Second-Year FU Fourth-Year FU

Treatmt (%)

T-C Diff (%)

Treatmt (%)

T-C Diff (%)

Treatmt (%)

T-C Diff (%)

Community service 53 0 60 4 74 4

Presented ideas to a group out of school

63 6† 64 9* 69 11*

Visited a college 72 28** 73 28** 78 25**

Read books out of school 74 3 79 4 85 8†

Writing out of school 73 7* 74 3 82 14**

Visited a business 56 4 61 8* 70 14**

Events with OST 77 10** 80 6† 84 9*

Academic contests at OST 68 13** 68 11** 71 16**

Conclusions

• Year-round, multi-year:high dosage yields results

• No effect after 1 year, only after 2 years:– Academic results take time

• Reading gains level with control group in 4-year follow-up:– Control and treatment made gains.

Reading can be more self-directed. Math requires more instruction.

Continual Improvement• Common Core Standards Alignment:– Curricula and Technology

• Explicitly teach writing skills• Staff training to improve inference skills

for reading comprehension• “How Children Succeed” &

Social/Emotional Skills– Piloted three assessments:

PEAR HSA, SAYO, Gallup Student Poll + Grit Scale

Annual Operating Plan and Individual Work Plans, tied to Strategic Plan.

Monthly Dashboard and Site Observation Discussions:

National & Local StaffSchool Partners

Quarterly Dashboard, linked to Financial Model

Mid-Year Retreats: Org & City course

correctionsUpdate individual work

plans

Annual Retreat: Outcomes and RCT Results

Continuous ImprovementCycle

Maximizing Mentors

• Orientation• Ongoing mini-trainings• Scripted lessons• Nightly session feedback• Monthly observations

One Mentor’s Perspective

• Ms. Liesa– Her involvement, academic gains of scholars– Mentor training and ongoing support– Best practices in academic mentoring

Table Activity– Think, Pair, Share!

• What are the goals for your mentoring program? • What challenges do you have to meeting your goals?• What resources do you have to meet your goals? What

resources do you have to overcome your challenges?• What support and/or trainings do you have to put in

place to maximize your resources and meet the goals of your mentoring program?

ACTION PLANNING

"Whatever affects one directly, affects all indirectly. I can never be what I ought to be until you are what you ought to be.”- Dr. Martin Luther King