Final Evaluation Report2013

This evaluation was done to ensure accountability for its delivery. Accountability includes meeting program objectives and evaluating the effectiveness of the program. One main purpose of accountability measures is to identify successes and failures of the performance of a program.

Purpose

The findings of this evaluation can be used to: Strengthen relationships; Strengthen the project design and

delivery; Maintain a record of the project’s

progress; and, report on the project’s outcomes to other stakeholders

Evaluation Findings:

16 Ready for School sessions ran from April 2012- August 2013

9 schools (locations) Number of parent/caregivers = 239 Number of children = 217 Number of siblings = 100

Introduction/Statistics:

Data Sources:The data for analysis in this study comes from quantitative and qualitative methods: 202 Pre/Post Evaluation reports (completed by

parents/caregivers who participated in the two-week sessional of the program) 85% Response rate.

16 Parent Focus Group Surveys (average of 12 parents in each group)

10 Partner Surveys 3 Staff Focus Group Survey (19 staff members in total)

Methodology:

Pre/Post Parent Evaluation survey results:

What affect does RfSC activities have on the parents’/caregivers’ perceptions of:

a) their children’s behaviour (level of independence in completing a task) associated with school readiness, and

b) their own knowledge of existing support systems (Canadian School System).

Methodology: Quantitative Data

Pre/Post Parent Evaluation survey (Part 1):

Methodology: Quantitative Data

Separates from a parent without getting upset  

1 2 3 4 5 6 7 8 9 10

Follows simple rules and directions 

1 2 3 4 5 6 7 8 9 10

Seeks out and maintains ongoing social interaction with at least one other child

1 2 3 4 5 6 7 8 9 10

Please circle the child’s current level of independence in completing the following tasks:Evaluation Scale: 1 - Adult does for child ------------------------------- 10 - child does on their own

Importance of parent involvement/ways to get involved in your child’s school

1 2 3 4 5 6 7 8 9 10

Kindergarten Curriculum 1 2 3 4 5 6 7 8 9 10

Getting support for your child so he/she can succeed in school (eg. Special needs, settlement worker, ESL, etc)

1 2 3 4 5 6 7 8 9 10

Methodology: Quantitative Data

Please circle your current level of knowledge on the Canadian School System:Evaluation Scale: 1 – I don’t know anything about this topic ----------- 10 – Very Knowledgeable

Pre/Post Parent Evaluation survey (Part 2):

Evaluation Process: Values were assembled in a data spreadsheet. Comparison was performed to measure the difference

between the pre and post program responses. Simple statistical operations like average and

percentage were used.

For example, for the task: Separates from parent without getting upset if:Pre-evaluation = 5 Post evaluation = 10Difference = 50% improvement in this behaviour

Methodology: Quantitative Data

Qualitative Data Tools:◦Parent/Caregiver Focus Groups (202); ◦Community Partners Questionnaire (10)◦Staff Questionnaire/Focus Group (19)◦Follow-up Phone calls (50)

Methodology: Qualitative Data

Parent Focus Group Questions:◦ 6 questions in the areas of:

Location/hours Program Staff Parent Workshops Children’s program Changes in child’s behaviour over the 2 week session Parent/Caregiver connections

Methodology: Qualitative Data

Community Partners Questionnaire:◦ 6 questions in the area of:

Overall experience with RfSC Communication between RfSC and your agency/school Feedback from parents Challenges experienced as a partner Improvements for next year Dream/vision for RfSC?

Methodology: Qualitative Data

Staff Questionnaire and Focus Group: Questions in the area of:

Suggestions for next year with respect to training Most important thing children learned Suggestions to increase independence, communication

and social skills Most useful part of the program for parents Experience with our community partners Dream for Ready for School Connects

Methodology: Qualitative Data

Evaluation process:

◦ All surveys/questionnaires were analyzed.

◦ Emerging trends were established.◦ Trends were interpreted to measure the

effectiveness of the RfSC program and its activities.

Methodology: Qualitative Data

Quantitative Data: Pre/Post Parent Evaluation Surveys: Impact on Children:

◦ An overall average of 79% of parent/caregivers reported an increase in level of independence after participating in RfSC activities.

◦ Highest increase was 95% which was found at Oakridge.◦ Lowest overall average was 53% increase which was found at

Crescent Town Session 4.◦ Subtask: Separating from Parents - parents reported seeing the most

improvement in their children.

Findings

School: Separating Follows rules and directions

Social Interaction

Oakridge 100% 92% 92%Carlton V. 100% 87% 94%Danforth G. 100% 77% 77%Pauline 75% 81% 81%Sprucecourt 1 90% 50% 50%Sprucecourt 2 100% 84% 92%Secord 1 93% 64% 86%Secord 2 92% 92% 92%

Findings:Percentage of parents reporting increase in level of independence:

Continued next page->

School: Separating Follows rules and directions

Social Interaction

Rose 1 82% 64% 55%Rose 2 70% 70% 60%Crescent T 1 93% 80% 53%Crescent T 2 100% 71% 71%Crescent T 3 86% 72% 64%Crescent T 4 58% 58% 42%George W 1 100% 92% 54%George W 2 100% 71% 71%

Findings: (Con’d) Percentage of parents reporting increase in level of independence:

Findings

Sepa

rates

from Pa

rent

Follo

ws rule

s

Socia

l inter

action

0

10

20

Overall Average % Increase in Level of Independence

Average % Increase23%

16%

11%

Findings

Oakrid

ge D G

Spruc

ecourt

1

Secor

d 1Rose

1CT 1 CT3 GW1

02468

1012

Pre SurveyPost Survey

Findings:Change in level of independence: Separating from Parents

Quantitative Data: Pre/Post Parent Evaluation Surveys: Impact on Parent/caregiver:

◦ 78% of parents/caregivers reported an increase in knowledge in Canadian School System after participating in RfSC workshops. Highest average increase - 96% (Pauline PS) Lowest average increase - 55% (Rose Ave session 1) Subtask: Importance of parent involvement – reported as most

improved – 25%

Findings:

FindingsParents/caregivers reported an increase in their knowledge of the Canadian School System at the end of the two-week RfSC session.

20.522

23.525

Overall Average % Increase in Parent/Caregiver Knowledge of Canadian Ed-

ucation System

Average % Increase25%

24%

22%

Findings

Oakrid

ge CV DGPa

uline

Spruc

ecourt

1

Spruc

ecourt

2

Secor

d 1

Secor

d 2Rose

1Rose

2 CT1 CT2 CT3 CT4GW1GW2

02468

1012

PrePost

FindingsChange in level of knowledge of Importance of Parent Involvement

Overall RfSC Program: Better outreach/promotion plan:

◦ provide the school with a detailed information package about RfSC.

◦ hold an orientation session for all stakeholders (School secretary, Vice-Principal, EY/Family & Parenting centers staff, Daycare staff, etc.) to attend.

Recommendations

Children’s Programming: Staff recommended taking the time to establish rapport

with community partners especially when it involves sharing a common space.

discuss expectations, and possibly draw up a mutually fulfilling contract that all parties can agree upon.

i.e.. Availability of indoor space and outdoor space, room setup, use of equipment/materials, etc. Make sure both parties are communicating and checking in regularly with one another to ensure expectations are being met.

Recommendations:

Parent/Caregiver Workshops Both workshop facilitators and

parents/caregivers have reported that the time allotted for the workshops is not sufficient for the amount of information that needs to be covered.

Presenters should ask participants prior to presenting what they would like to know. By customizing the presentation to the needs and interests of the participants,

allow them plenty of time to ask questions.

Recommendations

Parents prefer presentation style that is interactive, stimulating, lots of visual materials

Standardize presentation workshops to provide all presenters at all locations with an outline/template of an effective structure to delivering an interactive workshop

Recommendations

Program Evaluation improvements: Staff members can discontinue completing

the Pre/post Evaluation form for children due to the difference in perspective being negligible.

Recommendations

Kindergarten Teacher Post-Program Survey: This evaluation tool was introduced this year in

order to provide additional follow up information from the perspective of the children’s kindergarten teacher they were placed in September.

tool was not implemented uniformly across all locations and it was quite challenging to get back completed surveys from the kindergarten teacher.

strategies/methods in obtaining this critical information should be investigated further.

Recommendations

Partner Survey: Tool could undergo a transformation:

◦ Include an area that identifies the respondent’s role (i.e.. Principal, workshop facilitator, OEYC Manager, teacher, etc.)

◦ Should be reviewed from the perspective of different respondents to see which questions are applicable to which role.

Recommendations (Lastly…)

◦Data analyzed was from 44 families (four school locations: Danforth Gardens, Oakridge, Carlton Village, and Pauline)

◦78% response rate◦92% of parents reported having no concerns

with their child’s school, teacher or classmates.◦The most important things families learned:

Canadian School System School Expectations and Routines Nutrition Dealing with Separation Behaviour Management

Follow – up check-ins with Families:

◦70 % of parents accessed recommended services or resources.

◦66% of families visited their local library.◦68% of families were connected to either local

recreational programs at community centers or participated in OEYC.

◦52% of parents still see/interact with other families from the program.

Follow – up findings: Cont’d

Utilize standardized questions that are asked at all locations. Ensure the same questions are being asked at every location.

Ensure questions are being responded to correctly and completely. When asking parents the questions, read each question carefully, record the response, then reread the question to make the question was responded to correctly. i.e.. How is your child adjusting? Were you able to access any of the services or resources that were recommended to you this summer? Which ones?

Explore ways to increase family participation in accessing resources that were recommended to them in the summer program.

Recommendations for future follow-up work: